18 October 2011

The Importance of Learning From Failure

This is an entry in Tales From the Hood's aid blog forum on failure.  Be sure to go here to see all the other well thought out posts.  Definitely read the ones from Marc, Terrence, Ian and Shotgun Shack.

Failure is becoming more popular in the humanitarian world.  That is evidenced by last week's #FailFaireDC and Engineers without Borders's conference on failure earlier in the year.   Why is it important?

Visually, it brings us from here:

#FailFaireDC 2011 filled the World Bank auditorium

To here:

  #FailFaireDC afterparty: @tkb @LCMoy @kjpeterson @jerotus @viewfromthecave @jacquideelstra @meowtree @wayan_vota 

 To here:

OK, that is way too simple, but the point is that it can bring us from talking about it to workable solutions. A recent example of admitting failure comes from the report And Who Listens to the Poor on two pilots implementing BRAC's social safety net program called 'Targeting the Ultra Poor' (TUP) by Trickle Up in West Bengal, India and Organi Charitable Trust Sindh, Pakistan, along with 7 other NGOs. The report shows is that implementation is not so simple.  Trickle Up's West Bengal program proved to be a resounding success, but the Organi Sindh pilot failed. 
In the Sindh pilot, there was little transformation among participants and their households. Their household resource bases remained weak, with limited improvement in livelihoods, persisting constraints due to purdah and poor access to health care, and with no group formation or village assistance committees, their social resources continued to be based upon pre-existing kinship ties. Their material resources, however, slightly improved with greater financial capital through savings and less exploitation by intermediaries. It can be hoped that, with greater access to education of TUP children, resource bases for the next generation will strengthen.

We saw that both programmes often reinforced virtuous and vicious circles as they initially manifested in participants‟ lives. There were programmatic limitations, but despite these, both programmes were able to create trajectories out of extreme poverty for participants who had favourable starting points. OCT's programme, however, proved to be ineffective for the most vulnerable women – those who had feeble starting positions, faced constraints outside the remit of the programme, and lacked the agency to bypass programme flaws.
They continue in their conclusions to further identify why the Singh program failed and recommends what could be done to improve (emphasis added).
Such lessons enable us to understand the pivotal role that organisations play in getting all members to graduate, and not just a favoured few. Organisations seeking to implement similar programmes should be careful that they are defining needs and then devising objectives; defining eligibility and identifying the eligible; designing and delivering effectively; putting in place accountability mechanisms; and feeding ground realities back into programme design.

The cornerstone to successful implementation is defining needs before setting objectives. Conducting needs assessments prior to implementation would have revealed to OCT that the targeted communities preferred health support over assets. Trickle Up would have realised from the outset that participants required livelihood options beyond livestock. Such insights might have drastically altered the inputs provided, and enabled the organisations to respond more accurately to the constraints the communities were facing. A TUP member in Sindh captured this aptly: "How can they help us if they don't ask us what we want? Had they asked, we would've said health care, drinking water, better jobs for our husbands. But who asks the poor? Who listens to the poor?"


Amiss in OCT's programme design was an analysis that incorporated different strands of knowledge from local communities, staff, and livelihood experts where relevant. If participants are to succeed, they must also be given adequate services that complement the asset. These include veterinary access, intensive training and monitoring, a stipend to smooth consumption when assets are not generating income, and close mentoring support from staff.

Also, a feedback loop within the organisation is essential for effective delivery. During any pilot programme, a number of unexpected issues arise which call for modification of inputs and alterations in strategy. This is especially true for programmes that target extremely poor people, who may face unanticipated constraints. As field staff are most privy to ground realities, their experiences should be harnessed by programme management in order to respond and modify inputs as required. Accountability measures, such as incentives and repercussions, would help both sets of actors to perform these roles more optimally and transparently.

Lastly, and most importantly, it is worthwhile to identify the less dynamic participants early on, and to provide them the lion's share of mentoring support. We have seen that more dynamic participants have the resources to do well in this programme from the outset.
Organi Charitable Trust admits to making a series of mistakes.  Their recommendations are not new, but they will help to foster the design of future programs.  Looking at the implementation of the BRAC program, it would be easy to shrug off the failure in Sindh as a fluke and focused on the successes in West Bengal.  BRAC could decide to continue with one organization over the other.  That would miss the point.  By pairing the success of Trickle Up with the failure of Organi, the future impementors of TUP, including both organizations, avoid the mistakes that were made and make it a more effective program when introduced into new communities.


The challenge of admitting failure is twofold.  First it needs participation.  Secondly, it needs to lead to meaningful action.  Terrence Wood explains the first challenge in his post saying,
I’ve always thought that admitting you’d stuffed up in development work cut the other way: if everyone did it would be easy enough to do but if you’re the first NGO trying to do it you’ll find yourself at the sharp end of a ‘first penguin to leap off the ice sheet’ type collective action dilemma (i.e. it’s the first penguin that has the highest chance of getting chomped by the sea lions). Who’s going to keep giving money to the one NGO that’s forever feeding journalists with stories of what it did wrong.
Getting organizations to own up to failures will be really hard.  I think that it will have to start with some of the big guys to really get the ball rolling.  This is a place where the World Bank could take the lead as its Development Impact blog has been a place that openly discusses some program failures.

The second step is probably more important.  My optimism in the report is that BRAC and its partner implementors actually heed their own advice.  However, with few mechanisms to keep their feet to the fire, it is possible that they might not do anything.  I am optimistic, but there still need to be accountability measures to ensure that failures are not simply admitted but are learning opportunities.  Admitting from failure is nice, but learning should be one of the most important words in this discussion.

Photo Credit: 1 & 2 to Invenneo, 3 myself