21 May 2012

In Which UNICEF Discovers RCTs...and People React

An article in Financial Times interviewing the head of UNICEF, Anthony Lake, is meant to be cutting edge, but it shows that Lake is just a tad behind on the times.
Here Lake seems to be with the new “randomistas”, who say that to find what works in aid, you need to apply the sort of randomised control trials used in medicine. For instance: how to persuade teachers in rural India to turn up at school? Esther Duflo, the development economist, arranged for classrooms to have free cameras. Pay depended on how often teachers could show end-of-day photographs with their classes. Suddenly, more teachers showed up. 
“What works” can be simple and cheap, says Lake. “A great contribution to the child survival revolution were things like oral rehydration salts. Jim Grant, my predecessor, used to walk around with a package of salts in his pocket to show how simple this was.” The sachets, which cost just cents each, have saved millions of children with diarrhoea from dying. On a macro level, too, Unicef has been measuring what works. Consequently, it’s now targeting the poorest families and countries. Lake explains: “By definition, the same immunisation programme where there’s lots of disease will save more children’s lives than where there’s less disease.” Anyway, he adds, focusing on the poorest is “right”. 
But as we talk on about aid, he throws in a corrective: “All those who work in the international community on development tend to overstate the impact of what we’re doing. What’s far more important is the performance of governments.” Governments and markets, not aid agencies, have lifted hundreds of millions out of poverty since 2000. And in some countries that remain poor, governments have improved life nonetheless. When I ask for an example, Lake cites Bangladesh. Despite its “vigorous” politics, he says, rival parties usually come together on development. Bangladesh has dealt so well with problems such as open defecation – “which you never talk about in polite company, but which kills lots of children” – that it is now moving on to even tougher issues. Lake marvels: “The biggest killer of very young children in Bangladesh is now drowning.” Unicef is teaching them to swim.
The article concludes with Lake taking about UNICEF changing its communications policies away from poverty porn. "Unicef now uses pictures of recovering children. Thanks to smart aid, more and more do recover," concludes author Simon Kuper. As a bit of a self-proclaimed smart aider, I am not so sure what that actually means. As far as I can tell, smart aid equals RCTs and shunning poverty porn.

The strangest line comes earlier in the story that makes it sound like RCTs are earth-shattering science that are ushering in evaluations for the very first time. "Finally, academics, donors and even some aid agencies have begun measuring what works. Very slowly, development is becoming a science," says Kuper.
As Bill Easterly put it:

Though the RCT as a tool for evaluating aid interventions has been a welcomed step, it is not the be-all-end-all that the article suggests. To some extent they show what works, but have trouble telling why. Finding what works is important to achieving long term development, but that movement did not reach its peak in 2010 or with the RCT.

One lost aspect of the conversation is that the head of a large organization like UNICEF discussing this in a public manner is a good thing. It allows for external accountability for those who can reach out to Lake and hopefully offer some suggestions on how to do better evaluations that will start to uncover the 'why' and 'how' questions rather than just 'what.'

Oh, and then you also have the kicker of this image in the article. Bravo FT, for reducing Africans into specimens and using caricature.

HT Bill Easterly, Shotgun Shack, Ed Carr and Tales From the Hood....check out their tweets for further commentary.