18 January 2012

Global Health Numbers Tempting Towards Shore

Numbers are dirty liars. They are like the sirens singing to tempt Odysseus to steer his boat towards the dangerous rocks. Unfortunately, we are not prepared enough to either employ beeswax or tie ourselves to our boat's mast. So, they sing away; making us feel comfortable to believe them when confirming our long held beliefs.

Numbers are most dangerous in the aid and development world when championing great achievements. The most recent example is that of health gains in Afghanistan. November's USAID-funded Afghanistan Mortality Survey appears to be the most recent example numbers coming into question.

Comparing 2004 to 2010: life expectancy rose from 42 years to 62 years; child survival until the age of five fell from 25 percent to 10 percent; and the maternal mortality rate fell from 1,600 to 327 deaths for every 100,000 deliveries.

Take a moment and compare those numbers again. People were living 20 years longer and maternal mortality fell by 75%. Those are staggering returns.

NPR reports that the November release of the report was due to further review after the big numbers were gathered.
"Conditions for data collection are desperately difficult," he says. "Knocking on someone's door and asking them details about their children and their household is not something most people would want to do."

Three of the six experts on the survey's technical advisory group told NPR they still have doubts, even after the extra analysis. That includes Julia Hussein, of Aberdeen University.

"You've got to match what you know in terms of evidence with what you see with your eyes," says Hussein, who has worked on maternal mortality in Afghanistan since the 1990s.

"My instinctive reaction to figures reported in the survey — I just find them unbelievable, knowing what sort of care is available in Afghanistan," she says.

It is possible that some parts of the data in the USAID study are correct. The point is not to question the validity of the assessment (I am unqualified to do that anyways), rather it is to drive home a healthy skepticism about big data announcements. In collecting news stories each evening with a global health bend, the number of reports using new data points and studies are staggering. The challenge is how to be able to engage so as not to crash the boat or be forced to tie yourself to the mast.