09 May 2012

MVP Claims Child Mortality Reduction Victory; Critics Question Study

A study published in The Lancet says the Millennium Village Project (MVP) is responsible for a decline in child mortality three times faster than comparable villages that do not receive the MVP's suite of services. The authors, including MVP founder Jeffrey Sachs, point to the results of the success of the MVPs multi-pronged approach.

The basic premise is that spending is low when examined on the individual level. Additionally, a comprehensive approach to development that addresses an individual ranging from health to schooling to agriculture will lead to overall positive gains. A modest increase in spending through foreign aid and national budgets could take on issues such as infant mortality. The MVP represents a pilot program to show if and how the approach can work.

According to the researchers, the under-five death rates in control villages declined an average of 2.6% each year over the course of a decade. The MVs averaged of 7.8% rate of decline per year.

Jeffrey Sachs, Sonia Sachs and Prabhjot Sing, all authors on the study, write in the Huffington Post about the successes found in the study. They point to five areas where the MV health systems are succeeding:
  1. Community Health Workers. The Community Health Workers (CHWs) are the cornerstone of the community-based delivery system in the MVs... 
  2. Procedures and Decision Support: The simple idea of a checklist has been pioneered in operating rooms and hospitals throughout the world. It is equally critical for community health delivery, and can make life-and-death events like childbirths much less vulnerable to human error... 
  3. Mobile phones for health management: Mobile phones are revolutionizing how poor, rural communities can escape from poverty...
  4. Low-cost devices for disease detection and management: ...Many conditions are life threatening but preventable and treatable. The Millennium Villages has applied the pioneering work of innovators to rethink in fundamental ways the design of low-cost health systems.
  5. Verbal Autopsies and Management Responses: ...By carefully tracking the likely cause of deaths on every occasion, health systems can learn about their own weaknesses, and take corrective actions. The Millennium Villages have therefore implemented a "verbal autopsy" system to try to account for each death of mothers and children, and to use this information to strengthen the health system.
Data published in The Lancet study show promise, but raise further questions. An article in Nature looks into some of these concerns.
[CGD's Michael] Clemens says that these headline figures are misleading for a number of reasons. He points out that the control-village data include retrospectively estimated figures that are probably too high. And nationwide improvements in child mortality over the three years of the study were almost as good as in the Millennium Villages, he says, so it is unfair to compare the project’s success with a more gradual decadal trend. Furthermore, deriving trends from children monitored in a few villages for just three years introduces significant statistical uncertainty, he argues. 
Using figures in the paper, Clemens calculates that the study authors can be confident only that the annual rate of decline for child mortality in the Millennium Villages lies between 1.4% and 14.3%. “If you claim to triple rates of decline you must have the evidence to back this up,” he says.
Further complicating the issue is that the evaluation is done by the people who implement the MVP. Clemens and others have criticized this decision in previous studies. For critics, a new MVP established in Ghana will be evaluated by DfID. Though it is likely all parties will not be satisfied after the independent evaluation, an outside perspective on the project in cooperation with the MVP will quiet some of the criticisms.

Lee Crawfurd further points out the observed differences in 10 of the 18 indicators in the study are statistically insignificant. For example, the increase in the asset-based wealth index for households are nearly identical between the MVP and comparison villages.

The study does not attempt to establish any direct causality for the reduction in infant mortality to any individual aspects of the program.
As a complex intervention operating across many sectors, definitive statements about the specific mechanisms of mortality reductions are not possible. However, the project placed a strong initial health sector emphasis on so-called quick wins including optimisation of immunisation coverage and bednet distribution to all sleeping sites—with concurrent reductions in malaria parasitaemia. Early efforts to enhance health staffing and facility infrastructure, reduce access barriers such as user-fees, and increase cross-sectoral investments to improve roads, emergency transport, and mobile communication played potentially important parts in improving access to skilled birth attendance. 
This all not to suggest that the program was not successful. We have to take the gains at face value. The question is what the findings tell us about expanding the MVP. If there is little change in some areas while rapid change it others, is it possible that some aspects of the MVP are failing? Or, is it possible that some interventions are individually successful.

Given the structure of the MVP and the evaluations conducted, it is hard to tease out causality. But that is not what was intended when establishing the MVP. The theory is that they suite of interventions will make the difference rather than a few targeted ones. How can this be tested? Maybe an evaluation that looks at a host of random villages receiving some targeted interventions and others that get nothing could help to provide a comparison and further understanding as to what does and does not work.

By touting the success of the MVP, the authors are making the link between the program as a whole and the successful decline in infant mortality. The paper says the average cost per person per year is $116, of that $25 is attributed to health costs. If the idea is to increase per person spending to the same levels across SSA, the final bill would be $92.8 billion per year (800 million living in SSA). In 2009, $25 billion of official development assistance (ODA) went to sub-Saharan Africa. If countries need half of the cost to be covered by foreign government and investments the ODA to SSA would need to double.

If the health cost of $25 per person per year is the main target than it is all the more attainable to meet the individual spending needs through ODA and local government spending. The findings coupled with the outside concerns seem to point towards a positive direction that needs a bit further examination. In sum, the accomplishment of reducing infant mortality should be celebrated. Ensuring the survival of children is an important goal.

Additional reading: Matt Collin compares the results of the study to a recently publish study in The Lancet that shows an overall decline in infant mortality at the country level (see chart below): http://aidthoughts.org/?p=3330