Millennium Villages Project continues to systematically overstate its effects

by Michael Clemens and Gabriel Demombynes

The Millennium Villages Project (MVP) is an experimental anti-poverty interventionin villages across Africa. In October, we released evidence that the Project’s official publications were overstating its real effects, and we offered suggestions on improving its impact evaluation. On Tuesday the MVP, whose leadership and staff are aware of our work, continued to greatly overstate its impact.

It started last year. In a June 2010 report called Harvests of Development, the Project claimed that the impacts of the project included expanded cell phone ownership.  For example, the MVP claimed that increases in cell phone ownership at the Ghana site were caused by the project, in this extract from page 91 of the MVP report:

This claim has little basis, because cell phone ownership has been expanding at about the same rate all around the MVP site in areas untouched by the project. The graph below, from our paper, shows cell phone ownership at the MVP site in black compared with various other areas:

No reasonable person could look at these findings and conclude that the MVP intervention clearly caused any substantial increase in the rate of expansion of cell phone ownership. The evidence gives no clear reason to believe that cell phone ownership would have expanded any more slowly at the intervention sites if the Project had never existed.

But on Tuesday, months after multiple discussions we’ve had with MVP leaders on our research, a post on the MVP’s blog restated the claim that the increase in mobile phone ownership at the intervention sites was caused by the Project, calling the increase at the Sauri, Kenya site one of the MVP’s “achievements.”

As the World Bank Chief Economist for Africa Shanta Devarajan has observed, our evidence does suggest that the MVP has had some positive short-run impacts on people’s lives. So there is no reason to overstate the impacts.

Before-vs.-after comparisons implicitly assume that in the absence of the intervention, nothing would have changed. This assumption is demonstrably incorrect in the case of the MVP and cell phone ownership. The cell phones are just one example of how the Project overstates its impacts; in the paper we discuss others. For a brief summary and a discussion of my visit to the Sauri MVP site, see this earlier post

http://blogs.worldbank.org/africacan/millennium-villages-project-continues-to-systematically-overstate-its-effects

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: