The Millennium Villages Project (MVP) is an experimental anti-poverty interventionin villages across Africa. In October, we released evidence that the Project’s official publications were overstating its real effects, and we offered suggestions on improving its impact evaluation. On Tuesday the MVP, whose leadership and staff are aware of our work, continued to greatly overstate its impact.

It started last year. In a June 2010 report called Harvests of Development, the Project claimed that the impacts of the project included expanded cell phone ownership.  For example, the MVP claimed that increases in cell phone ownership at the Ghana site were caused by the project, in this extract from page 91 of the MVP report:

This claim has little basis, because cell phone ownership has been expanding at about the same rate all around the MVP site in areas untouched by the project. The graph below, from our paper, shows cell phone ownership at the MVP site in black compared with various other areas:

No reasonable person could look at these findings and conclude that the MVP intervention clearly caused any substantial increase in the rate of expansion of cell phone ownership. The evidence gives no clear reason to believe that cell phone ownership would have expanded any more slowly at the intervention sites if the Project had never existed.

But on Tuesday, months after multiple discussions we’ve had with MVP leaders on our research, a post on the MVP’s blog restated the claim that the increase in mobile phone ownership at the intervention sites was caused by the Project, calling the increase at the Sauri, Kenya site one of the MVP’s “achievements.”

As the World Bank Chief Economist for Africa Shanta Devarajan has observed, our evidence does suggest that the MVP has had some positive short-run impacts on people’s lives. So there is no reason to overstate the impacts.

Before-vs.-after comparisons implicitly assume that in the absence of the intervention, nothing would have changed. This assumption is demonstrably incorrect in the case of the MVP and cell phone ownership. The cell phones are just one example of how the Project overstates its impacts; in the paper we discuss others. For a brief summary and a discussion of my visit to the Sauri MVP site, see this earlier post

http://blogs.worldbank.org/africacan/millennium-villages-project-continues-to-systematically-overstate-its-effects

Published by Inter-Parliamentary Union.

http://www.odi.org.uk/resources/details.asp?id=4799&title=making-aid-work-practical-guidance-parliamentarians

I wanted to bring your attention to the 5-day Executive Education course from MIT’s Jameel Poverty Action Lab (J-PAL), “Evaluating Social Programs”. Courses will be held in Cambridge, MA (USA); Rabat, Morocco; Bogota, Colombia; and Chennai, India.

Information on content, dates, and the application process can be found here:
http://www.povertyactionlab.com/course/

Marc Shotland
Director of Training
Abdul Latif Jameel Poverty Action Lab
Massachusetts Institute of Technology
shotland@mit.edu

This conference focuses on how evaluative practice can be improved, given the need to view much of development as a process of social transformation and, therefore, complex. Current evaluation practice has not yet embraced the full implications of assessing `the complex’ and existing approaches often fall woefully short. During the conference, participants can explore concrete evaluation practices that reconcile an understanding of complex societal change processes with quality standards, including rigorous, ethical concerns, appropriateness and feasibility.

Effective development is being understood in many quarters as revolving around people-centeredness and institutional transformation, thereby inherently complex, i.e. non-linear, emergent, unpredictable. However, discussions on measuring development effectiveness have zoomed in on methodologies suited to measure simpler, more linear interventions. The risk is that development, in order to facilitate its measurement, is reduced to simple interventions. These two mutually influential trends sit side-by-side in increasing discomfort for those who understand development as societal transformation.

The Conference aims to contribute to clarity in the development sector about what constitutes robustness, i.e. core values and quality standard, for evaluative practice in development that recognises the complexity of societal transformation.

You can find the flyer and registration form at: http://tinyurl.com/evaluation-revisited.

The Organising Committee:
Jan Brouwers (Context, international cooperation)
Irene Guijt (Learning by Design)
Cecile Kusters (Wageningen UR Centre for Development Innovation)

Social Audit is a way of planning and measuring organisational performance against social, environmental and commercial objectives. A Social Audit verifies an organisation in terms of its governance procedures, social and environmental responsibilities and its equitable structure, its internal and external stakeholder responses to operations, and then it accounts to stakeholders. It is particularly useful for Social Enterprises, NGOs, and not for profit organisations. The course is aimed at managers, staff and board members who are actively involved in organisational performance measurement and democratic planning.

Delivered by Freer Spreckley on behalf of Local Livelihoods at live@locallivelihoods.com

Info: http://www.uk.locallivelihoods.com/Moduls/WebSite/Page/Default.aspx?Pag_Id=168

http://www.oecd.org/pages/0,3417,en_40033426_40033828_1_1_1_1_1,00.html

The Global Project is hosted by the OECD and run in collaboration with other international and regional Partners. A representative from each Partner organisation sits on the Global Project Board. The Board has two Co-Chairs: the Chief Statistician of the OECD (ex-officio) and another member of the Global Project Board, elected every two years. The two co-chairs are currently Mr. Enrico Giovannini (OECD) and Mr. Pedro Conceicao (UNDP). You can read more about the structure of the Global Project here.

Organisations can become involved with the Global Project in a variety of ways.  In addition to the Partners, the other affiliates are Associates, Sponsors, Technical Advisors and Correspondents. The Global Project works closely with these organisations on planning events, training courses, a work programme, research projects and more.

Partners

Major International/supranational Organisations that play a key role in the overall Global Project, investing substantial resources – financial or in-kind – over several years and assuming responsibility for the management of the Global Project and/or for specific tasks.

WHAT we are doing

Send Print

Also available:

[ Aid effectiveness ]

Tackling the roots of poverty

In the first ever “European Report on Development” (ERD), independent researchers assess ways to overcome fragile statehood in sub-Saharan Africa. Dysfunctional institutions, protracted crises and conflicts are important obstacles to fighting poverty and fostering sustainable development. This is a complex issue, aid critics often tend to neglect.

[ By Françoise Moreau ]

In the December issue of D+C, Volker Seitz questioned the effectiveness of German and European aid to Africa, suggesting that development assistance has “brought about precious little change to African misery in the past 50 years”.

The picture is not quite as bleak as Mr Seitz paints it. Despite the turmoil engulfing many parts of sub-Saharan Africa, successes have been registered. For example, many sub-Saharan African states have, despite setting of from a lower starting point than other developing countries, made good progress towards some of the Millennium Development Goals (MDGs), such as improving their primary school enrolment and reducing their child mortality rates. Seitz argues that “the situation in sub-Saharan Africa will not improve as long as Africans do not take care of their own future”. But this is really a case of throwing out the baby with the bathwater. While it is undoubtedly the case that “only the African people can help themselves”, Europe and other donors can and do play a crucial supporting role.

In fact, insistance on “local ownership” for the development process has been a guiding principle of EU co-operation policy for many years, and partner countries and local stakeholders do play an active and leading part in setting their own priorities. Senegal, Ghana and Mozambique – to name examples from Francophone, Anglophone and Lusophone Africa – have a recent history of drafting and implementing policies in close cooperation with several donors. On the basis of these countries’ policies, donors have been able to draft joint assistance strategies and pool funds they then grant the governments concerned in the form of budget support. These countries have made visible progress, and there can be no doubt that aid has helped them do so.

However, in so-called “fragile states” that are marked by conflict and violence, the call for local ownership leads to a plethora of challenges. In these countries, the state is typically dysfunctional, and government agencies are either unable or sometimes unwilling to take up their responsibilities vis-à-vis its citizens. In sub-Saharan Africa, there are around 30 fragile states, according to international organisations, and this fragility is hindering development.

The toughest challenges

Although only about 15% of the developing world’s population live in fragile countries, they account for a third of the world’s poor and half of the children dying before the age of five. That is why the first-ever European Report on Development (ERD), which was recently released in Stockholm and Brussels, is dedicated to the issue of fragile statehood in Africa and how to build resilience.

The causes of fragility are as complex and diverse as the countries suffering from it. Many factors lie at the roots of fragility, including historical legacies, such as colonialism, conflicts, ethnic tensions, poor governance, weak institutions, the clamouring for natural resources, small economies, and much more.

Although many roads can lead a society into fragility, the path to resilience is long and difficult. This is illustrated notably by the fact that 35 countries defined as ‘fragile’ by the World Bank in 1979 are still fragile in 2009, three decades later.

So what can be done to move countries away from fragility? The ERD does not offer any hard and fast prescriptions. This is partly because each country suffering from fragility is unique in terms of its socio-economic reality and the underlying factors behind its fragility. Rather the ERD provides a framework for the EU to think about and rethink its involvement in sub-Saharan fragile states.

For instance, both Angola and Zimbabwe are in the OECD’s 2009 list of countries in situations of fragility, but they are very different. Despite being formally at peace since 2002 and having experienced a number of years of double-digit economic growth fuelled by the rise in commodity prices, Angola still has a low literacy and a high infant mortality rate. In contrast, Zimbabwe has seen its economy shrink, yet Zimbabweans are among the most literate in sub-Saharan Africa and the infant mortality rate is still relatively low.

These examples show that there can be no model ‘one-size-fits-all’ approach to dealing with them. As the ERD emphasises, the EU should draft “general policies to address specific issues and adapt them to individual contexts”.

It should be noted that the EU is not unprepared when it comes to dealing with fragility. The issue has been on the Union’s radar for some years now. For example, in 2001, fragile states were declared a priority by Belgium’s EU presidency. In 2005, the European Consensus on Development identified “state fragility” as one of the key challenges of development policy. The Consensus outlined an EU approach based on governance reforms, establishing the rule of law, combating corruption, building viable state institutions and boosting state capacity. The ERD builds on this bedrock.

Engagenment at many levels

The report finds that the EU is well placed to make a difference when it comes to helping its fragile African partners build up their resilience. This is because fragility requires a broad response that taps a wide range of policies and the Union is a political actor which deals with the whole gamut of policy areas – from security to trade, agriculture, climate and energy policies – and not only those directly related to a narrow understanding of “development”. Another forte of the EU is that it is not only engaged in cooperation with governments. Rather the EU cooperates with various other actors, ranging from grassroots communities to regional organisations that span several countries. The recently established partnership between the African Union and the EU is a good example of this.

While immediate problems and crises should not be overlooked, efforts and activities must have a long-term strategic focus because the persistent challenges facing fragile states are mainly structural, and hence require a stable and sustained commitment.

In addition to this long-termism, the ERD identifies four other key priority areas for EU engagement:
– enhancing human and social capital;
– supporting state-building and social cohesion;
– supporting better governance at a regional level, including regional integration; and
– strengthening security.

To avoid a culture of dependence and to promote true sustainability and accountability, underutilised domestic resources need to be harnessed and expanded by building up a domestic tax base and developing robust local financial institutions. Social cohesion and a state’s legitimacy and accountability are key objectives here. Making better use of local knowledge, through formal and informal institutions, including existing conflict prevention mechanisms, must be part of the process.

Ultimately, while the EU can do much to promote development in fragile states and more generally to strengthen developing countries’ resilience, it is important to recognise that its role is essentially a supportive and facilitating one, and that truly sustainable development begins and ends at home.