The challenge of Doing Development Differently

I attended the first Doing Development Differently (DDD) workshop organised by the BSC gang at Harvard CID and ODI; read more about the workshop here. See Day 1 summary; and Day 2 summary. Some thoughts, over time:

  1. DDD is the big picture: DDD is about the details and and the beauty of innovation and creativity on the ground. But more importantly, DDD is about the big picture. As the workshop signalled (at least) to me, the battleground for the DDD conspirators/crusaders is the top table, with donors and policymakers; the moneybags, decision-makers and influencers. Expressed in an extremely cliched way, the goal ought to be to facilitate d on the ground by changing the rules of the D game. This makes sense to me. Gathering and influencing activists and local champions is a necessary, but not sufficient condition for real change. Unless those determining the overarching policy environment for localised development work are willing to change their fixed ways, there can can be little progress. At the same time, this workshop definitely missed a trick by not having participants from governments (I am sure they considered this long and hard), which in many middle income countries have come to be all of the above – the moneybags, policy/decision-maker,etc. This is also the way DDD will be able to go beyond aid.
  2. DDD is not new: Much of the conversation was about familiar themes: participation, logframes, being locally embedded, stakeholder analyses, decentralisation, etc; neither are the champions (think-tanks, academics, NGOs and donors) a new species in development. But that shouldn’t be a dampener – we can never have too many good ideas. What will be interesting to follow is the contours of the coalitions DDD is able to draw up; how members find support with and from each other; and eventually, how this shapes the D game. What is new for these actors possibly is some of the language – which references the strategies used by private sector players.   
  3. But, DDD is damn hard and risky: There are plenty of enemies out there – the rigid project structures, the business cases, the impatient donor desk officer, the weary community (of “beneficiaries” and on-lookers), etc. Attempting DDD comes with its risks and in most jobs in the development world (with the possible exception of research-y ones), it is unclear if the rewards at the end of the tunnel outweigh the numerous manholes on the way. So why would anyone try out DDD? And if they did, how will they muster up the personal/institutional capital to do so? For how long can one fight the establishment?
  4. Therefore, be smart:Thinking and Working Politically; Politically Smart, Locally Led – all of these arguably are only a difference of semantics compared to DDD. Creating space within the aid establishment requires forming alliances. Be it the UN, World Bank, DFID or the Government of Bihar (once upon a time), I have always held that there are some fantastically creative bureaucrats that thrive within these systems who either possess (or are backed by) powers that can create space for DDD. Joel Hellman, from the World Bank talked about the classical “project” and agreed that it is at the root of many problems, but also encouraged us to think of ways in which one can find space to manoeuvre within the system – such as custom-made financing facilities, perhaps. Some of the younger participants (and I will include myself in this group) found it difficult at times to agree that the “project” is the problem – so thoroughly trained we all are in the mainstream development model. For us then, the idea of finding room for manoeuvre sounds that much more attractive, although that might mean that the revolution will be delayed.
  5. PDIA is not a model (I checked with Matt!) – rather, PDIA is one of the ways of DDD and falls within that tent. I wondered about the abuse of the term PDIA, i.e, if PDIA gets ‘mainstreamed’, everyone and their nephew will produce case studies that mimic the essential ingredients of PDIA. This has happened with ‘participation’ – at one point, everything everywhere was participatory and local. Many a stick was handed over just in time for a telling picture or a stirring case study. Important then, to find a balance between a large inclusive camp and an intellectually elitist one. This is a balance every movement has to strike and in the spirit of PDIA, continually revisit over its lifetime. However, it is important to anticipate this challenge. That the World Bank is so enthusiastic about it should make them wary already – its as clear a sign as any that PDIA and DDD are well on their way to being ‘mainstreamed’. But then, remember the ‘top table’ argument – that is the high stakes DDD game. Also, hopefully, the DDD tribe will ensure that DDD doesn’t get confined to a single model – not a Payment by Results (PbR) one; nor an Embedded Technical Assistance (ETA) one. We truly need to allow a thousand flowers to bloom.
  6. DDD is spreading! – The principles of DDD are finding their ways into many donor RfPs. Those proposing projects are being forced to regurgitate the jargon, hopefully in ways that make sense. Do I see donors reaching out to their partners and contractors to make sure they ‘get’ it? Not as much as I would like.

All of this is very exciting. Getting deeper into this has implications for everything: in our personal lives as much as in our professional ones; in how we do and learn. Will stay tuned.

NGOs, dont back off – be more political

In my latest livemint column, I argue that NGOs need to be far more political than they currently are, if they wish to really serve the communities they work with. Malicious political masters and their misguided tools such as the IB should not be allowed to dictate the choice of strategies on the ground.

While every organisation must face questions – both from the inside and outside – about its mode of functioning, the response to these questions should be aimed at improving organisational performance

It is easy to brand NGOs as utopian or Luddite as opponents struggle with their conception of public interest. What gives an NGO the legitimacy to pick a particular issue over another? Why is it water and not housing; or watershed development and not skills training; gender equality and not income generation? In making these choices, NGOs either use their own narrative about the situation on the ground, or allow themselves to be driven by needs expressed by communities. One can argue for or against either approach. It is also fair to argue that NGO activity that focuses purely on mounting pressure on the state neglects many immediate needs that communities face and therefore, ignores potential fixes to those problems. It should be evident to anyone who thinks through this that depending on the issue at hand and the context, these are all legitimate questions that must be asked of organisations that are supposedly working in public interest.

Allowing ‘revisibility’ in decisionmaking

This is a joint post with Heather – fifth in our series on decisionmaking. Now also on the World Bank PSD blog

***

Throughout this series of posts (1, 2, 3, 4), we have considered two main issues. First, how can evidence and evaluation be shaped to be made more useful – that is, directly useable – in guiding decision-makers to initiate, modify, scale-up or drop a program? Or, as recently pointed out by Jeff Hammer, how can we better evaluate opportunity costs between programs, to aid in making decisions. Second, given that evidence will always be only part of policy/programmatic decision, how can we ensure that decisions are made (and perceived to be made) fairly?

For such assurance, we primarily rely on Daniels’ framework for promoting “accountability for reasonableness” (A4R) among decision-makers. If the four included criteria are met, Daniels argues, it brings legitimacy to deliberative processes and, he further argues, consequent fairness to the decision and coherence to decisions over time.

The first two criteria set us up for the third: first, decision-makers agree ex ante to constrain themselves to relevant reasons (determined by stakeholders) in deliberation and, second, make public the grounds for a decision after the deliberation. These first two, we argue, can aid organizational learning and coherence in decision-making over time by setting and using precedent over time – an issue that has been bopping around the blogosphere this week.

These criteria, and an approach ensuring A4R more generally, are also a partial response to increasing calls for donor transparency, made loudly in Mexico City this week via the Global Partnership for Effective Development Co-operation. These calls focus on the importance of public availability of data as the key ingredient of donor (and decision-maker) transparency. We concur on their importance. But we argue that it is incomplete without an inclusive process of setting relevant reasons on how those data are used (recognising that they will always only be part of the process) and making the decision criteria as well public.

The publicity and transparency around decision-making opens the door for A4R’s third criterion (and the subject of this post): the possibility to appeal and revise decisions. As Daniels notes, this condition “closes the loop between decision-makers and those who are affected by their policies.”

As a quick reminder of our guiding scenario: we specifically focus on the scenario of an agency deciding whether to sustain, scale, or shut-down a given programme after piloting it with an accompanying evaluation — commissioned explicitly to inform that decision.

In most decision-making of this kind, some stakeholders — often would-be beneficiaries — will not agree with the decision and even feel or be adversely affected. While we suggest that stakeholders be involved in the earlier process of setting relevant reasons, a grievance-redressal or dispute-resolution mechanism, as provided by the revisibility criterion, gives these stakeholders an opportunity to voice their perspectives, based on the original grounds of the decision.

They can do this because the decision-criteria are made public, via criterion 2. This “visible and public” space for further deliberation provides stakeholders have a route “back into the policy formulation process.” Stakeholders can use evidence available to them to advocate a certain way forward; it also allows for stakeholders to revisit the decision-making criteria and the decisions they fostered. Stakeholders therefore have the opportunity to make a case for a change in the decision.

Why might past decisions be questioned? Since the appeals process is largely based on the original decision criteria, appeals come if circumstances around those reasons changed. For example, in considering relevant reasons, feasibility was one category of criteria we proposed, such as government’s capacity to scale a program or their interest in the program. One can imagine that over time, over changes in regime, and over changes in politics and policy, the original answers to these criteria could change, opening space for appeals. An additional set of proposed relevant reasons related to cost, effectiveness, and cost-effectiveness. The costs of technologies and materials may change over time or fresh evidence could come out about long-term benefits of programs This alters the original cost-benefit ratio, again, opening a space for appeals against the original decision.

Such appeals may come from members of civil society (or government) that would like to see the program brought back to life (or to see it go away). These may also come from donors themselves wanting to look at their decision-making over time and implement changes in line with the changing context.

Daniels is careful to note, and we emphasise, that the power and purpose of this criterion is not that citizens will always overturn prior decisions. Decisions on limits are requisite, as needs generally outstrip resources.* Rather, the revisability criterion allows for reconsideration and reflection on those decisions by those knowledgeable about the topic and empowered to alter decisions, if seen fit and feasible. This can, Daniels notes, bring further legitimacy to decision-making processes and, again, improved decision-making over time.

We want to stress that these deliberations over decisionmaking and their ‘revisibility’ has to be situated in a rational and ethical decision-making framework, predicated on meeting needs fairly when not all can be met (distinct from, say, a legal framework). Appeals will have to be judged on the original merits of the arguments as well as with recognition that aid resources have limits (although obviously, a different argument can be made that aid budgets should simply be bigger).  Moreover, appeals need to be judged by people who understand the original decision and have the power to change it, if that is the decision taken. When decision-making criteria are set, they set the roadmap for a possible appeals process and should be accordingly discussed and agreed upon.

We started this series of posts by admitting the limited role evidence plays in decision-making — even when those commissioning evidence intend specifically to inform that decision. We considered how planning for decision-making can help in the production of more useful evidence and also how decisions can be made fairly, through the delineation of relevant reasons, the publicity of the decision criteria ultimately used, and now, the possibility of revisiting through criteria and revising decisions.

Our thoughts in this series of posts should not make fair decision-making seem like an impossible task. Not all aspects of each of these considerations can be taken into account – the constraints of the real world are not lost on us and A4R remains an ideal, though we think one that can be approached. In our final post of this series, we therefore attempt to close the loop by looking at enforcement – asking how these ideas can be enforced and decision-makers held accountable.

***

*See. e.g., Richard Horton’s recent slide about the limit-breaking decisions by courts and the effects on health care systems, as in cases like Colombia. Experiments with health courts may be instructive. Picture via @fanvictoria, citing @richardhorton1.

‘Going public’ with decisionmaking

This is a joint post with Heather – 4th in our series on decisionmaking; now also on the World Bank PSD blog

***

In our last post, we discussed how establishing “relevant reasons” for decision-making ex ante may enhance the legitimacy and fairness of deliberations on resource allocation. We also highlight that setting relevant decision-making criteria can inform evaluation design by highlighting what evidence needs to be collected.

We specifically focus on the scenario of an agency deciding whether to sustain, scale or shut down a given programme after piloting it with an accompanying evaluation — commissioned explicitly to inform that decision. Our key foci are both how to make evidence useful to informing decisions and how, recognizing that evidence plays a minor role in decision-making, to ensure decision-making is done fairly.

For such assurance, we primarily rely on Daniels’ framework for promoting “accountability for reasonableness” (A4R) among decision-makers. If the four included criteria are met, Daniels argues, it will bring legitimacy to deliberations and, he further argues, consequent fairness to the decision.

In this post, we continue with the second criterion to ensure A4R: the publicity of decisions taken drawing on the first criterion, relevant reasons. We consider why transparency – that is, making decision criteria public – enhances the fairness and coherence of those decisions. We also consider what ‘going public’ means for learning.

Disclaimer: logistical uncertainties / room for conversation and experimentation

From the outset, we acknowledge the many unanswered questions about how much publicity or transparency suffice for fairness and how to carry it out.

  • Should all deliberations be opened to the public? Made available ex post via transcripts or recordings? Or, is semi-transparency — explicitly and publicly announcing ex post the criteria deemed necessary and sufficient to take the final decision — acceptable, while the deliberation remains behind closed doors?

  • Who is the relevant public?

  • Can transparency be passive – making the information available to those who seek it out – or does fairness require a more active approach?

  • What does ‘available’ or ‘public’ mean in contexts of low-literacy and limited media access?

We do not address these questions — which are logistical and empirical as well as moral — here. As the first-order concern, we consider why this criterion matters.

Fairness in specific decisions

Any decision about resource allocation and limit-setting will be contrary to the preferences of some stakeholders – both those at and not at the decision table. In our scenario, for example, some implementers will have invested some quantity of blood, sweat and tears into piloting a program and may, as a result, have opinions on whether the program should continue; or, those that were comfortable in their inaction (as a result of lack of directives or funds or just plain neglect) who will now have to participate in a scale-up. There will be participants that benefitted during the pilot – and those who would have done so if the programme were scaled – that may prefer to see the programme maintained.

These types of unmet preferences shape Daniels’s central concern: what can an agency* say to those people whose preferences are not met by a decision to convince them that, indeed, the decision “seems reasonable and based on considerations that take… [their] welfare into account?”** Being able to give acceptable explanations to stakeholders about a decision is central to fairness.

Coherence across decisions

The acceptability of criteria for a given decision contribute to the fairness of that decision. But long-run legitimacy of decision-makers benefits from consistency and coherency in organizational policy. Transparency, and the explicitness it requires, can foster this.

Once reasons for a decision are made public, it is more difficult to not deal with similar cases similarly – the use of ‘precedent’ in judicial cases aptly illustrates this phenomenon. Treating like as like is an important requirement of fairness. Daniels envisions that a series of explicated decisions can function as an organizational counterpart of ‘case law’. Future decision-makers can draw on past deliberations to establish relevant reasons. Deviations from past decisions would need to be justified by relevant reasons.

Implications for learning, decision-making and evaluations

If all decision-makers acknowledge that, at least, the final reasons for their decisions will be publicly accessible, how might that change the way they commission an evaluation and set about using the evidence from it?

  • It should encourage a review of past deliberations to help determine currently relevant reasons. Second, it might encourage decision-makers and evaluators to consider as relevant reasons and measures that will be explainable and understandable to the public(s) when justifying their decisions.

  • In planning evaluations, decision-makers and researchers will have to consider the clarity in methods of data collection and analysis — effectively, will it pass a ‘grandmother test’? Moreover, does it pass such a test when that granny is someone affected by your allocative decision? Remember the central question that makes this criterion necessary: what can an agency say to those whose preferences are not met by a decision that, indeed, the decision “seems reasonable and based on considerations that take… [their] welfare into account?”

  • There are reasons that decision-makers might shy away from transparency. In his work on health plans, Daniels notes that such organizations speculatively feared media and litigious attacks. In our pilot-and-evaluate scenario, some implementers may not be comfortable with publicising pilots that may fail; or from raising expectations of beneficiaries that are part of pilots.

  • The fear of failure may influence implementers; this may lead to low-risk/low-innovation pilots. Again, this is an important consideration raised above, in the questions we did not answer: when and how much transparency suffices for fairness?

In our last blog, we stressed on the importance of engaging stakeholders in setting ‘relevant reasons’ before a project begins, as a key step towards fair deliberative processes as well as a way of shaping evaluations to be useful for decision-making. Ensuring publicity and transparency of the decision-making criteria strengthens the perception of a fair and reasonable process in individual cases and over time.

This also sets the stage for an appeals process, where stakeholders can use evidence available to them to advocate a certain way forward; it also allows for stakeholders to revisit the decision-making criteria and the decisions they fostered – the subject of our next post in this series.

***

*We note that donors don’t actually often have to answer directly to implementers and participants for their decisions. We do not, however, dismiss this as a terrible idea.

**We are explicitly not saying ‘broader’ welfare because we are not endorsing a strictly utilitarian view that the needs of some can be sacrificed if the greater good is enhanced, no matter where or how  that good is concentrated.

 

Politics of Evidence @ IDS

IDS just concluded its ‘Politics of Evidence’ conference – thankfully having moved from the ‘big push-back’ to ‘the big push forward’.

My friend Heather has a summary here. Loving the typical IDS-speak that seems to have dominated the conference!

Waiting for an update here from Ros

Now, if only a conference such as this could be held closer to the field – where the politics of the evidence plays out…

Why the traditional model of NGO-led development must end

My latest column for livemint, on how the space for traditional NGO-led development model is shrinking
***
The 1980s and 1990s were the heydays of the NGO-led development model in India, where significant funds flowed through them into development programmes targeting the poor. NGOs played a critical role in developing model interventions of service delivery and giving a voice to the poor through long-term engagement and institution-building at the local level. Concurrently, powerful networks of NGOs spoke passionately on behalf of the poor at various national and international forums. For many years, NGOs faced criticism that they were taking over the role of the state in providing basic services. Over the last few years, however, we have seen a shift in the way development projects are conceptualized and executed in India. Not only does the state have the largest development footprint in India, it also retains the power to “allow” citizens to exercise their rights and claim their state-granted entitlements; and give NGOs the regulatory space in which to operate while working for the people.
In order to understand this phenomenon better, it is helpful to look at research on “power”. John Gaventa is one of the leading researchers on the subject of “power” and “participatory development”. One of the key aspects of “power” is that of “spaces”. Spaces may be understood as opportunities or platforms for the powerless to participate and negotiate with the power holders. Gaventa suggests a continuum of spaces as closed (spaces from which the power holders exclude others and often have to resist opening up under pressure from civil society); invited (spaces created by the power holders who invite others to participate); and claimed (spaces that exist as informal arrangements that have developed organically through individual or joint efforts by the powerless).
Gaventa’s definition of spaces also shows how these are shaped and transformed through interactions between various stakeholders, all of whom hold varying degrees of power. Often, the spaces that come to exist are a combination of the three types outlined above. Closed spaces may be opened up by invitation and claimed spaces may be institutionalized through legislation. In the Indian context, this gives us a framework to look at the role and reach of the state and that of external agencies, such as NGOs that attempt to work with the state. In earlier decades, NGOs would mobilize citizens to form pressure groups and demand that the state open up hitherto closed spaces.
Over the last decade, the state has expanded its welfarist wings by pursuing a path of “granting” rights. I choose my words intentionally here: in India, the state has “granted” the rural poor the right to wage employment, to information, to education and possibly, to food security. More fundamentally, the decentralization reforms of 1993-94 have decisively opened up these “closed spaces”, inviting people to contest elections and run their own affairs at the local level. At the same time, granting rights to citizens is an invitation to citizens to engage with the state and expand the “claimed spaces” of policy formulation, implementation and holding representatives of the state (both the politicians and bureaucrats) to account. Thus, whether we talk about NGOs that execute programmes or individual/institutional donors that fund or activists who protest, they all function in the spaces created by the state.
Even in their heydays, NGOs suffered from serious criticisms of perpetuating dependency and of their inability to scale up. In a context where the state seems to be constantly redefining its role, NGOs that positioned themselves as the representatives of the unrepresented citizens have had to cede ground to the 3 million democratically-elected panchayati raj institution members across the country. These are 3 million men and women, in leadership positions in their community, who people can hold to account for the delivery of basic services. There is pressure on NGOs on the ground to transform their strategies in order to stay alive and relevant. This is already visible in many parts of the country. While aid funds continue to whittle down, NGOs are increasingly positioning themselves as catalysts that promote and strengthen people’s institutions, primarily through strengthening local government institutions that are meant to represent and serve the people. In terms of facilitating development through equitable access to services for the millions living in poverty, that is not just the best way ahead, it is the only one.
***
There are no doubt NGOs that are still pursuing the traditional service provider model. They all risk being left behind in the tide of policy evolution, as communities recognise that engaging with the government remains the only way towards achieving sustainable improvement in their quality of life.

Did the EU deserve its Nobel?

H/T to Chris Blattman, who links to this paper by Aronow, Carnegie and Marinov. Starting with a clever way to tackle the problem of endogenous aid (giving and taking) decisions, relying on the rotational system of determining EU presidency –

We identify a process that drives aid allocation and is exogenous to the rights and governance in recipient countries: the rotating presidency of the Council of the European Union (EU Council). Since the country holding the presidency is determined exogenously, the set of countries that happen to be former colonies of the president is also exogenous. We find that when a country’s former colonizer is the president of the EU Council during the budget-making process, the country is allocated considerably more EU aid. This exogenous shock to a recipient country’s aid allocation serves as the basis of our estimation strategy. We use the colonial relationship to the current EU Council president as an instrumental variable for the amount of foreign aid from the European Community in the following year

Establishing that the EU presidency has a significant influence on its aid budget and recipients, the authors show that a strong and significant positive effect exists of being a former colony (of the country holding the EU presidency) in being sanctioned EU aid. Also,

The effect on CIRI Human Empowerment occurs immediately, and then (nearly monotonically) declines each year, demonstrating the short-lived nature of the effect of the exogenous shock to foreign aid. Similarly, the effect of Polity peaks in year t + 3 and then rapidly declines. The delayed effect for Polity is understandable, as the CIRI Human Empowerment score measures behavioral changes which can show up quickly, such as freer speech and association, while Polity measures more structural changes that may take time to show up, such as electoral freedom…Taken in sum, the year-by-year effects point to the same conclusion: the increase in foreign aid induced by the rotating presidency yields non-trivial, but relatively short term improvements in human rights and governance.

To sum up,

We find that the EU’s ‘recipe’ of success can work beyond the scope of potential members; even when the benefits offered are substantially smaller than the benefits of membership, the EU can still elicit movement in rights and freedoms. Broadly speaking, our paper provides striking evidence for the ability of an international institution to widely improve human rights and governance