Rating development interventions is not as simple as GiveWell suggests

In a previous post, I wrote about GiveWell’ charity rating criteria and their flip-flop on AMF. In the latest rating released earlier this month, GiveWell dropped AMF from its ‘top three’ and instead included ‘Deworm the World Initiative’, in addition to Give Directly and Schistosomiasis Control Initiative (SCI). GiveWell also announced a modification to their rating criteria, a slight departure from the criteria used in 2012. However, GiveWell continues to get it wrong, or at least, it furthers the process by which its donors get it wrong when they pick which charities to support.
GiveWell continues to promise its audience: “best opportunities we’re aware of to help low-income people with relatively high confidence and relatively short time horizons”. GiveWell may well be recommending worthy charities, but focusing on charities* that rely on a direct hand-out model (whether bednets or deworming pills) reflects a narrow ideology of what charities should be doing. By GiveWell’s standards and criteria, no charity that brings people together for any form of collective action or those that work on interventions with a long gestation period will ever make the cut. You can see how this is a big problem – looking for quick fixes alone is not all of the development industry is about.
In a comment on our last blog, we were asked how the AMF issue was dissimilar to private companies that get a lot of cash suddenly and grow. Well, the dotcom boom etc come to mind, where much money was pumped in on an attractive idea that promised short-term gains, before it all crashed spectacularly. But with AMF and GiveWell, it is really important to look at what the role of charities are and what their ability to engage with the government and other key stakeholders actually is – and this is another area where GiveWell seems to get it wrong.
Private companies, I think, operate in a slightly different space, where their aim is to create a market for their goods and services within the regulatory framework in the geographies where they operate, keeping with the space created for them by states. Keeping within the regulations is a sufficient maker of their engagement with governments. Charities usually attempt to do a better job than the state in delivering the same set of goods/services and thereby setting up demonstration pilots, or making up for the state’s oversight, neglect or weaknesses in general. Or else, they should be helping the state do better by collaborating with them on the ground, or by mobilising resources. The problem is not ‘more money’, it is the resource deployment and local engagement strategy
(*I hate the word charities)

6 Replies to “Rating development interventions is not as simple as GiveWell suggests”

  1. I think it’s helpful to distinguish between not-for-profits/charities and the issues of delivering commodities (i.e. not all charities deliver commodities, not all commodities are delivered by charities). True that commodity-delivery alone probably isn’t going to ‘solve’ poverty but we do have some sense of efficacious commodities that can be delivered (e,g, https://www.unfpa.org/public/home/publications/pid/12042). GiveWell’s focus on trackability will always probably keep them in the commodity space – and as long as they are clear that they are really ranking from among a subset of ‘charities’ that focus on supply-side issues, that’s their prerogative.

    The bigger issue for me is whether these efforts also strengthen the state’s capacity to deliver to its citizens (a ‘diagonal’ approach in global health jargon, http://www.who.int/social_determinants/resources/frenk.pdf) rather than leaving ‘nothing of lasting value,’ substituting or paralleling the public (and for that matter, for-profit private) sector.

    Like

    1. Cannot agree more. I think primarily of development NGOs – agencies that deliver goods/services or mobilise them for self-help/activism etc. GiveWell’s charities seem to only deliver tangible commodities that can be distributed to individuals directly – this is at best, a small sub-set of what development NGOs are doing out there, and its highly debatable, if it is the most important one. This also relates to the concept of aid as band-aid as opposed to other medium to long-term interventions.

      Like

  2. I’m a research analyst at GiveWell.

    Thanks for the thoughtful comments on AMF and on some limitations of our top charities. We definitely agree with you that giving is not simple (some posts on this subject: The risks of giving and Why we should expect good giving to be hard).

    We also agree that there are limitations to focusing on charities that can help people with relatively high confidence and short time horizons. Donors may be able to do even more good by thinking about philanthropy more broadly, which is why we’re investigating philanthropic opportunities that fall outside our traditional criteria through GiveWell Labs. Here’s our latest update on this work. We agree that promoting collective action and changing the behavior of governments might be high impact, which is why we wrote a series of blog posts on policy-oriented philanthropy. These beliefs have affected our traditional top charities recommendations, too. One of our top charities, the Deworm the World Initiative, primarily supports government deworming programs.

    Like

    1. Thanks for engaging, Howie. I look forward to reading the ‘update’. I must confess that while it is easy to critique the criteria (which I know are evolving), it is very difficult to arrive at an alternative that can satisfy everyone. Community engagement and collaborating with governments is hard enough for agencies on the ground to pull off. Evaluating them with an eye on ‘value for money’ for an individual donor is even more difficult. So it is exciting that you guys are working on this…and very encouraging!

      Like

    2. Hi Howie, if I may chime in as well… One, it’ll be exciting to watch what comes out of GiveWell Labs! And, two, thanks for not only engaging here but for GiveWell’s commitment not only to the transparency of the orgs it recommends but also your own. In development work, it’s unfortunately rare to find groups that really practice and live what they preach and impose. I find this to be pretty inspiring.

      Three, we definitely recognize that Deworm works with governments. For me, the issue in spotlighting an organization for increased financial support is not only they have worked with *a* government but whether there is evidence that they can work with *the* government with whom they would likely work given funding boost. More money may allow them to scale from the state to the national level in a given country or to expand to additional countries. It would be nice to know if organizations being recommended had already laid at least some groundwork for these new partnerships, such that GiveWell’s ratings reflected both proof-of-success and probability-of-success.

      Four, I read through the posts on policy-oriented philanthropy, which are quite interesting. If I read correctly, this is largely focused on advocacy groups focused on big policy change. I think this is really great but there seems to be some important middle ground between advocacy and the present GiveWell approach. As noted on the site, GiveWell’s standard focus is on organizations that can make statements such as “$X delivers Y bednets, which saves Z lives.” I’d be interested in seeing GiveWell move into the space of expanding that basic sentence to read something along the lines of: “Q org delivers B bednets for $X in Y years, which, given usage statistics, means Z people protected from malaria. In addition, $X and E efforts are devoted to assisting the relevant government to deliver bednets on its own.” While policy change is hard, implementation can prove even harder and it would be great to see GiveWell requiring organizations to leave behind structures or processes ‘of lasting value’ that can facilitate future implementation. The debate 35+-year within global health on horizontal approaches versus vertical approaches versus diagonal approaches may prove instructive and GiveWell may be able to move into the ‘diagonal’ space.

      And, finally, as a not-quite-semantic issue, why does GiveWell use the word ‘charities?’ Perhaps it is an American or even more idiosyncratic thing on my part but charities make me think of orphanages and used-clothing drives and soup kitchens and Dickens. Do you use ‘charity’ synonymously with ‘not-for-profit?’ In any case, ‘charity’ – again, to me – perpetuates a feeling of giving-handouts-to-(temporarily?)-alleviate-poverty-and-provide-succor. Terms like ‘development partners’ and ‘change agents’ are, of course, wildly overused and have become hollow but… some sort of reframing in that direction could be helpful even if, again, largely semantic.

      To circle back to the beginning, thanks again for your engagement and transparency and looking forward to see what is being cooked up in the Labs! Also, as a disclaimer, Suvojit is not responsible for any of what I have written above!

      Like

Leave a comment