Reading ICAI’s review of DFID WASH results

The Independent Commission for Aid Impact (ICAI), UK’s aid watch dog, today, released its review of DFID’s programming and results in water sanitation and hygiene (WASH). – In this impact review, they take a close look at the results DFID reported in its 2015 Annual Report; results that cost £ 713 million between 2010 – 2014.

Do read the full report here.

Some thoughts on the areas of concern in the report:

  • The focus on ‘leaving no one behind’ is spot on. It is easy to stack up impressive WASH numbers if one ignores the poorest and the most vulnerable in communities. Safe sanitation and hygiene need to be universal for health benefits to accrue to communities. Within WASH, sanitation is specifically complex, sometimes also called a ‘wicked problem’ – a challenge foremost, of inducing lasting behaviour change. The very nature of careful social engineering required to bring about this behaviour change seems to run contrary to some of the factors that make an intervention scalable – an ability to standardise inputs and break programme components down to easily replicable bits.
  • Within the broad basket of ‘service delivery’ interventions, WASH is one of the trickier sectors when it comes to measuring sustained impact, especially at scale. Naturally then, ICAI find that while DFID’s claims of having reached 62.9 million people are broadly correct, it is very hard to establish if the benefits are sustained. Therefore, the results reported remain at the ‘output’ level and that is what ICAI ends up assessing, even though what they set out to do is an ‘impact’ review. While the report speculates on sustaining benefits beyond the 2011-15 period, I wonder whether those that accessed the programme in 2011-12 continued to experience any benefits in 2015.
  • The link with government systems, in terms of implementation, monitoring and sustenance remains unclear: another typical WASH issue. Barring say, India, (and this is true especially in sub-Saharan countries, government WASH budgets are highly inadequate. A lot of the work that happens is funded by donors and this implies that monitoring and maintenance happens outside the official system. Achieving local ownership in such a context is a challenge.
  • ICAI finds it difficult to assess value for money (VfM) in DFID’s WASH programmes. On one hand, it finds that there isn’t enough competitive procurement, but also there is a lack of established metrics and benchmarks to analyse VfM. Following DFID’s own 3Es framework, an Economy and Efficiency analysis should be possile across the portfolio, and as far as I can tell, is rapidly being developed in the sector, and within DFID. However, partly as a consequence of the lack of ‘outcome/impact’ data, cost-effectiveness studies are likely to remain a challenge. This work by an OPM-led consortium should be particularly relevant in improving VfM analysis across the sector

How could this review have been better?

  • We could do with a better feel for DFID’s global portfolio – what the variationa in context-specific experiences, challenges and results are. But probably fair to concede that these boundaries to scope must have been considered when the methodology was being finalised.
  • Use administrative data: The review states that several DFID programmes contributed towards larger national programmes, or attempted to strengthen national/local data management systems. It would be great to hear more about experiences from different countries – with varying state capacities – on the extent to which it was feasible to use administrative data as an effective monitoring tool.
  • While in absolute terms, 62.9 million is an impressive figure, we are looking at it against the backdrop of a staggering 2.4 billion who lack access to reliable WASH services. In the countries where DFID made significant investments, was there significant progress made towards reaching the MDG targets? Yes, the MDGs were output-based too, but it would be great to get a sense of the scale of impact achieved at the national-level.
  • Finally, on learning, the report is critical of DFID, saying that not enough is being done to share lessons across programmes, or from related sectors such as health and education. But surely, there are lessons programmes have learnt over the five years (2010-15)? Further, in the absence of good impact data, was there at least good quality process data? Have we learnt how to implement a WASH programme better?

As I have written previously, ICAI reports are a great addition to the sector, both in terms their scrutiny function of one of the biggest donors in play, as well as because of the questions they ask. So do keep an eye on this space.

The false dichotomy amongst ‘sanitation for all’ advocates

(di·chot·o·my  noun: a difference between two opposite things : a division into two opposite groups)

**

A good way of blocking progress in an argument is to present two aspects of a whole as a dichotomy. The sanitation debate, in recent years, has suffered from a seemingly irreconcilable dichotomy when it comes to identifying the best approach towards ‘sanitation for all’. This is the one that pits subsidies against motivation and correspondingly, construction against behaviour change communication. And yet, in a comprehensive and prudent programme design, there is no need for these ideas to be opposed to each other. I call this then, the false dichotomy in the world of sanitation advocates.

The current sanitation programme in India has at its centre, a subsidy/incentive for individual households constructing toilets. This is a programme that has clearly not worked – irrespective of the minister or bureaucrat at the helm of affairs. India holds the ignominious record of having the largest number of people defecating in the open. At the same time, the popularity of the Community-led Total Sanitation (CLTS) approach has risen. This approach depends on using ‘shame and motivate’ as a call to action to build basic pit latrines (rejecting subsidies completely) and has worked in multiple countries around the world, as well as in certain states in India.

There is a catch here. Although there is sufficient evidence from locations that have used this approach that households construct latrines and start using them, there is little evidence that shows that these latrines continue to be used in the long-term. For instance, this study from Kenya, Uganda, Ethiopia and Sierra Leone reveals a slippage rate of nearly 90% – meaning, nearly 90% had gone back to unhygienic sanitation practices over a period of two years after the intervention. The study calls for identifying ways to help families upgrade their sanitation infrastructure alongside greater follow-up and continuous health messaging. CLTS does not address this infrastructure gap, neither does it support the poorest families in availing usable and lasting toilets.

This way of framing the debate ignores a key question – how can we ensure that all families in a geographic cluster (village/neighbourhood/slum) use a working toilet and the associated safe sanitary practices? Irrespective of the side one chooses of this false dichotomy, it is clear that an either-or solution is doomed for failure.  Most importantly, one often forgets that open defecation is a public health hazard which can only be tackled if entire neighbourhoods adopt safe sanitation practices. Even one family refusing to use a toilet – whether they lack awareness or funds – poses a risk to everyone else, irrespective of how safe their own sanitation practices are. This in particular implies that solutions to open defecation have to involve a mix of motivation and tangible assistance with infrastructure.

Recent research that has been widely reported show that a large number of our rural countrymen (more than countrywomen) do not care about toilets. It then follows that what is required indeed is a change in attitudes and demand creation. Very well, thus far. However, the current reality is that the government’s existing sanitation programme churns out un-usable toilets that destroys any demand that might actually exist – for households that receive it, as well as anyone living in their vicinity. Therefore, the first step towards fixing this government programme is to ensure high standards in construction.

An intervention that sets out to do so will have to include multiple components: creating awareness so households buy-in to the programme; dealing with corruption on the ground; and community participation in monitoring and maintenance. Awareness campaigns for behaviour change is thus a key component that holds the intervention together – in ensuring both quality construction as well as usage thereafter. This might seem like a costly and drawn-out process, but not for nothing is sanitation often called a ‘wicked problem’.

What is also lost in this dichotomy is the opportunity to make progress in the way we design our interventions. Can provision of financing be structured in ways that it puts the onus back on local governments and individual households? Who should take the lead on communication campaigns and how should these be sequenced in combination with the construction work? What kinds of messages work for particular communities? How can households be encouraged to contribute in cash and kind towards their toilets, as well into the creation of a sanitation system (toilets, sewers/drains, maintaining clean drinking water sources, etc) that needs to be managed as a public good?

It is easy to see why the behaviour change-led zero subsidy approach makes for a seductive argument. Years of sanitation subsidies in countries like India have seen little impact and have sometimes lead to perverse incentives amongst both households and local governments. On the other hand, phlegmatic bureaucrats and contractors have no incentive in diluting the focus on infrastructure which maintains the status quo. However, unless we break out of this false dichotomy and bring both these approaches together in equal measure, our efforts at presenting a sustainable, inclusive and dignified solution is bound to fail.

How Bharat squats

In this column for livemint, I disagree that in our sanitation policy, all we need to worry is about behaviour change communication. If we dont get our infrastructure right, little else is going to make sense. Many other poorer countries may have succeeded with little funds and pit-latrines, but the same has not worked for India and is unlikely to, irrespective of what we do by way of motivating those squatting in the open to use a toilet instead.

I peg my argument on the findings from the recent SQUAT survey by RICE.

Read the column here

Nirmal Bharat Abhiyan: beyond the hype

In my latest livemint article, I write about Nirmal Bharat Abhiyan (NBA) which is the new avataar of the Total Sanitation Campaign

…those who expected a clear expression of policy-intent and innovative implementation strategies are probably disappointed with the end-product. The new scheme does little by way of increasing the government’s financial commitment on the ground; and that the new guidelines have little new to say about institutional convergence on the ground or about enhancing sustainability of the outputs of this programme. It must be mentioned though that the guidelines acknowledge the need for quality with references to a ‘toilet unit with a superstructure’—and for this they must be appreciated. However, there is very little clarity on how this new avatar of the sanitation campaign will be operationalized in the field

More here 

Waking up to shit

An international emergency, killing millions every year from among the 2.6 bn people worldwide who dont have access to toilets. The story here in The Guardian, that of Liberia and its sanitation champion, Ellen Johnson-Sirleaf, none other than the President herself.

She has no doubt how critical sanitation is…

…Half the hospital beds in sub-Saharan Africa are filled with people suffering the consequences of bad sanitation. But, of course, the president sees endless statistics. Only when she looked into why so many Liberian women were dying in childbirth, and why children were dying of something as banal as the squits, did she realise “there is a relationship with water and sanitation. I needed to understand why that was so, and partly it’s because people don’t have access to clean water. That was an eye-opener for us.”

…and is well aware of how un-sexy the subject is –

…In Monrovia, ministers and NGOs hold a weekly crisis meeting about refugees, but not about the 18% of Liberians who die because they have no toilets or clean water.

Towards the end of our interview, I ask the president why that is…”The humanitarian system responds to these things that get sensational,” she says. “They want to be seen as responsive. The ordinary village, that no one is taking care of, that doesn’t come to mind.” And with that she takes her leave, to get back to the job of fixing her country, one latrine at a time.

Toilets for all

Could he (Union Rural Development Minister, Jairam Ramesh) name one area where individual contribution would make a difference? His answer is surprising: toilets.

“To my mind, the single biggest issue that we face is sanitation,” he says. “Fifty-eight per cent of all open defecations in the world are in India. Fifty to seventy-five per cent of all Indian women defecate openly. It is a national shame, a national blot. We are trying to entice people to use toilets. We have built lakhs of toilets but people aren’t using them. We are trying to start a major Clean India campaign; communication programmes and such.”

In this article, the author seems to have been struck by a new realisation – that toilets are important. And that’s part of the problem. Sanitation is urgent, and complex – and should have been our priority five decades back.

The great sanitation stink

Recently, addressing a workshop of state secretaries in-charge of Rural Water Supply and Sanitation, Jairam Ramesh​, the minister for rural development, termed it a shame that 58% of the world’s open defecation happens in India. Anyone who has worked on sanitation would take this figure with a pinch of salt, knowing how misleading sanitation coverage figures are – possibly not just in India, but all over the developing world.

A country that aspires to be a global superpower lags behind numerous other poorer countries when it comes to a ranking on the basis of the proportion of population with access to improved sanitation. In India, sanitation seems to suffer from a policy blind-spot, being overshadowed as policymakers miss the link between sanitation, hygiene, health and productivity. This has led to numerous unsuccessful policy attempts to address the issue. There are of course hundreds of successes, but rarely have they been replicated or scaled up to a degree that matters. In this piece, we would like to highlight a few broad challenges.

Excerpt from our piece published on livemint