If using ‘Theories of Change’ cannot transform the way you operate, why bother?

In a new (and commendably short) paper, Craig Valters advocates ‘modest radicalism’ in the use of Theories of Change (ToC) as an approach to improving reflection and learning in the development sector. In this paper, Craig reflects on the role of the ToC in the context of the ‘results agenda’ and suggests four principles that could help development organisations develop knowledge and improve practice: Focus on processes; Prioritise learning; Be locally-led; and ‘Think compass, not map’. Do read the full paper!

In this post, I share some additional thoughts on the use of ToCs and how they might be improved. I start with two problems in the way we do things.

  • In development, failures are hard to detect: Often, organisations that fail find ways to mask failure – by either refusing to acknowledge failure, finding external factors, or moving on to a different desk officer/donor/location. So within the aid industry, we have a peculiar situation where it is real hard to fail – or at least, it is hard to know when a project has failed.
  • It’s harder still to ensure that projects that fail face significant consequences of failure: Organisations that implemented the failed projects should be required to make significant changes to key aspects of design or management.

Learning lessons and being accountable to stakeholders (including, but not just donors) is at the heart of both of these issues above. In the wide world outside the aid industry, projects and enterprises are set up and when they fail, those that are invested in these projects/ideas are forced to innovate, or reinvent. The margins of failure are finer and value-for-money considerations are built in to the system (as opposed to being introduced from the outside as tools for accountability and reporting). Now I certainly don’t mean that doing development is like selling cola – but it’s worth reflecting on some fundamental institutional attributes that can offer lessons. It’s a bit like the “‘why not cash?’ challenge” that conventional humanitarian aid is faced with – an uncomfortable question that signals that we can no longer tolerate ‘business as usual’.

So the challenge I have is not a modest one: with ToCs, if you cannot transform the way development organisations operate, why bother?

A ToC is supposed to set out clearly, assumptions behind how change happens (sadly, in my head, the term also summons complicated images with too many boxes, and arrows going in multiple directions). It is almost impossible to have a ‘simple’ ToC, because the world we operate in, is inevitably uncertain and complex. And as both a tool and an approach, as Craig points out, ToCs should assist in organisational learning and accountability.

But there are serious obstacles on the path towards this ideal. In his paper, Craig points to the ‘time-consuming and misleading results agenda’ and refers to Chambers’ ‘accountability v/s accountancy’ culture that’s sweeping through the donor world. These are pressing issues, and allow me point out a couple of additional ones.

An example is DFID’s Smart Rules, introduced last year, ostensibly in response to a call for greater flexibility and innovation from its country offices. It is meant to be a broad operating framework focused on programmes, intended to support DFID staff in improving programme delivery. Out of the 37 mandatory rules listed, I saw no more than 5 rules that refer directly to the design and implementation of programmes, and on managing the complexity of the environments in which they operate. The rest were (possibly justifiably) quite focused on what one may call the ‘administration’ of projects. So while this may have been ‘handing over the stick’ to DFID staff lower down, I am not sure it sets out a clear vision to reduce the upward accountability (often defined in a narrow ‘accounting’ sense) that the bureaucratic aid system perpetuates.

In addition, I also worry about the fate of new ideas in development. For instance, the four principles in Craig’s paper emphasises on attributes that are not too far removed from what David Korten once saw in a ‘learning organisation’. According to Korten, learning organisations are those that “(a) embrace error; (b) plan with the people; and (c) link knowledge building with action”. This is one reminder of how the learning agenda has been part of the conversation in various forms – and then have either remained on the periphery, or have been brutally mainstreamed. It is the latter that I find more interesting, and brings to mind most prominently, the fate of tools such as the ‘logframe’ and approaches such as ‘participation’. Once mainstreamed, these concepts are implemented in a manner that make them unrecognisable to those that initially developed them. But this is indeed a serious dilemma – the trade-off between a ‘pure, but narrowly adopted radical concept’, and a ‘widely-applied, but contaminated standard’. The same easily holds for how much of the monitoring and evaluation data is treated in average organisations – there is no sense that a follow-up is required, and when faced with external pressure (from a donor, for instance), a cursory effort – to satisfy – follows.

Theories of Change are therefore on tenuous ground. On one hand, they are increasingly being used alongside logframes, as a ‘qualitative’ complement; but on the other hand, they are also being seen as a static ‘project design phase requirement’ that pass through without rigorous scrutiny. This brings me back to my original question – if we cannot make ToCs truly transformative, why bother?

So in turn then, I have two very modest proposals on how we can make a start (don’t ask me to explain what my theory of change behind these suggestions for making ‘Theory of Change’ work is)

  • Donors should invest more in creating and updating ToCs – formative work in project design is vital, so is space for review and reflection that have tangible outcomes. This means setting aside funds and time (sometimes, timelines are a bigger constraint than funds) for formative research and recognising the contribution of this phase towards the final outcomes of the project.
  • Invest in training and building capacity on understanding and using ToCs. Two decades back, when logframes appeared on the scene, there were a million trainings on the “science” of LFAs/logical frameworks. In my graduate programme in India, logframes were part of the curriculum and we were tested on them. Notwithstanding the actual utility of logframes, the point I am making is that if ToCs are to be used more widely and intelligently, development sector personnel need training and skilling in their development and use.

As projects and programmes are designed and implemented, one should set out clearly, what success looks like. A Theory of (good) Change will do just that, along with spelling out the complex nature of the path involved. Being bold about what success looks like in a particular context will also force one to be explicit about what constitutes ‘failure’. With this, it should be possible to continuously review and reflect on progress, and transparently tell between success and failure. It follows then, that projects that fail should prompt a re-evaluation of strategies and options and lead to concrete changes. As Craig says in his paper and I agree, “This may not sound too radical to those outside the industry, but within it, this is an important and pressing need”.

***

If you are interested in the subject, do also read these posts by Heather, Duncan Green and Jennifer Ambrose

Advertisements

6 thoughts on “If using ‘Theories of Change’ cannot transform the way you operate, why bother?

  1. Thanks for this excellent blog. I share many of your concerns. I have no doubt that Theories of Change are on tenuous ground. Indeed, I’ve argued previously along the same lines: if a Theory of Change approach is not transformative, then lets scrap it. But not without giving it a good shot first! And the principles I’ve outlined seek to take us in the ‘right’ direction.

    I fully agree that we need to pay attention to past debates and practices on the learning agenda. Readers will find lots of references to such past attempts and I really encourage those working on these issues to follow the paper trail, not least some of Korten’s work that you mention. Current donor reformers may find the paper by Edgren (1999) on Sida’s reform process helpful too. There is more on that in the Eyben et al. book on Playing the Game to Change the Rules (which I reference a lot in the paper) but that’s not available online.

    On mainstreaming/radicalism: my main argument is that if it is to become a mainstream tool/approach then it has to maintain a radical learning agenda within it. I share your doubts on whether this is possible. Lots needs to happen. Donors/implementers/etc need to buy into this way of thinking about Theories of Change and to drive it forward in their organisations. If those using Theories of Change can really get involved in the process, and see its value (not for money, but for people who should benefit from programmes), then that’s a good start.

    There are trade-offs when donors mainstream Theories of Change (by making them mandatory). It can create all kinds of dodgy incentives; it doesn’t have to, but when brought in alongside the normal top-down accountability demands, it often does. If just encouraged, it is more likely to retain its radical edge, which can leave space for creativity and genuine bottom-up reflections. Yet who has the time and space for that kind of process if not actively provided to them (by money and demand) by a donor? Furthermore, a related question is does it really matter whether it is mandatory or not, when really this kind of deliberation is a product of a huge number of things beyond a given tool or approach: personalities, org culture, country context, etc?

    For me, that’s why the search for a good tool or approach like Theories of Change only ‘remains relevant as part and parcel of a broader discussion on how to generate the right incentives to enable critical reflection to become the norm’. We need critical thinking to tackle the constant recycling of programme approaches, inability to tackle failure, and other issues in the industry you highlight.
    That means that your recommendations, which are good, will only be effective if they are part of a broader shift in thinking on the part of donors. Not sure the pendulum has swung back far enough in the direction of Korten et al just yet, but lots of people are working on it!

    Rather rambling end of the day comments from me, my apologies. Thanks again for getting involved in the discussion with some great practical thoughts.

    Like

  2. also, i would like to echo craig’s comment: “yet who has the time and space for that kind of process if not actively provided to them (by money and demand) by a donor?” donors have to play a role in not just asking for a theory of change once but for seeing iterations of it (at a minimum before and after the project, since critical learning occurred between the ex ante and the ex post version — and a willingness to share and publish both versions represents a real commitment to transparent learning) and for allowing space and time for the necessary formative and implementation research (loosely defined for now). moreover, donors can play a role facilitating — financially and otherwise — the necessary time and resources to reflect. without that, statements about commitments to learning among donors are probably more talk than walk. the need for donors to play an active role is isn’t because (all) practitioners and researchers are unaware that they should take time, step back, and reflect. rather, because we all usually need someone or something else to really force us to do so.

    Like

  3. Agree with both of your comments. Donor pressure is important, but we don’t yet know what is the best way to ensure that this external pressure is put in a way (or call it mainstreamed) that retains the spirit of learning and iteration. And that’s what a lot of my blog is about – about how there is an sector-wide tendency to subvert these good ideas. This is not always done intentionally, but is probably an organisational impulse to find a mechanical solution that can be replicated and compiled in a list, etc.

    So yes, I agree that donor money is key in creating space for it – and as I say in my blog towards the end, timelines are critical too. Programmes and projects still take off with very restrictive timelines, for various reasons – spending targets, etc. And when these targets have to be ‘managed’ and ‘outputs’ reported, it squeezes out the time available for learning and reflection. Accountability suffers automatically, not in the least because in such situations, the entire chain (from the donor to the end-implement) are complicit in finding ways to ‘just do enough’.

    (I am going to be provocative here), but do you realise that in a sense, both of you are suggesting quite a top-down way to bring about this shift towards ‘learning and accountability’ in the sector? I don’t quite believe in a top-down/bottom-up dichotomy, but we need to approach this from both sides. It is a bit like what we (Heather and I) explored in one our blogs on decision-making – on the question of who should enforce ‘accountability in decision-making’ https://suvojitchattopadhyay.wordpress.com/2014/06/10/enforcing-accountability-in-decision-making/

    Like

  4. I do realize that I am suggesting a top-down push towards learning and accountability. Or, rather, to ensuring it (which may mean more negotiation between donors and specific organizations about how that org wants to structure its learning and how the donor can help facilitate and ensure). For two reasons. 1. If donors are going to embrace a learning agenda, then they need some ideas about how to actually actualize that. I hope I have provided a few. 2. Learning and reflective documentation are something many organizations may want to do… but something I suspect less are doing. Maybe they take a one-day retreat and think about things but what happens on retreat too often stays there. Perhaps I am projecting based on my want-to-do and actually-done lists but sometimes you need external pressure to just get on with some of the things you want to do anyway. To be overly simplistic, I suspect that a lot of pressure from “below” is to *do* more. So, the pressure to *stop and think* may have to come from elsewhere — and “above” may make some sense. How to have this all not fall into satisficing is an important question to take up later.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s