Is Liberia’s PSL pilot headed for failure?

I have previously written about how Liberia deserves our support as it experiments to find fixes to its currently-struggling education sector. The Partnership Schools for Liberia (PSL) covering about 120 schools will generate evidence, and provide decision-makers in Liberia with the tools to iterate, and reform their largely dysfunctional schooling system. The pilot is accompanied by an independent impact evaluation, carried out by researchers, Mauricio Romero, Justin Sandefur, and Wayne Aaron Sandholtz.

Well, the initial results are in. The main findings from the first round of data collection (one year into the pilot) are:

  1. Student learning outcomes improved significantly (equivalent to 0.6 extra years in school, or 60% more learning) in schools except those providers who were visibly absent from the programme.
  2. Teachers spent more time in the classroom (time-on-task improved significantly); parent and student satisfaction has improved.
  3. Questionable management practices, most prominently in schools operated by Bridge International Academies (hereafter BIA or Bridge). Wider issues of compliance with contract conditions were found, and for some reason, BIA that had unique contracting arrangements remained a part of this evaluation.
  4. Private operators have brought in significant amounts of additional funding to top-up the government grant of $50 per child. Even as it looks fiscally unviable, it may be worth waiting for more evidence.

It is important to note that while schools were randomly allocated to private providers, the entire sample of schools selected for the pilot is not quite the ‘average’ Liberian school. The schools selected were typically better resourced, and had better road and internet connectivity in some cases. The government provided a selected from amongst its best teachers to these schools.

Even in this favourable scenario, the learning gains registered are substantial. The change in management of schools seems to have helped. Private providers and their modified pedagogical methods have yielded learning outcomes. At the same time, how the schools function is directly determined by how they are run, and if private providers are seen to be functioning in ways that hurt the wider schooling system, the chances of their being adopted and scaled up are slim. At the very least, there will be a popular blowback from any such scale-up. Further, the biggest private provider in this pilot, BIA, by sacking teachers, and enforcing class-size caps, indulged in practices that potentially hurt poor children, and negatively impacted the quality of education delivered to children in schools outside the PSL pilot.

As the researchers explain in their paper:

Large-scale dismissal of teachers was unique to one operator (Bridge International Academies), but successful lobbying for additional teachers was common across several operators. This contradicts a core selling point of PSL as we understood it, which was that the program would improve the management and training of government teachers, rather than replace them. Although weeding out bad teachers is important, a reshuffling of teachers is unlikely to raise average performance in the system as a whole… 

and…

…While enrollment increased across all contractors, the smallest treatment effect on this margin is for Bridge, which is consistent with that contractor being the only one enforcing class size caps…in classes where class-size caps were binding (10% of all classes holding 30% of students at baseline), enrollment fell by 20 students per grade. This issue was mostly restricted to Bridge International Academies

While we keep in mind that these are early results, these concerns are important, and makes one worry that the pilot is headed to fail. Surely, some critics will react with “I-told-you-so”, but the nuances of PPP implementation and the mechanisms that yield improvements in learning outcomes merit investing in an evaluation such as this one. So here are a few related questions, to which I hope we will have some answers over the coming months:

  • State capacity to run PPPs: Is there scope for genuine partnerships between the public and private sector? Enforcing contracts isn’t easy for the best of governments, and this is Liberia. If we accept that the government is weak at delivering education outcomes on its own, can it play the role of an effective regulator? How do we work with governments to enhance its capacity to regulate better?
  • Learning: At the end of this pilot, we may well conclude that this is too expensive to scale-up. Private providers would withdraw, citing financial reasons. What then? What lessons can the Ministry of Education in Liberia learn from this pilot? What changes are they making to the way other schools in the country are run? What would they do differently now, and what could they be doing differently if they eventually decide that this programme will not be scaled up?

    Capture
    Cost-effectiveness comparisons (from the paper)
  • Resourcing: Linked to the above – if Liberia decides that the programme delivered cost-effective results, what are the resources at its disposal? What are the domestic revenues that can be allocated to this, and how much of foreign aid would they be able to attract? Have donors made any commitments? Is there an ongoing dialogue?
  • Finally, on Bridge: Even the government now seems to acknowledge that BIA, by sacking teachers, and enforcing class-size caps, indulged in practices that potentially hurt poor children, and negatively impacted the quality of education delivered to children in schools outside the PSL pilot. If Bridge were subject to the same conditions that other providers had to work under, and the private players at large, could not raise additional financing to subsidise their management models, would they be able to deliver learning gains of the magnitude we see in these early results? And if they do not subscribe to the same conditions that apply on the other providers, what is the benefit of retaining their schools in this evaluation?
Advertisements

2 Replies to “Is Liberia’s PSL pilot headed for failure?”

  1. One question no one seems to ask – why did Bridge and other providers remove/transfer some teachers? As I understand it, it’s because many were functionally Illiterate. I don’t see how removing such teachers from classrooms can be seen as “questionable business practices”.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s