EDITORIALS

Performance-based financing in low- and middle-income countries: still more questions than answers

Atle Fretheim^I,*; Sophie Witter^II; Anne Karin Lindahl^III; Ingvar Theo Olsen^IV

^INorwegian Knowledge Centre for the Health Services, PO Box 7004, St Olavs plass, Oslo, 0130, Norway
^IIImmpact, University of Aberdeen, Aberdeen, Scotland
^IIINorwegian Knowledge Centre for the Health Services, Oslo, Norway
^IVNorwegian Agency for Development Cooperation, Oslo, Norway

Performance-based financing is promoted as a promising strategy for improving health service delivery and helping to reach the Millennium Development Goals.^1,2 But what is the evidence supporting its use?

Performance-based financing may be defined as "the transfer of money or material goods conditional on taking a measurable action or achieving a predetermined performance target".² We recently conducted a Cochrane review of the available evidence on the effectiveness of performance-based financing in low- and middle- income countries.³ Nine studies, of which fewer than half had been published in scientific journals, fulfilled our inclusion criteria: one randomized controlled trial (RCT) and two interrupted time series conducted in Asia, and six controlled before–after studies conducted in Africa. Only two outcomes related to health care utilization – institutional deliveries and antenatal care – were assessed in more than one trial. Inconsistent results across studies made summarizing and interpreting the evidence difficult.

The most rigorous African study reported a moderate increase in institutional deliveries, from around 35% to 42% (i.e. 7 percentage points; 95% confidence interval, CI: 1–14).⁴ Findings from studies in Burundi,⁵ the Democratic Republic of the Congo⁶ and Rwanda⁷ showed disparate findings: one reported a significant increase in institutional deliveries, another found little or no change and the third showed a significant decrease. For these findings the risk of bias is high, partly because intervention and control areas were not randomly allocated and the same people who implemented the programmes also evaluated them. Two additional studies reporting on institutional deliveries were programme evaluations with a substantial risk of bias due to questionable data quality. The results on antenatal care attendance were also heterogeneous.

These findings clearly show that no general conclusion can be drawn regarding the likely impact of performance-based financing in low-and middle-income countries. For one thing, most of the studies found through the review were methodologically weak and had poor internal validity. Furthermore, since the impact of complex interventions such as performance-based financing depends largely on the context in which they are implemented, results may vary. Finally, the studies differed substantially in the way in which the performance-based financing scheme was designed and implemented.

In our view, minimizing the risk of producing biased results in future performance-based financing evaluations is essential. Since performance-based financing schemes are not likely to have a large impact, effect estimates need to be as reliable and precise as possible to avoid missing real effects or being misled by seemingly positive findings. Thus, trials should be conducted in settings where a sufficiently large number of facilities or geographical areas can be randomized.

Sometimes RCTs are too expensive, time-consuming or impractical. The most recent study of performance-based financing, from Rwanda, illustrates the practical setbacks researchers sometimes face. The study was set up as a well-designed RCT with districts randomly assigned to performance-based financing intervention and non-intervention groups,⁴ but right before the trial began, the government redefined the administrative district boundaries and several of the districts in the control group were shifted to the intervention group. This may have weakened the validity of the study findings.

If conducting an RCT is not feasible, other robust evaluation designs should be considered. One option is an interrupted time series, in which outcome data are collected at regular intervals during baseline and post-intervention periods (e.g. for one year before and after). This type of study, however, normally requires access to reasonably good routine data from hospital records or other reliable sources not always available in low-resource settings.

Rather than focus exclusively on targeted indicators, future performance-based financing evaluations should consider untargeted effects and systems effects by starting with a conceptual framework of how the intervention might work and what the important outcome measures are. Also, study protocols and evaluation plans should be prepared and published as early as possible, preferably before programme implementation. It is of concern that, in a search conducted on 3 April 2012, we found no ongoing performance-based financing trials from low- and middle-income countries in the International Clinical Trials Registry Platform. We hope that more and better evaluations will emerge in future updates of our Cochrane review to enable us to formulate forthright conclusions about the effects of performance-based financing in low- and middle-income countries.

Competing interests: ITO is employed by the Norwegian Agency for Development Cooperation (Norad), which funds performance-based financing projects in several countries. Also, the Norwegian government is one of the contributors to The World Bank's Health Results Innovation Trust Fund (HRITF).

References

1. Meessen B, Soucat A, Sekabaraga C. Performance-based financing: just a donor fad or a catalyst towards comprehensive health-care reform? Bull World Health Organ 2011;89:153–6. PMID:21346927        

2. Eichler R. Can "pay for performance" increase utilization by the poor and improve the quality of health services? Discussion paper for the first meeting of the Working Group on Performance-Based Incentives. Washington: Center for Global Development; 2006. Available from: PBI%20Background%20Paper[1].pdf [accessed 21 June 2012]          .

3. Witter S, Fretheim A, Kessy FL, Lindahl AK. Paying for performance to improve the delivery of health interventions in low- and middle-income countries. Cochrane Database Syst Rev 2012;2:CD007899. PMID:22336833        

4. Basinga P, Gertler PJ, Binagwaho A, Soucat AL, Sturdy J, Vermeersch CM. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet 2011;377:1421–8. PMID:21515164        

5. Soeters R, Kimanuka C. Rapport de l'Etude d'Evaluation du programme Achat de Performance dans les Provinces Bubanzaet Cankuzo, basé sur les résultats des enquêtes ménages, qualité et infirmiers titulaires réalisées en 2006 et 2008.The Hague: Cordaid; 2009. French. Available from: http://www.epistemonikos.org/fr/documents/bfed0db792341e6f8528a61a0b38f8aa564ae001 [accessed 21 June 2012]          .

6. Soeters R, Kimanuka C. Résultats de l'enquête ménage, l'enquête qualité, et l'enquête infirmiers titulaires. Pour le Programme Achat de Performance dans les Zones de Santé du District Sanitaire Nord du Sud Kivu 2006–2009. The Hague: Cordaid; 2008. French. Available from: Evaluation-PBF-South-Kivu-DRCongo-(Feb-2008)-(FR)[1].pdf [accessed 21 June 2012]          .

7. Soeters R, Musango L, Meessen B. Comparison of two output based schemes in Butare and Cyangugu provinces with two control provinces in Rwanda. Global Partnership on Output-Based Aid & World Bank & Ministry of Health of Rwanda; 2005. Available from: http://www.multicountrypbfnetwork.org/Comparison.pdf [accessed 21 June 2012]          .

* Correspondence to Atle Fretheim (e-mail: atle.fretheim@nokc.no).