ROUND TABLE
DISCUSSION
Looking at the effects of performance-based financing through a complex adaptive systems lens
Jean Macq*; Jean-Christophe Chiem
Université Catholique de Louvain, 1 Place de l'Université, B-1348, Louvain-la-Neuve, Belgium
The debate on PBF is misdirected. As is too often the case in international aid financing, agencies try to prove the effectiveness of their contribution by isolating it as the main reason for success.1 In reaction, opponents will often use the same approach in an attempt to prove that another factor is actually the cause of an observed change. We argue that this endless and futile debate, often present among experts in health systems strengthening, will not contribute to improving public health in low-income countries.
Rather than searching for the impossible proof of whether PBF works or not, we should instead try to learn useful lessons from experiences. We agree with Ireland et al. that the focus of PBF assessment should be on "why" and "how" the intervention works.2 Comprehensive evaluation of PBF is needed as part of complete health system reform.
We think that, to respond to some of these key questions, health systems should be analysed using a complex adaptive systems lens, as others have advocated in the past.3,4 A complex adaptive system is a collection of interacting components, each of which has its own rules and responsibilities. The behaviour of this kind of system is different to the sum of the behaviour of each of its components. Examples of complex adaptive systems include the human brain, ecosystems and manufacturing businesses.
Health system "behaviour" and particularly counterintuitive behaviour (unexpected changes or lack of change) can be analysed using a complex adaptive systems lens when PBF is introduced, often with a mix of other interventions such as in a context of system reform. The purpose of this analysis is not to isolate causal factors but rather to identify "macro" characteristics of the system that may explain behaviour change.
Although it has often been ignored in health system evaluation, social simulation can be useful for this approach. The most frequently used technique, agent-based modelling, uses computer simulation centred on a collection of autonomous agents whose interactions are based on a set of rules. These simulations can integrate empirical data or existing knowledge or opinions.5 One of the powerful features of agent-based modelling lies in its capacity to study complex phenomena in a simple and flexible way. Indeed, this approach does not require a high level of mathematical or programming skills, making it accessible to many researchers. Furthermore, it allows for an iterative learning process that is easy to set up compared to long and costly data collection processes.
While this methodological approach may not "prove" the effectiveness of an intervention, it could provide insight into the reason a health system behaves in a given way (whether it changes or remains in a steady-state) when PBF is introduced. We believe that this type of information, although maybe less appealing to the usual stakeholders in development aid debates, is much more useful in evaluating PBF. 
Competing interests: None declared.
References
1. Earl S, Carden F, Smutylo T. Outcome mapping: building learning and reflection into development programs. Ottawa: International Development Research Centre; 2001.
2. Ireland M, Paul E, Dujardin B. Can performance-based financing be used to reform health systems in developing countries? Bull World Health Organ 2011;89:695698.
3. Sterman JD. Learning from evidence in a complex world. Am J Public Health 2006;96:50514. doi:10.2105/AJPH.2005.066043 PMID:16449579
4. Plsek PE, Greenhalgh T. Complexity science - the challenge of complexity in health care. BMJ 2001;323:6258. doi:10.1136/bmj.323.7313.625 PMID:11557716
5. Bandini S, Manzoni S, Vizzari G. Agent based modeling and simulation: an informatics perspective. J Artif Soc Soc Simul 2009;12:4.
* Correspondence to Jean Macq (e-mail: jean.macq@uclouvain.be).