EDITORIALS
Time for innovative dialogue on health systems research
Bruno MeessenI; Godelieve van HeterenII,*; Robert SoetersIII; Gyuri FritscheIV; Wim van DammeI
IInstitute of Tropical Medicine, Antwerp, Belgium
IIRotterdam Global Health Initiative, Erasmus University-iBMG, PO Box 1738, 3000 DR Rotterdam, Netherlands
IIISina Health International, The Hague, Netherlands
IVThe World Bank, Washington, United States of America
A forthcoming report by the World Health Organization (WHO) on evidence for policy is already fuelling passionate debate, especially among scientists. Opinions are divided: some scientists believe strongly in the systematic assessment of evidence on the effectiveness of health system interventions and reforms; 1 others are skeptical. 2, 3
This is not just another academic debate. Health policy is a noisy field involving many stakeholders. Health reform approaches that seem promising in practice are assessed critically by institutional researchers, whose methodological dictums are sceptically received by field practitioners. In a recent Bulletin editorial, for example, Fretheim et al. reported the results of their Cochrane review of performance-based financing studies, which yielded few studies meeting Cochrane criteria. When the authors concluded that little could be said about performance-based financing, 4,5 advocates of this approach responded with an avalanche of critical commentaries. 6
These debates are typical, necessary and energizing. They hail a new era in the worldwide scrutiny of health policy and its knowledge architecture. As legislators, researchers, knowledge managers and contributors to health reform, we welcome the deepening interest in the intricacies of health reform and acknowledge the global relevance of an ambitious new knowledge agenda.
A balanced dialogue between different knowledge holders - researchers, practitioners and policy-makers - is, in our view, a conditio sine qua non for successful health policies. It concerns us all to consolidate the body of evidence surrounding promising approaches, including data on their possible pitfalls and side-effects. For such a dialogue to take off and become truly global, the following prerequisites should be fulfilled.
All contributors should realize that the knowledge agenda for successful reforms extends beyond their own niches. 7 Serious knowledge programmes for improving health systems in low- and middle-income countries go well past merely scrutinizing evidence, and certainly outcome evidence. Our own experience has taught us that most schemes anywhere fail because of poor design, insufficient funding, weak governance and muddled implementation. These dimensions of the health system development process warrant proper attention from researchers. In addition to the hierarchy for the strength of the evidence, there should also be a hierarchy for practical relevance.
We should revisit current evaluation methods or at least their interpretation. The broad transformative character of many health system reforms, such as performance-based financing, poses challenges for evaluation methodology. 8 Promising features of health reforms can stem from dimensions not amenable to randomization, such as professional culture and accountability; research methods limited to one or two outcome parameters are too reductionist.
Scientists should always strive for rigour in evaluating reform interventions. The Cochrane standards, for instance, provide useful guidance for better design and reporting of primary studies. However, health reform is not a simple intervention: context matters a lot and contents resist homogenization. Health sector reform is often about modifying institutional arrangements, which differ considerably across countries and settings. Hence, the findings in one country will frequently have limited applicability in another. Researchers seeking to conduct impact studies or synthesize their findings should fully familiarize themselves with local contexts, stakeholder development dynamics and implementation strategies in the field.
We should be aware that nesting an impact evaluation within a comprehensive health reform effort is not a neutral operation. It affects the policy process and may exhaust or undermine its momentum. Ironically, the research may produce systemic outcomes not captured by the randomized design. These should be meticulously documented.
Finally, we urgently need to think about where to conduct the interactions on the knowledge agenda for better health policies. Engaging environments are required for open and productive communication across professional boundaries. A new research dialogue cannot be staged exclusively in scientific journals. Communities of practice, such as those affiliated with Harmonization for Health in Africa, gather large numbers of people with fair distribution of input and experiential knowledge. While cohesive and animated, they build trust and allow for critical collaborative dynamics, desperately needed by health systems research. 9
In the end, any dialogue in a tortuous policy process also requires realism. At the country level, health decision-makers face huge uncertainties. The windows of opportunity are frequently unrelated to the rigour of any evidence available. The recent nationwide roll-out of a new treatment approach for pregnant mothers infected with the human immunodeficiency virus (HIV) in Malawi was based on expert opinion (and advocacy), not a randomized controlled trial. 10 Similarly, the Burundian government may not wish to wait for a four-year study to end if it senses promise in a strategy that can meet pressing health needs today. All the more reason to innovate researchers' dialogue with parties in the thick of it. 
References
1. Lewin S, Bosch-Capblanch X, Oliver S, Akl AE, Vist GE, Lavis JN et al. Guidance for evidence-informed policies about health systems: assessing how much confidence to place in the research evidence. PLoS Med 2012;9:e1001187. doi: 10.1371/journal.pmed.1001187 PMID: 22448147
2. Peters DH, Bennett S. Better guidance is welcome, but without blinders. PLoS Med 2012;9:e1001188. doi: 10.1371/journal.pmed.1001188 PMID: 22448148
3. Humphreys K, Piot P. Scientific evidence alone is not sufficient basis for health policy. BMJ 2012;344:e1316. doi: 10.1136/bmj.e1316 PMID: 22371864
4. Fretheim A, Witter S, Lindahl AK, Olsend T. Performance-based financing in low- and middle-income countries: still more questions than answers Bull World Health Organ 2012;90:559-559A. doi: 10.2471/BLT.12.106468 PMID: 22893735
5. Witter S, Fretheim A, Kessy FL, Lindahl AK. Paying for performance to improve the delivery of health interventions in low- and middle-income countries Cochrane Database Syst Rev 2012;2:CD007899. PMID: 22336833
6. Health Financing in Africa, Le Blog [Internet]. An online debate on "Performance-based financing in low- and middle-income countries: still more questions than answers". Meessen B, editor. Harmonization for Health in Africa. Available from: http://www.healthfinancingafrica.org [accessed 4 September 2012] .
7. Jansen MW, Van Oers HAM, Kok G, De Vries NK. Public health: disconnections between policy, practice and research. Health Res Policy Syst 2010;8:37. doi: 10.1186/1478-4505-8-37 PMID: 21194428
8. Meessen B, Soucat A, Sekabaraga C. Performance-based financing: just a donor fad or a catalyst towards comprehensive health-care reform? Bull World Health Organ 2011;89:153-6. doi: 10.2471/BLT.10.077339 PMID: 21346927
9. Meessen B, Kouanda S, Musango L, Richard F, Ridde V, Soucat A. Communities of practice: the missing link for knowledge management on implementation issues in low-income countries? Trop Med Int Health 2011;16:1007-14. doi: 10.1111/j.1365-3156.2011.02794.x PMID: 21564426
10. Schouten EJ, Jahn A, Midiani D, Makombe SD, Mnthambala A, Chirwa Z et al. Prevention of mother-to-child transmission of HIV and the health-related Millennium Development Goals: time for a public health approach. Lancet 2011;378:282-4. doi: 10.1016/S0140-6736(10)62303-3 PMID: 21763940
* Correspondence to Godelieve van Heteren (e-mail: vanheteren@bmg.eur.nl)