EDITORIALS
Health systems performance assessment in the Bulletin
Ties BoermaI,*; Mickey ChopraII; David EvansIII
IDepartment of Health Statistics and Informatics, World Health Organization, 20 avenue Appia, 1211 Geneva 27, Switzerland
IIMedical Research Council of South Africa, Cape Town, South Africa
IIIDepartment of Health Systems Financing, World Health Organization, 20 avenue Appia, 1211 Geneva 27, Switzerland
In 2000, WHO published its first attempt to assess the performance of the world's health systems in The world health report 2000.1 This report generated enormous interest but, in many ways, the scientific progress was overshadowed by the political debate related to the estimates of country-level performance and the associated league tables. Since then, the WHO European and Eastern Mediterranean Regional Offices have maintained health system observatories, with detailed descriptions of country systems. The considerable interest in measuring the performance of health systems worldwide is illustrated by the recent European Ministerial Conference on Health Systems, which culminated in the Tallinn Charter entitled Health systems for health and wealth.2 In developed countries, primary concerns include costs, quality of care, aging and chronic diseases. In developing countries, health system constraints have restricted progress towards the UN Millennium Development Goals.3
The methods and metrics to assess progress and performance of specific health programmes are often well established. This is less so when trying to assess the extent to which health systems achieve their goals. When strengthening health systems and establishing the most cost-effective ways of delivering care, there is a need for a basic set of indicators of health system functions and of scientifically sound, practical and user-friendly tools.
An increasing number of countries conduct self-assessments of their performance, and these are mostly performed by government-funded research institutes in partnership with academic institutions. Comparisons are made with other countries, efficiency is examined by comparing outputs and outcomes to inputs. Given the investment, what are the results?
In the United States of America, for example, 37 indicators were selected to assess performance in the domains of long, healthy and productive lives, quality, access, efficiency and equity.4,5 The indicators were benchmarked against best performers, typically those achieved by the top 10% of countries, states, health plans, hospitals and other providers. In England, considerable investments are being made in assessing performance of the National Health Service through monitoring a large set of indicators with targets.6 In Canada, the emphasis is on developing a monitoring system for primary health care performance that focuses on population-based data sources.7,8 The Netherlands publishes a bi-annual Dutch health care performance report, focusing on quality, access and costs using more than 100 indicators.9
Indonesia conducted a comprehensive within-country assessment using WHO's health systems performance assessment framework, as part of its Healthy Indonesia 2010 policy.10 In South Africa, the Health Systems Trust has now published three editions of its district health barometer, which monitors about 20 indicators.11 In addition to comparisons across districts, metropolitan areas and parts of the country considered to be severely disadvantaged were included. In Afghanistan, the rapid expansion of health services was monitored using a balanced scorecard that focused on service delivery through a comprehensive facility survey.12 In Mexico, a report card for all states was used to assess the effects of the health system reforms during 2001-2006, including a summary measure based on 11 indicators derived from a variety of clinical- and population-based data sources.13
Methodological progress has been made in key components of health systems performance assessment, such as responsiveness, catastrophic expenditure, equity and effectiveness, but less progress has been made in terms of summary outcome measures. Ways to measure efficiency have received considerable attention and more sophisticated econometric methods were developed in response to the methods used in The world health report 2000.14 However, the debate is by no means resolved. Moreover, the question of deciding the appropriate outcome measure for use in efficiency analysis, or simply to summarize overall health system performance, is still under debate. A variety of summary measures have been proposed ranging from life expectancy and avoidable mortality to complex measures combining multiple dimensions of health system goals, including financial fairness, responsiveness, mortality and morbidity.
In addition to the challenges of developing indicators and analytical methods, the actual availability and quality of core data on basic health system building blocks, such as financing, human resources, medicines and service delivery, remains a challenge in low-income countries. WHO, The World Bank and partners have proposed a core set of indicators and measurement approaches and ways to address these information gaps (available at: http://www.who.int/healthinfo/statistics/toolkit_hss/en/index.html).
The field of health systems performance assessment is closely related to health systems research and evaluation. Increased demand for accountability and demonstration of results mean that evaluation research is also receiving much more attention. Evaluation of the different ways of scaling up health programmes, and of relaxing key health systems constraints through global health partnerships or other initiatives, is likely to yield information that is highly relevant to the assessment of the performance of health systems.
The Bulletin invites papers on health systems performance assessment. Policy papers may include country applications, results and implications of the assessment for policy, while research papers should address issues related to the measurement and analysis of health systems performance. 
References
Available at: http://www.who.int/bulletin/volumes/87/1/08-061945/en/index.html
References
1. World health report 2000: health systems: improving performance. Geneva: World Health Organization; 2000.
2. The Tallinn charter: health systems for health and wealth. In: WHO European Ministerial Conference on Health Systems, Tallinn, Estonia, 25-27 June 2008.
3. Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, et al. Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet 2004;364:900-6. doi:10.1016/S0140-6736(04)16987-0 PMID:15351199
4. Schoen C, Davis K, How SKH, Schoenbaum SC. US Health systems performance: a national scorecard. Health Aff 2006;25:w457-75. doi:10.1377/hlthaff.25.w457
5. Why not the best? The Commonwealth Fund Commission on a high performance health system. New York: The Commonwealth Fund; 2006.
6. Lakhani A, Coles J, Eayres D, Spence C, Rachet B. Creative use of existing clinical and health outcomes data to assess NHS performance in England: Part I- performance indicators closely linked to clinical care. BMJ 2005;330:1426-31. doi:10.1136/bmj.330.7505.1426 PMID:15961815
7. Esmail N, Walker M. How good is Canadian health care? 2006 report. Vancouver: Fraser Institute; 2006.
8. Centre for health services and policy research. Measuring the performance of primary health care: existing capacity and future information needs. Vancouver: University of British Columbia; 2006.
9. Westers G, Verkleij H, eds. Dutch health care performance report. Bilthoven: RIVM. 2006.
10. National Institute of Health Research and Development. Indonesia: subnational health system performance assessment. Jakarta: Ministry of Health; 2005.
11. Barron P, Day C, Monticelli F. The district health barometer 2006/07. Durban: Health Systems Trust; 2007.
12. Hansen PM, Peters DH, Niayesh H, Singh LP, Dwivedi V, Burnham G. Measuring and managing progress in the establishment of basic health services: the Afghanistan health sector balanced scorecard. Int J Health Plann Manage 2008;23:107-17. doi:10.1002/hpm.931 PMID:18435428
13. Gakidou E, Lozano R, González-Pier E, Abbott-Klafter J, Barfosky JT, Bryson-Cahn JC, et al. Assessing the effect of the 2001-06 Mexican health reform: an interim report card. Lancet 2006;368:1920-35. doi:10.1016/S0140-6736(06)69568-8 PMID:17126725
14. Greene W. Distinguishing between heterogeneity and inefficiency: stochastic frontier analysis of the World Health Organization's panel data on national health care systems. Health Econ 2004;13:959-80. doi:10.1002/hec.938 PMID:15455464
* Correspondence to Ties Boerma (e-mail: boermat@who.int).
doi:10.2471/BLT.08.061945