Editorial
Health systems performance whats next?
Suwit Wibulpolprasert1 & Viroj Tangcharoensathien1
The World Health Report 2000 health systems: improving performance, published a year ago this June, received mixed reactions from policy-makers and academics around the world. One of the most controversial points is its attempt to rank the health systems of 191 countries by relating goals attained to resources spent. Not surprisingly, there were strong criticisms about the way performance was evaluated. Widespread discussion ensued, on matters not only of international public health but of local politics. Whatever the rights and wrongs involved, the report has drawn a great deal of attention to the importance of health systems.
The report draws the operational boundary of the health system to include the resources, actors, and institutions related to the financing, regulation, and provision of actions whose primary purpose is to improve or maintain health. Within this boundary, three sets of intrinsic goals are formulated as the main basis for health system assessment. They are health itself, responsiveness to demand for health services, and fairness in health financing. Accordingly, the main indicators for WHO in assessing health system performance are the level and distribution of health, the level and distribution of responsiveness, and the distribution of cost. For the purposes of global comparison of performance, the composite measure of health system goal attainment is calculated from these five indicators. The performance index for each goal as well as the composite measure is then evaluated by comparing them with the level of resources invested.
The point of health system performance assessment is definitely not just to compare countries by looking at their position on the table at the end of the report. It is to monitor the status of health goals in countries in relation to resources spent. In this way the assessment can be a diagnostic tool to evaluate policy reform and facilitate evidence-based decision-making.
However, it is not an easy task, especially in the case of a global evaluation. The measurement has to be valid, reliable, sensitive and specific. Furthermore, it would be preferable if it could be made without substantial needs for additional resources or data sets.
In translating a conceptual framework into operational measurement, several constraints are encountered, both conceptually and operationally. For example, there is no agreement about the best method for measuring health attainment or level of responsiveness. Methods for measuring goal attainment are still very limited. Responsiveness measurement requires significant methodological improvements. In fact each method has limitations in terms of its validity or reliability.
The problem is not only with measurement. The methodology used for health goal evaluation contains normative value judgements, especially in the case of weighting domains or goals and the preferred distribution pattern for health, responsiveness, and financing. This is a matter of serious debate, especially about what is right or fair. Also, there may be great differences between the preferences or norms of developed and developing countries, for instance in the weighting assigned to health, responsiveness, and fairness in financing. Within the same country, remarkable differences in perception were observed between those with higher and lower levels of education.
Many secondary data sets are unavailable or unreliable. Primary data collection for evaluation purposes is expensive and time-consuming. Owing to these limitations, several estimation techniques were used, which in their turn make the result less convincing.
Given the multifactoral nature of health determinants and the existing inequitable infrastructure within any given country, we believe that in the short run there can be no rapid improvement of health system performance, especially in the level and distribution of health and in the fairness of financing. Hence, there is no need to make performance assessments frequently. Improvement in the level and distribution of responsiveness could, however, be a promising entry point for actions, as it is within the control of the health system and can benefit from a strong civil society movement.
As noted, there is room for improvement in performance assessment. Open discussions, constructive criticism and participation by member states would ensure the process of improvement. This would not only give the analysts in WHO access to recent data from reliable sources, but strengthen local capacity for health system evaluation and knowledge-based health system development.
WHO should encourage all its member states to monitor their own health system performance regularly. This would help to ensure their ownership and use of the evaluation, the inclusion of national trend assessment and subnational variations, and the formulation of policy which fits the local context. Furthermore, the health system is an aggregate of several subsystem components such as human resources, hospitals and public health programmes. An understanding of the performance of various key components will make policy recommendations more specific. For example, the performance of the hospital sector, which takes up more than half of national health resources, should be rigorously assessed. Several key public health programmes such as HIV/AIDS prevention could be assessed for their level of financing on prevention and care, and for equity of access to preventive services. WHO technical cooperation and capacity-strengthening within countries, to devise tools for subsystem performance assessment, are highly recommended.
The World Health Report 2000 has significantly increased awareness of health systems. It also exhibits an innovative way to evaluate their goals and performance. There is still room for plenty of methodological improvement at both the conceptual and the operational level, and the involvement of all those concerned in the use of information for health systems improvement. There is no doubt that strong policy support should be given to this effort. Current weaknesses should not impede this very important initiative. 
Methods for measuring health system performance are currently the subject of lively debate. The Bulletin would like to publish a group of research and policy articles on this subject in early 2002. Contributions (subject, of course, to our normal process of peer review) will be welcomed. They should be addressed to the Editor, and reach us by 28 September 2001.
1 Ministry of Public Health, 5th floor, 1st building, Tiwanond Road, Nonthaburi 11000, Thailand
(email: suwit@health.moph.go.th).
Ref. No. 01-1328