THEME PAPERS

A framework for assessing the performance of health systems

Un cadre pour l'évaluation de la performance des services de santé

Un marco para evaluar el desempeño de los sistemas de salud

Christopher J.L. Murray^I; Julio Frenk^II

^IDirector, Global Programme on Evidence for Health Policy, World Health Organization, 1211 Geneva 27, Switzerland
^IIExecutive Director, Evidence and Information for Policy, World Health Organization, Geneva, Switzerland

Correspondence

ABSTRACT

Health systems vary widely in performance, and countries with similar levels of income, education and health expenditure differ in their ability to attain key health goals. This paper proposes a framework to advance the understanding of health system performance. A first step is to define the boundaries of the health system, based on the concept of health action. Health action is defined as any set of activities whose primary intent is to improve or maintain health. Within these boundaries, the concept of performance is centred around three fundamental goals: improving health, enhancing responsiveness to the expectations of the population, and assuring fairness of financial contribution. Improving health means both increasing the average health status and reducing health inequalities. Responsiveness includes two major components: (a) respect for persons (including dignity, confidentiality and autonomy of individuals and families to decide about their own health); and (b) client orientation (including prompt attention, access to social support networks during care, quality of basic amenities and choice of provider). Fairness of financial contribution means that every household pays a fair share of the total health bill for a country (which may mean that very poor households pay nothing at all). This implies that everyone is protected from financial risks due to health care. The measurement of performance relates goal attainment to the resources available. Variation in performance is a function of the way in which the health system organizes four key functions: stewardship (a broader concept than regulation); financing (including revenue collection, fund pooling and purchasing); service provision (for personal and non-personal health services); and resource generation (including personnel, facilities and knowledge). By investigating these four functions and how they combine, it is possible not only to understand the proximate determinants of health system performance, but also to contemplate major policy challenges.

Keywords: outcome and process assessment, health care; health care rationing; health services accessibility; social justice; health systems plans; financing, health.

RÉSUMÉ

Les systèmes de santé présentent de grandes différences de performance et des pays ayant un niveau équivalent de revenu, d'éducation et de dépenses de santé peuvent faire preuve d'une capacité différente à atteindre leurs buts principaux en matière de santé. Cet article propose un cadre permettant de mieux connaître la performance des systèmes de santé. Ce cadre se compose de quatre grands éléments: préciser les limites des systèmes de santé; identifier les buts des systèmes de santé; définir le concept de performance; et comprendre les fonctions clés des systèmes de santé en tant que déterminants directs de la performance.

Les limites du système de santé sont basées sur le concept d'action sanitaire, que l'on peut définir comme un ensemble d'activités dont l'intention première est d'améliorer ou de maintenir la santé. Ce critère conduit à une définition large du système de santé, qui inclut par exemple les efforts visant à améliorer la sécurité routière, y compris celle des véhicules, lorsque l'intention première est de réduire les accidents de la circulation; il inclut également les services de santé personnels, qu'ils contribuent ou non à la santé. L'un des principaux avantages de ce critère est qu'il fait intervenir dans l'évaluation de la performance du système de santé tous les acteurs et institutions dont le but premier est de contribuer à la santé.

A l'intérieur de ces limites, le concept de performance s'articule autour de trois buts fondamentaux: améliorer la santé, augmenter la capacité à répondre aux attentes de la population, et assurer l'équité de la contribution financière. Le premier but, améliorer la santé, est le principe fondateur du système de santé. Il comporte deux aspects: améliorer l'état de santé moyen, et réduire les inégalités en matière de santé.

Le deuxième but, augmenter la capacité à répondre aux attentes de la population, comporte deux éléments majeurs: le respect des personnes (dignité, confidentialité, droit des personnes et des familles à disposer de leur propre santé), et orientation client (attention immédiate, accès à des réseaux de soutien social pendant les soins, qualité des éléments de base du service et choix du prestateur de soins).

Pour atteindre le troisième but, assurer l'équité, le financement du système de santé doit relever deux défis majeurs. D'abord, les ménages ne doivent pas tomber dans la pauvreté ni dépenser une part excessive de leur revenu pour accéder aux soins de santé dont ils ont besoin. En d'autres termes, l'équité de la contribution financière exige une importante mise en commun des risques financiers. Ensuite, les ménages pauvres devraient payer moins pour le système de santé que les ménages riches. On arrive ainsi à une proposition normative selon laquelle la participation de chaque ménage au coût du système de santé devrait être équitable. (Dans le cas des ménages très pauvres, le terme «participation équitable» pourrait signifier la gratuité des soins). Le paiement devrait être basé sur le revenu et non pas tenir compte de l'utilisation des services ni des risques financiers. Cela implique que chacun soit protégé contre les risques financiers associés aux soins de santé.

En disposant d'un ensemble clairement défini de buts et de moyens de les mesurer, il est possible de comparer leur niveau de réalisation pour différents systèmes de santé. Le concept de performance va cependant au-delà du simple enregistrement du niveau de réalisation d'un but. Il s'agit de relier le niveau déjà atteint à celui qui pourrait être atteint. En d'autres termes, le concept de performance est un concept relatif. La mesure de la performance établit une relation entre le niveau atteint et les ressources du système de santé.

Tout essai systématique d'évaluation de la performance d'un système de santé doit comprendre l'étude des facteurs qui peuvent y contribuer. La performance est fonction de la manière dont le système de santé organise quatre fonctions clés: l'administration générale (un concept plus large que celui de régulation), le financement (perception des recettes, mise en commun des fonds et achats), la fourniture de services (pour les services de santé personnels et non personnels) et la création de ressources (personnel, équipements et savoirs). En examinant ces quatre fonctions et la façon dont elles se combinent, il est possible non seulement de connaître les déterminants directs de la performance des systèmes de santé, mais également de pouvoir envisager des défis majeurs en matière de politiques de santé.

Un tel cadre pour l'évaluation de la performance des systèmes de santé est déjà utilisé. L'une de ses principales applications sera de structurer les annexes statistiques du Rapport sur la santé dans le monde. Dès l'an 2000, le rapport présentera des informations sur la performance du système de santé de chaque pays. Ce type d'exercice aidera les pays, dans le monde entier, à mieux répondre aux besoins sanitaires complexes et changeants de leur population.

RESUMEN

El desempeño de los sistemas de salud varía ampliamente, y países con niveles similares de ingresos y de gastos en educación y salud pueden diferir en su capacidad de alcanzar las principales metas sanitarias. En este artículo se propone un marco para profundizar en el conocimiento del desempeño de los sistemas de salud. Este marco se compone de cuatro elementos principales: especificación de los límites de los sistemas de salud; identificación de las metas de los sistemas de salud; definición del concepto de desempeño; e interpretación de las funciones clave de los sistemas de salud como determinantes inmediatos del desempeño.

Los límites del sistema de salud están determinados por el concepto de acción sanitaria, que puede definirse como cualquier conjunto de actividades cuyo fin principal sea la mejora o el mantenimiento de la salud. Este criterio del fin principal conduce a una definición amplia del sistema de salud. Se incluyen en ella las actividades de mejora de la seguridad de las carreteras y los vehículos, cuya finalidad principal es reducir el número de accidentes de tráfico; se incluyen asimismo servicios de salud personal, contribuyan o no a la salud. Una importante ventaja del criterio del fin principal es que incluye en la evaluación del desempeño del sistema de salud a todos los actores e instituciones cuyo propósito principal es contribuir a la salud.

Dentro de esos límites, el concepto de desempeño gira en torno a tres metas fundamentales: mejorar la salud, acrecentar la capacidad de respuesta a las expectativas de la población y asegurar la equidad de la contribución financiera. La primera, mejorar la salud, es la meta definitoria de un sistema de salud, y abarca dos aspectos: aumentar el nivel medio de salud y reducir las desigualdades en salud.

La segunda meta, acrecentar la capacidad de respuesta, abarca dos componentes principales: el respeto a las personas (incluidas la dignidad, la confidencialidad y la autonomía de las personas y familias para tomar decisiones sobre su propia salud); y la orientación del usuario (inclusive la atención pronta, el acceso a redes de apoyo social durante la asistencia, la calidad de los servicios básicos y la posibilidad de elegir al dispensador de atención).

Para lograr el tercer objetivo, la equidad, hay que hacer frente a dos desafíos principales en lo que concierne a la financiación del sistema de salud. En primer lugar, los hogares no deben verse obligados a empobrecerse o a pagar una parte excesiva de sus ingresos para recibir la atención sanitaria necesaria. En otras palabras, para garantizar la equidad en materia de contribución financiera se requiere un grado importante de mancomunación de los riesgos financieros. En segundo lugar, los hogares pobres deberían pagar al sistema de salud menos contribuciones que los ricos. Estas consideraciones se traducen en la propuesta normativa de que todos los hogares paguen una parte equitativa de los gastos sanitarios. (En el caso de los hogares muy pobres, «parte equitativa» significa que queden exentos de pago.) La suma pagada debería depender de los ingresos, y sólo mínimamente del uso de los servicios o del riesgo de enfermedad. Así todo el mundo queda protegido de los riesgos financieros que conlleva la necesidad de atención sanitaria.

Contando con una serie de metas claramente definidas y con la manera de medirlas, podemos comparar el nivel de consecución de las metas en diferentes sistemas de salud. Sin embargo, el concepto de desempeño es más complejo que el simple registro del grado de logro de las metas. Para determinar el desempeño de un sistema de salud hay que relacionar los resultados logrados y los resultados que podrían haberse obtenido; en otras palabras, el desempeño es un concepto relativo. La medición del desempeño relaciona el grado de logro de las metas con los recursos de que dispone el sistema de salud.

Cualquier intento sistemático de comprender el desempeño de los sistemas de salud deberá incluir el estudio de los factores potencialmente determinantes de esa variable. El desempeño dependerá de la manera en que el sistema de salud organice cuatro funciones clave: la rectoría (concepto más amplio que el de reglamentación), la financiación (incluidas la recaudación de ingresos, la combinación de fondos y las compras), la prestación de servicios (de salud personal y no personal) y la generación de recursos (personal, instalaciones y conocimientos). Estudiando esas cuatro funciones y la forma en que se combinan es posible no sólo comprender los determinantes inmediatos del desempeño del sistema de salud, sino también plantearse cambios de política radicales.

Este marco para la evaluación del desempeño de los sistemas de salud se seguirá perfeccionando. Una de sus aplicaciones más importantes será la estructuración de los anexos estadísticos del Informe sobre la salud en el mundo. A partir de este mismo año 2000, el informe facilitará información sobre el desempeño del sistema de salud de cada país. Esta labor analítica ayudará a los países de todo el mundo a articular una respuesta más eficaz a las complejas y cambiantes necesidades de salud de sus poblaciones.

Introduction

There is wide variation in health outcomes for countries with similar levels of income and education (1, 2). Some of this variation is due to differences in health system performance. Differences in the design, content and management of health systems translate into differences in a range of socially valued outcomes, such as health, responsiveness, or fairness. Decision-makers at all levels need to quantify the variation in health system performance, identify factors that influence it and articulate policies that will achieve better results in a variety of settings. The performance of system sub-components, such as regions within countries or public health services, also needs to be assessed. Meaningful, comparable information on health system performance, and on key factors that explain performance variation, can strengthen the scientific foundations of health policy at international and national levels. We believe that a convincing and operational framework for assessing health system performance is vital for the work of governments, development agencies and multilateral institutions.

Several frameworks for measuring health system performance have been proposed (3–10) and are testimony to the importance given to this enterprise. Taken together, these frameworks are a rich source of ideas and approaches. Nevertheless, we believe that there is room for improvement. For example, approaches to health system performance often fall into two related traps. Some are inclusive lists of multiple, and often overlapping, desirable attributes of health systems. Various frameworks, for example, have included goals related to health, health inequalities, coverage, equitable financing, quality, consumer satisfaction, allocative efficiency, technical efficiency, cost containment, political acceptability, and financial sustainability. Other approaches start from a consideration of which indicators are readily available, and construct a performance assessment that replicates the conceptual and technical inadequacies of available measures. Both approaches are unsatisfactory for a comprehensive and meaningful assessment of health system performance.

We believe that a coherent and consistent framework should begin with addressing a simple question: what are health systems for? Once the intrinsic goals of health systems have been clearly articulated, these goals must be measured, and both the concept of performance and the key factors that influence performance must be explored. This paper is both a framework and a blueprint for further refinement and development work that WHO will pursue over the coming years. A major application of this framework will be to structure the statistical annexes of the World Health Report. Beginning in 2000, the report will present information on health system performance for each country.

In developing the conceptual basis for health system performance, this paper covers eight topics: the boundaries of the health system, the difference between intrinsic and instrumental goals, mapping between social systems and social goals, the main goals of a health system, the instrumental goals for health systems, the concepts of performance and efficiency, applying the concept of performance to subsystems or institutions, and key factors influencing health system performance. Some implications and future directions are presented in the discussion.

Boundaries of the health system

Many boundaries have been proposed for separating the health system from elements outside of it (11–16). Some components, such as individual health services delivered at clinics, are included by all boundary definitions. But there is more controversy with other components. For example, are seatbelt regulations and their enforcement part of the health system? Clearly, all boundary definitions are arbitrary, but to undertake an assessment of health system performance an operational boundary must be proposed.

We construct such an operational definition by beginning with the concept of a 'health action'. A health action is defined to be any set of activities whose primary intent is to improve or maintain health. And a health system includes the resources, actors and institutions related to the financing, regulation and provision of health actions. This primary intent criterion leads to a broad definition of the health system. For example, it includes efforts to improve road and vehicle safety where the primary intent is to reduce road traffic accidents, and it also includes personal health services whether they contribute to health or not. One major advantage of the primary intent criterion is that it includes in the assessment of health system performance all actors and institutions who see their primary purpose as contributing to health.

Many actions that profoundly influence health, such as educating young girls, are not part of the health system according to this definition, as the primary intent of education is not to improve health. Excluding these actions from the definition of a health system does not, in any way, question the importance of determinants of health that are outside the health system. In addition, it is critical to recognize that efforts to influence other sectors are clearly part of the health system when they improve determinants of health, such as educating young girls or reducing social inequalities. These efforts at intersectoral action are intended to improve health and therefore fulfil the primary intent criterion.

Even with a clearly articulated definition of a health action, questions remain about a range of actions. For example, what is the primary intent of food supplementation programmes: to improve health by preventing malnutrition, or to redress income inequality? For ambiguous cases, particularly in the area of water and nutrition, some arbitrary decisions must be made so that health systems are consistently defined. Such codification of the boundaries of health systems is essential to this exercise and to related efforts, such as national health accounts. While important for practical measurement, resolution of this limited number of ambiguous cases is not relevant to the rest of this paper.

Intrinsic and instrumental goals

Many laudable goals have been articulated for social systems, including the health system. To have an informed debate about goals, however, we need to distinguish between intrinsic goals — goals valued in themselves — and instrumental goals, whose pursuit is really a means to another end. Intrinsic goals fulfil the following two criteria:

• It is possible to raise the level of attainment of the goal, while holding the level of all other intrinsic goals constant. In theory, a given intrinsic goal is at least partially independent of all others.

• Raising the level of attainment of an intrinsic goal is desirable. If it is not, it is probably an instrumental goal and not an intrinsic goal.

Desirable goals that do not fulfil both of these criteria are likely to be instrumental goals. We will use the distinction between intrinsic and instrumental goals to keep the list of goals for the health system parsimonious and to simplify measurement of goal attainment.

Social systems and social goals

As shown in Fig. 1, the organized activity of society can be divided into various systems such as education, health, economic, political, etc. For each of these systems there is a defining goal, the reason for which the system exists. The goal of the education system, for example, is to educate individuals; and for the health system it is to improve health. In addition to the defining goal, there are two other goals common to all systems. They are the responsiveness of the system to the legitimate expectations of the population; and fairness in the financial contribution required to make the system work. For all systems and defining goals, the population will have expectations about how institutions and actors should interact with them as they attempt to achieve a goal. For example, are human rights respected? To what extent are individuals autonomous and involved in decisions? Are people treated with dignity?

Similarly, there is the goal of fairness in financial contribution for every system. The definition of fairness may vary considerably for different systems. Perhaps the market mechanism — you pay for what you get — is appropriate for most consumption goods. But for health, education, security and some other systems, the concept of fairness in financial contribution may be very different.

The health, education or security systems may and probably do affect (positively or negatively) attainment of the defining goals of other systems. Recognizing these interactions, we can define a series of cross-system goals for each system. The matrix of systems and goals emphasizes the interdependence of all parts of society and the multiple social goals to which all systems may contribute. At the same time it also recognizes the primacy of the defining goal and the common goals of responsiveness and fairness of financing.

Health system goals

Based on the matrix in Fig. 1, we can more explicitly formulate the three main goals for the health system: health, responsiveness and fairness in financial contribution. These intrinsic goals should be routinely monitored by all countries and should form the main basis for assessing health system performance in programmes facilitated by WHO. Therefore, assessments of goal attainment are focused on measuring these three goals, and on relating goal attainment to resource use, to evaluate performance and efficiency.

There are also cross-system goals for the health system that are potentially important and that should be the subject of special analysis and evaluation. For example, how much does the health system help or hinder education, democratic participation, economic production, etc? One of the more important cross-system goals is the contribution of the health system to economic production, since health and health systems may increase or decrease economic production. For example, some methods of organizing health financing, such as certain employment-based insurance, may hinder labour mobility and macroeconomic performance (17, 18). At the same time, increasingly there is evidence that improvements in health can enhance economic growth (19, 20). These are important areas for further research, but the nature of the cross-system relationships and the complexity of measurement precludes them from routine assessment of health system performance.

The following discussion aims to articulate more precisely the content of each of the three goals.

Health

The defining goal for the health system is to improve the health of the population. If health systems did not contribute to improved health we would choose not to have them. The health of the population should reflect the health of individuals throughout life and include both premature mortality and non-fatal health outcomes as key components. We are concerned both with the average level of population health and with health distribution inequalities within the population (21).

Responsiveness

The second intrinsic goal is to enhance the responsiveness of the health system to the legitimate expectations of the population. Responsiveness expressly excludes the health improvement expectations of the public, as these are fully reflected in the first goal above. The term "legitimate" is used to make it clear that although some may have frivolous expectations for the health system these should play no part in articulating responsiveness.

We propose that responsiveness has two major components. The first can be called "respect for persons", and it captures aspects of the interaction of individuals with the health system that often have an important ethical dimension. Respect for persons has three aspects.

• Respect for dignity. Health systems might be able to achieve higher levels of health by incarcerating individuals with a communicable disease or sterilizing individuals with a genetic disorder, but this would be a violation of basic human rights (22– 24). Respect for dignity also includes interactions with providers, such as courtesy and sensitivity to potentially embarrassing moments of clinical interrogation or physical exploration (25–29).

• Respect for individual autonomy. The individual should be able to act autonomously when making choices about his/her own health. Individuals, when competent, or their agents, should have the right to choose what interventions they do and do not receive (25–27, 30–32).

• Respect for confidentiality. When interacting with the health system, individuals should have the right to preserve the confidentiality of their personal health information (33, 34). Respect for confidentiality serves an instrumental goal of improving the quality of health care; when individuals have confidence that the confidentiality of their personal health information will be respected, they are more likely to give important medical history information to health care providers. In addition, respect for confidentiality is intrinsically valuable because it upholds a core notion of privacy and individual control over personal information (27, 35–39).

The second component can be called "client orientation", and it includes several dimensions of consumer satisfaction that are not a function of health improvement. Client orientation has four aspects.

• Prompt attention to health needs. Surveys of population satisfaction with health services routinely demonstrate that prompt attention is a key dimension (40–44). Individuals value prompt attention because it may lead to better health outcomes; this instrumental value is captured in the defining goal of health. Individuals may also value prompt attention because it can allay fears and concerns that come with waiting for diagnosis or treatment. Both the intrinsic and instrumental value of prompt attention are critically affected by factors such as physical, social and financial access.

• Basic amenities. The basic amenities of health services, such as clean waiting rooms or adequate beds and food in hospitals, are aspects of care that are often highly valued by the population (45).

• Access to social support networks for individuals receiving care. Even when care is promptly available, if it is provided far from the individual's family and community, access to social support networks during care and recovery may be hampered. An expectation of access to social support is not only an instrumental goal, because it may enhance health outcomes, but it is also an intrinsically valued attribute (46).

• Choice of institution and individual providing care. Patients may want to select who provides them with health care (43). This concern is most often for the individual provider and only secondarily for the institution providing care. Choice is a legitimate component of responsiveness and takes on an increasing importance as other items in this list have been satisfied.

As with health, we are concerned not only with the average level of responsiveness, but also with inequalities in its distribution. A concern for the distribution of responsiveness across individuals means that we are implicitly interested in differences related to social, economic, demographic and other factors.

Fairness in financial contribution

To be fair, financing of the health system should address two key challenges. First, households should not become impoverished, or pay an excessive share of their income in obtaining needed health care. In other words, fairness in financial contribution requires an important degree of financial risk pooling. Second, poor households should pay less towards the health system than rich households. Not only do poor households have lower incomes but a larger share of their income goes to basic needs such as food or shelter. Contribution to the health system should reflect this difference in disposable income between rich and poor.

These considerations translate into the normative proposition that every household should pay a fair share towards the costs of the health system. (In the case of very poor households, "fair share" might mean no payment at all.) Payment should be based on income and for the most part should not reflect use of services or risk. Acceptable notions of a fair share for the poor depend on the role assigned to the health system in general income redistribution. In some political settings, it may be easier to redistribute income by providing free health services to the poor than through direct redistributive mechanisms. From the perspective of the health system, however, it should perhaps be assumed that society is redistributing general income through other mechanisms, such as direct transfers, when evaluating fairness in financial contribution. The broad social acceptance of a financing mechanism requiring that everyone contribute some fair share may also lead to more sustainable health financing.

Fair financing begins from an ex post perspective since it refers to past household payments for health care through all mechanisms (out-of-pocket expenditures, private voluntary insurance, social insurance, general taxation, excise taxes, etc.) compared to their income. This ex post view is very closely related to the ex ante view of the risk a household faces for health care expenses in the coming year (47). Indeed, the ex post distribution of household health expenditure is the realization of the ex ante distribution of financial risks.¹ The difference between the two for a population is the effect of chance. That difference could be considerable for a small sample of households but not for a large one. In other terms, a particular ex post distribution of household health expenditure could be caused by a relatively small range of distributions of ex ante health expenditure risks. Because of the close relationship between the two, the goal of fair financing encompasses concerns about financial risk protection in the population.

In summarizing our view of health system goals, Fig. 2 shows that we are interested in assessing both the average level and the distribution of the first two goals, namely health and responsiveness. By contrast, however, for the third goal, concerning financial contribution, we are interested only in the distribution and not the average level. The level of health financing is a key policy choice in any society, but it is not an intrinsic goal. Whereas it is always desirable to achieve more health and more responsiveness, it is not intrinsically valuable to spend an ever-increasing amount of money on the health system. Rather, what matters is that the financial burden should be fairly distributed across groups. The level of resources invested in the health system is the variable against which performance is measured, and in the following sections we stress the importance of comparing health system resources when assessing goal attainment.

The proposed framework is directly related to the familiar concepts of quality, equity and efficiency. Thus the level of goal attainment for health and responsiveness can be considered as the overall quality of the health system. This is a broader concept than definitions of quality focused on personal health services alone (48, 49). As explained more fully below, quality is a subset of overall goal attainment, not a performance measure.

Similarly, the distribution of the three goals can be considered as the total equity of a health system. As with quality, equity of the health system is a subset of overall goal attainment and not a performance measure. Our concept of the equity of the health system is broader than simple health inequalities, reflecting the broad assessments in recent work (50).

A third, related concept is efficiency, or composite goal performance. Efficiency is how well the socially desired mix of the five components of the three goals is achieved, compared to the available resources. Composite goal performance and individual goal performance are discussed in more detail below.

Societies will inevitably differ on the weighting they give to these components. Nevertheless, we believe that for the purposes of global comparison, it will be useful to develop a consensus weighting function. This would allow composite measures for both goal attainment and efficiency to be calculated for health systems. Such a consensus weighting function for global comparisons should be built on some common values framework (51), and take into consideration empirical measurements of individual preferences for the different goals in various societies. Ultimately, the use of a composite measure of goal attainment will be limited but, like the Human Development Index (52), it may spark increased attention to the performance of health systems and the factors explaining such performance.

Instrumental goals

In addition to the three main goals for health systems there are many goals that have been given prominence in discussions of health system performance, such as access to care, community involvement, innovation or sustainability. While we do not doubt their importance, these are instrumental goals whose attainment will raise the level of health, responsiveness and fairness in financing. For example, consider access to care. If we fix the level and distribution of health, responsiveness and fairness in financing, but change the level of access, we argue that this would not be intrinsically valued. Improved access to care is desirable insofar as it improves health, reduces health inequalities and enhances responsiveness. But it is an instrumental, rather than intrinsic, goal for health systems. If attainment of the three intrinsic goals is measured adequately, this would fully reflect the impact of access to care (or other instrumental goals) on health outcomes valued by society. Likewise, coverage of many effective public health programmes, such as DOTS for tuberculosis, immunization, or impregnated bed-nets for malaria, are instrumental goals that would be captured in the measurement of health and responsiveness.

Goal performance, composite goal performance and efficiency

With a clearly defined set of goals and their measures, we can compare the level of goal attainment for different health systems. There is a long history of comparing measures of health across countries. The concept of performance, however, is more complex than simply recording the level of goal attainment. Performance of the health system involves relating goal attainment to what could be achieved (21, 53). In other words, performance is a relative concept. A rich country has higher levels of health than a poor one, but which country has a higher level of performance relative to health system resources? We argue that performance should be assessed relative to the worst and best that can be achieved for a given set of circumstances.

Fig. 3 illustrates the concept of performance with respect to the goal of improving population health. The y-axis shows health, and the x-axis shows resources spent on the health system. The top line shows for populations A and B the maximum attainable level of health for each level of health expenditure, given the non-health system determinants such as the level of education. It is clear from the figure that performance is related to the level of health expenditure; for example, population A has a lower level of health than B, but the two populations have approximately the same levels of performance and expenditure.

Measuring goal performance is related to the question: Does one hold the health system accountable for the level of key health determinants that are not entirely the responsibility of the health system, but can be influenced by it? We argue that the answer must be "yes". The health system should be held accountable for these broader determinants of health, to the extent that the best health system can influence them compared to the worst health system. For example, it has been argued that we should not judge the performance of a health system to be poor simply because the population level of tobacco consumption is high, which leads to low levels of health. The implication of this argument is that health systems should not be held accountable for determinants such as tobacco consumption, diet or physical activity. Yet the counter-argument is strong: a good health system should pay attention to the level of tobacco consumption, or to the composition of the population's diet. In defining performance, we believe careful attention should be paid to determinants that the health system can be held accountable for. Thus, health systems should be held substantially accountable for the levels of tobacco consumption; but probably not for levels of educational attainment.

The extent to which a health system should be held accountable for a health determinant can be determined from the degree to which the best health system can change the level of that determinant. However, the degree to which a health system can change a determinant is related to the time frame of the analysis. Many health sector reforms and institutional changes may take several years to have their full effect and thus require a longer-term perspective for assessment. For example, what a health system can achieve in one year to reduce tobacco consumption is different from what a health system can do in five or ten years. Thus, measurements of performance should not be overly sensitive to the starting point for a given year. We argue that performance should be assessed with a longer time horizon, and when the maximum goal for a given level of health expenditure is calculated this should include what can be achieved over many years. Otherwise, health systems are never held accountable for past mistakes, or given credit for past successes.

Performance can be assessed for each of the five components of the three goals. In the face of scarce resources, societies will choose the relative importance of these goals. In other words, by choice, a society may perform poorly on responsiveness, but well on health inequality, because more resources from the total budget are assigned to redressing inequalities than to enhancing responsiveness. Ultimately, a single budget for the health system is used to achieve all of the goals. Because there will be trade-offs between some of the goals, performance in achieving the composite of the three goals is also a useful construct. In economics, the concept of efficiency means producing a desired output at least cost or producing the maximum quantity for a fixed budget² (54). In this sense, composite goal performance — how well a health system achieves the desired outcomes given available resources — is also the efficiency of the health system. We will use interchangeably the term 'composite goal performance' and 'efficiency' in the rest of this work.

As noted in the section on health system goals, societies may value the five components of the three goals differently. Each society must make its own policy choices and this can affect the extent to which each of the three goals are attained. Nevertheless, we believe there will not be great variance in the importance attached to different goals for most societies in the world. Shared conceptions about the importance of these goals (51) will allow for meaningful performance comparisons between countries for each of the five components of the three goals and for composite goal performance. These comparisons will provide an empirical basis for debates on the effect of different patterns of health system organization. By facilitating these comparisons of performance and efficiency, the framework proposed here can enrich policy debates on the design, organization and operation of health systems. Measuring performance and efficiency using a common framework does not mean that one particular health system design will emerge from the analysis as a blueprint for all countries.

Performance of subsystems or organizations

Performance of the entire health system must be related to the performance of various subcomponents, or even organizations such as hospitals, within the health system. Work on the performance of providers of personal health services is converging with work assessing the overall performance of health systems. For example, the US Agency for Health Care Policy and Research is exploring the development of a national quality of care measure, constructed from quality measures of component provider organizations (4). The conceptual framework we have presented for performance of the entire health system is largely applicable to subcomponents, such as the non-personal health services, or to specific organizations such as a hospital or outpatient care provider. The key would be to compare the level of goal attainment for the entire population to the level of goal attainment that would be achieved with the best and worst performance of that subsystem or organization. Of course, the challenge will be how to define the best and worst attainable lines for a given sub-system or organization.

This mapping from the framework for the overall system to its subcomponents will work well for the level of health and responsiveness. In contrast, the potential contribution of any one organization to the distributional goals will be much smaller and much more difficult to estimate. As this framework for assessing health system performance is developed and implemented, further work on its application to organizations and subsystems will be needed.

It is also important to note that the applicability of the approach to specific organizations or subsystems does not imply that the overall performance of the system is a simple (additive or even multiplicative) function of the performance of different subsystems or organizations. Interactions between, for example, the performance of intersectoral activities in reducing exposure to tobacco through excise taxes, and the performance of hospitals and clinics, will be highly complicated. Ultimately, it may not be possible or even useful to try to define the relationship between overall performance and the performance of various subsystems or organizations.

Factors explaining health system performance

Any systematic attempt to understand the performance of health systems should include a study of factors that potentially explain it. Reforms improving performance require information on explanatory factors. Of the extensive array of reform candidates, which ones should be measured and analysed? This task could be approached through the development of extensive lists of technical and institutional factors. In developing such lists tension is often revealed between those groups focused on the technical content of health services (e.g., immunizations or intensive care units) and those focused on institutional arrangements of the health system (e.g., provider payment mechanisms or social insurance). It is necessary to provide some framework for thinking about the dimensions of health systems that may influence performance and to put in place ways of measuring these dimensions. This topic is treated more extensively elsewhere (55–57).

Developing a categorization and a series of measures of key factors that may explain variation in health system performance is a very different task than articulating the goals of health systems. Each proposed factor represents a hypothesis that should be tested empirically. For this reason, any list is provisional and subject to expansion or contraction as evidence accumulates.

As shown in Fig. 4, in every health system organizations have to perform four basic functions: financing, provision, stewardship and resource generation (human, physical and knowledge) (56). Every health system grapples with the key problems of designing, implementing, evaluating and reforming the organizations and institutions that facilitate these functions. It is possible to compare solutions to these problems along three major categories, which form a continuum from broad policy directions to specific operational attributes: strategic design, structural arrangements and implementation management. In the following sections, we specify the elements for each function. This is a useful beginning, but further research is required to understand how organizational and institutional solutions affect performance.

Financing

Health system financing is the process by which revenues are collected from primary and secondary sources, accumulated in fund pools and allocated to provider activities. For the purposes of analysis, it is useful to divide health system financing into three subfunctions: revenue collection, fund pooling and purchasing.

Revenue collection refers to the mobilization of money from primary sources (households and firms) and secondary sources (governments and donor agencies). Funds can be mobilized through eight basic mechanisms: out-of-pocket payments, voluntary insurance rated by income, voluntary insurance rated by risk, compulsory insurance, general taxes, earmarked taxes, donations from nongovernmental organizations and transfers from donor agencies.

The strategic design of revenue collection is likely to affect health system performance in a profound way. Some of the key dilemmas that decision-makers face include compulsory versus voluntary payments, prepayment versus payment at the point of service, and progressivity of insurance premiums or fees. A key aspect of the structural arrangements of revenue collection, with potential effects on performance, is the governance of institutions, mostly as it refers to the extent of public versus private participation. Another important decision matter has to do with the number of organizations that carry out this function, which in turn raises questions of economies of scale and concentration. Implementation management issues that might affect performance include measures to avoid evasion, specific collection procedures and earmarking of taxes.

Fund pooling refers to the accumulation of revenues for the common advantage of participants. Indeed, pooling means that financial resources in the pool are no longer tied to a particular contributor and contributors share financial risk. Pooling is distinct from revenue collection, as some mechanisms of revenue collection, such as medical savings accounts, do not share financial risks across contributors (58).

Aspects of the strategic design of fund pooling that are likely to affect performance include the extent to which there are separate fund pools for different population groups, separate fund pools for personal and non-personal health services and cross-subsidization between low-risk and high-risk contributors. Structural arrangements of fund pooling include the size and number of fund pools, mechanisms to transfer funds among pools, choice and competition among fund pools for enrolment, and governance of institutions maintaining fund pools. A critical implementation management question that might affect performance refers to rules that guide the entry and exit of organizations performing this function, including procedures for protecting contributors in the case of insolvency or bankruptcy. Another regulatory matter is the rules governing the financial management of the funds, including the degree of investment risk that is allowed.

Purchasing is the process through which revenues that have been collected in fund pools are allocated to institutional or individual providers to deliver a set of interventions. Purchasing can range from simple budgeting exercises in highly integrated public systems, where the government collects revenue through general taxation and allocates it to programmes and facilities for staff and other costs, to more complicated strategies where specified units of inputs, outputs or outcomes are purchased.

Strategic design includes decisions on what is purchased, how it is purchased and from whom it is purchased. Choosing what is purchased has to do with the criteria used to select interventions for inclusion or exclusion. Alternative mechanisms include direct purchasing of interventions (coronary artery bypass grafts, childhood immunizations etc.), more general purchasing of service types (physician services, hospital bed-days, etc.), or purchasing of inputs (doctors, hospital beds or vehicles). Another aspect of strategic design is the choice of providers to deliver interventions, including issues such as command and control procedures or contractual processes that relate the purchaser to providers, the criteria for choosing providers, and provider payment mechanisms.

Structural arrangements that may affect performance include the size and number of purchasers, the mechanism of funding purchasers from revenue pools, choice and competition between purchasers for enrolment, and governance of purchasers. The most important matters of implementation and management are methods for controlling the quantity and the quality of purchased services. It is beyond the scope of this paper to examine the vast array of utilization review and quality assurance techniques that may be used to orient purchasing decisions. Many of them form the core of what has been termed "managed care".

Provision of health services

This function refers to the combination of inputs into a production process that takes place in a particular organizational setting and that leads to the delivery of a series of interventions. In analysing provision, it is useful to keep in mind the conventional distinction between personal and non-personal health services. The former refer to services that are consumed directly by an individual, whether they are preventive, diagnostic, therapeutic or rehabilitative, and whether they generate externalities or not. The latter refer to actions that are applied either to collectivities (e.g., mass health education) or to the non-human components of the environment (e.g., basic sanitation).

Personal health services. The strategic design of providing personal health services deals mostly with the relationships of each provider organization to its environment. One set of such relationships has to do with vertical integration among functions. While this matter is analysed in more general terms below, it is necessary to underscore its importance for the provision function, since in this case the most fundamental design question refers to the extent of integration with the purchasing function. Indeed, national health systems are often distinguished according to whether these two functions are carried out by the same organization, or whether providers are independent contractors for purchasing entities. One of the main current reform proposals for integrated systems is precisely the "purchaser–provider split". Closely linked to this matter is the issue of decentralization and governance of provider institutions, particularly with respect to the extent of autonomy, even in a framework of public ownership. The extent to which such fundamental design options affect the performance of health systems is an important research question with major policy implications.

Structural arrangements have to do mostly with the relationships of provider organizations with each other. A central question is the extent to which such organizations are separate entities, or whether they form networks at different levels of complexity (i.e., primary, secondary and tertiary care facilities). The structure of such networks will determine whether access to a facility is direct, or whether there is a gatekeeping role for first-contact providers. This is one instance of the more general problem of organizing referrals among levels of care, including whether referrals can cross public–private boundaries, or whether private and public networks are segregated from each other.

In contrast to the first two aspects of provision, issues related to implementation management have to do mostly with the internal dimension of each provider organization, including the formal and informal ways in which each organization articulates its tasks, control systems and relationships of authority (59). Staffing patterns for both management and clinical services are important elements (60). Typical issues include whether top management of provider organizations is in the hands of physicians or of professional administrators. At the clinical level, the question of skill mix among the various categories of providers (physicians, nurses, community health workers, etc.) is also likely to be a determinant of performance.

Non-personal health services. Conceptually, the same issues apply to non-personal health services as to personal services. Nevertheless, in most countries the public sector, often the Ministry of Health, takes a dominant role in providing non-personal health services.

A key question for strategic design is the extent to which single organizations provide a wide array of non-personal health services, or whether multiple specialist organizations provide specific services such as health promotion, occupational safety or road traffic safety. As with personal health services, the extent of integration with purchasing is important, as is the nature of governance and autonomy.

A main issue for structural arrangements appears to be the degree of integration of non-personal and personal health service provision. In public systems, are the same organizations and managers responsible for both categories of services? In systems with private providers, are there incentives and other mechanisms for them to deliver some non-personal services as well, such as mass health education?

Finally, the same issues of intra-organizational management that were highlighted for providing personal health services would apply here as well.

Resource generation

Health systems are not limited to institutions that finance or provide services, but include a diverse group of organizations that produce inputs to those services, particularly human resources, physical resources such as facilities and equipment, and knowledge (15). This set of organizations encompasses universities and other educational institutions, research centres, and companies producing specific technologies such as pharmaceutical products, devices and equipment.

Questions of strategic design, structural arrangements and implementation management will vary according to the specific subset of organizations involved in resource generation. For example, in the case of training institutions, such as medical and nursing schools, a key issue refers to their primary organizational location, i.e. whether they belong to the ministry of health or the ministry of education. Organizational ownership may determine the match between supply and demand for health personnel. In the case of research organizations, questions of autonomy in determining priorities become salient. In the case of the pharmaceutical and other technological industries, common questions of industrial structure, such as the degree of concentration and the extent of competition, are very likely to influence health system performance.

Stewardship

A neglected function in most health systems, stewardship goes beyond the conventional notion of regulation. It involves three key aspects: setting, implementing and monitoring the rules for the health system; assuring a level playing field for all actors in the system (particularly purchasers, providers and patients); and defining strategic directions for the health system as a whole. To deal with these aspects, stewardship can be subdivided into six subfunctions: overall system design, performance assessment, priority setting, intersectoral advocacy, regulation, and consumer protection (56).

Overall system design. This has to do with policy formulation at the broadest level. It involves the way in which all the other health system functions are put together. The various issues that have been analysed above under the rubric of strategic design form the substantive content for this subfunction of stewardship. In this respect, stewardship can be thought of as a meta-function, insofar as it deals with the organization of all the other functions of a health system.

Performance assessment. An essential ingredient for providing strategic direction and assuring a level playing field is to assess the performance of institutions involved in revenue collection, purchasing, provision and resource development. This is another meta-function.

Priority setting. Choosing criteria for setting priorities and building a consensus around them are major elements of stewardship. This has both a technical and a political aspect.

Intersectoral advocacy. This is the promotion of policies in other social systems that will advance health goals. As mentioned earlier, social and economic determinants of health status, such as female education, are not themselves part of the health system. However, advocating progress on those determinants, with the purpose of improving health, clearly fits our definition of a health action and therefore falls within the boundaries of the health system.

Regulation. Strictly speaking, regulation means setting rules. In the health system there are two main types of regulation: sanitary regulation of goods and services, and health care regulation. The former refers to conventional efforts by sanitary authorities to minimize the health hazards generated by the goods and services throughout the economy, especially those associated with foodstuffs. Health care regulation refers to organizations charged with the financial, provision and resource development functions of the health system. In this respect, regulation is again a meta-function directed at institutions charged with other functions, through instruments such as accreditation, certification and rate setting.

Consumer protection. Both the insurance and the health care markets are characterized by information and power asymmetries between consumers and producers. Therefore, part of the stewardship function is achieving a level playing field for the actors of the heath system.

The mix of these six subfunctions of stewardship is the key decision regarding its strategic design. Indeed, there seems to be remarkable variation in the extent to which health authorities assume responsibility for each subfunction. With respect to structural arrangements, the main policy consideration refers to the locus of responsibility. While stewardship generally entails a set of core public functions, there is variation in the allocation of responsibility to different branches (executive versus legislative) and levels (national versus local) of government.

Finally, an important question of implementation management that affects performance refers to the actual skills to carry out stewardship functions. In particular, the reorientation of most ministries of health, from their traditional function as providers of services, to the new challenges of stewardship involves major organizational re-engineering for which the skill mix may not be adequate.

Vertical integration

As indicated earlier, health system performance is related not just to the organization of each separate function, but also to the way in which each function relates to the others. For example, vertical integration, whereby one entity is responsible for more than one function, is the norm, not the exception. The extent of vertical integration may matter for some combinations, such as revenue collection and purchasing or purchasing and provision, but not for others. Fig. 5 illustrates four different polar examples of vertical and horizontal integration or segmentation of financing, provision and stewardship functions. In this figure, each vertical section represents an organization which undertakes one or more function. The horizontal sections represent coverage of the population.

In the vertical and horizontal integration model the functions of financing, provision and much of stewardship are integrated into a single public sector organization covering the entire population. The UK National Health Service before the Thatcher reforms was somewhat close to this degree of complete integration. Fig. 5 also illustrates a system with vertical integration but horizontal segmentation: different organizations, each integrating the financing and provision functions, cater to different population groups. Examples that approximate this model include the segmented systems of much of Latin America. Then Fig. 5 illustrates a system with vertical segmentation and horizontal integration. Examples of this type of organization include Australia, Colombia and several autonomous communities of Spain. In fact, many of the current reform proposals aim at this type of functional organization for the health system. Health systems where there is vertical and horizontal segmentation are also illustrated.

A key policy question is the extent to which functional specialization or integration affects health system performance.

Factors external to the health system

So far, the discussion of factors that influence health system performance has focused on the organization of five key functions of health systems and the extent to which they are vertically integrated. There are, however, factors outside the health system that will influence the performance of these functions. For example, the presence of an effective judicial system can have a substantial influence on the ability of purchasers to enforce contracts on providers. General attributes of government, such as ethical codes of conduct and the tolerance of corruption can also influence the performance of stewardship and other functions. These interrelationships show that some ultimate determinants of health system performance lie outside the formal architecture of the health system.

Discussion

A main concern for those who work on health system performance is the scope of accountability for health systems. Simply put, should health systems be narrowly accountable for actions within their organizations, or more broadly for key determinants of goal attainment outside the boundaries of the health system? Two viewpoints can be put forward: it is unfair to hold health systems accountable for things that are not completely under their control; and health systems can achieve greatest impact through influencing non-health system determinants of health. Through our framework for assessing health system performance, we confront this debate in three ways.

First, by describing the levels to which health system goals have been attained, regardless of the reasons that explain them, national and world attention can be brought to bear on those countries that have done well in improving health, reducing health inequalities, enhancing the level and distribution of responsiveness, and in financing the system in a fair manner.

Second, the assessment of relative performance should include how well a country is controlling the level of non-health system determinants through effective intersectoral action. Ultimately, the reason why these non-health system determinants must be included to a greater or lesser degree is that if the health system is not held accountable for them, no one will be the advocate in a country for addressing these issues. The best hope for change in many cases lies with the health system's ability to explain the importance of tackling these problems. Problems such as tobacco consumption, diet, and unsafe sexual activity must be included in an assessment of health system performance. The potential for the health system to influence other determinants, such as educational attainment, general social inequalities and biodiversity, is much less and the assessment of health system performance should reflect this fact.

Third, the scope of accountability is much narrower when assessing the performance of subsystems and institutions. Broad accountability for non-health system determinants of health belongs with the overall health system. Performance assessment for components of the system naturally narrows the scope of accountability to those actions directly influenced by the institution in question. By approaching the problem at three levels, we feel that we can focus attention on the overall attainment of health system goals, the performance of national health systems and the performance of subsystems and institutions.

By clearly distinguishing the intrinsic goals of health systems from the functional organization of the system and from the technical content of provision, we hope to facilitate a more constructive dialogue on health policy. There has long been a debate over whether health system architecture matters more than the intervention mix delivered by the systems. Interventions must be delivered by health systems and health systems without effective interventions are useless. These two views should not be seen as competitive but rather as complementary. By introducing clear measurements of health system goal attainment and performance, of the organization of key health system functions, and of the technical content of health service provision, we hope to facilitate a more reasoned and informed debate on the interaction of system architecture and intervention mix.

Much of the debate on health system design is couched in claims and counter-claims about what works and what does not work. The only support for some claims may be theoretical models or anecdotal evidence. Economists are fond of defending health policies with basic utility models. For example, if consumers are perfectly informed and providers are competitive then user fees will enhance efficiency (61). Given the extraordinary diversity of health system organization, for every anecdote there is often a counter-anecdote, and few decision-makers or health system experts have access to information on different systems and their levels of achievement.

The implementation of this framework for assessing health system performance will lay the basis for a shift from ideological discourse on health policy to a more empirical one. Over time, we should be able to provide empirical answers to such questions as the relationship between the organization of health financing and the level and distribution of health and responsiveness. This line of work should make it possible to ascertain, for example, the extent to which competition among purchasers or providers enhances responsiveness. If the framework encompasses the main intrinsic goals for health systems and the key candidate factors for explaining variation in performance, it will lay the basis for a more scientific discourse on health policy.

Annual assessments of health system performance will focus attention on the policy options available to governments for improvement. Global institutionalization of performance assessment may contribute to the ongoing reflection on the role of the state in health systems. What policies can enhance performance? What evidence is there that the state can enhance performance through the adoption of these policies? An enlightened role for the state then becomes to enhance performance, where the evidence suggests it has the potential to do so.

This framework for assessing health system performance is work in progress that will undoubtedly evolve as its operationalization proceeds and evidence accumulates on the link between health system organization and performance. The development of such a long-term agenda will help countries all over the world to articulate a better response to the complex and changing health needs of their populations.

Acknowledgements

The framework presented in this paper has been developed and improved as part of the discussions for The World Health Report 2000 – Health systems: improving performance. The core technical writing team of Philip Musgrove, Andrew Creese, Alex Preker and Christian Baeza has made fundamental contributions to the framework. We have also benefited from the valuable comments provided by the members of the Steering Committee, the Advisory Group and the Regional Reference Group for the World Health Report. We are grateful for comments on a previous draft of this paper by Kei Kawabata, David Evans, Dan Wikler, Pierre Lewalle, Emmanuela Gakidou, Michael Reich and Phyllis Freeman.

References

1. Preston SH. Mortality and development revisited. Population Bulletin of the United Nations, 1986, 18: 34–40.        

2. World development report 1993: investing in health. New York, Oxford University Press for The World Bank, 1993.        

3. Jee M, Or Z. Health outcome measurement in OECD countries: toward outcome-oriented policy making. Paris, Organization for Economic Co-operation and Development, 1999 (DEELSA/ELSA/WP1(98)6/ANN).        

4. Meyer G. Quality measurement initiatives in the US Federal Government. Presentation at the Commonwealth Fund, March 1999.

5. Aday LA et al. Evaluating the medical care system: effectiveness, efficiency, and equity. Ann Arbor, University of Michigan, Health Administration Press, 1993.        

6. Knowles JC, Leighton C, Stinson W. Measuring results of health sector reform for system performance: a handbook of indicators. Bethesda, MD, 1997 (Partnerships for Health Reform, Special Initiatives Report No.1).        

7. Hurst J. An assessment of health system performance across OECD countries. Paris, OECD, Health Policy Unit (DEELSA/ELSA/ WPI(99)3, in press).        

8. Guidelines for the preparation of the health services system profile in the countries of the region. Washington, DC, Pan American Health Organization, 1999.        

9. Hoffmeyer UD, McCarthy TR eds. Financing health care., The Netherlands, Kluwer Academic Publishers, 1994.        

10. Hsiao CW. A framework for assessing health financing strategies and the role of health insurance. In: Dunlop D, Martins J, eds. An International assessment of health care financing. Lessons for developing countries. Washington, DC, The World Bank, 1995: 15–29.        

11. Chernichovsky D. Health system reform in industrialized democracies: an emerging paradigm. Milbank Quarterly, 1995, 73: 339–372.        

12. Frenk J. Dimensions of health system reform. Health Policy, 1994, 27: 19–34.        

13. Hsiao WC. Comparing health care systems: what nations can learn from one another. Journal of Health Politics, Policy and Law, 1992, 17: 613–636.        

14. North DC. Institutions and institutional development. Cambridge, Cambridge University Press, 1990.        

15. Roemer MI. National health systems of the world. Vol. 1: The countries. New York, Oxford University Press, 1991.        

16. Terris M. The three world systems of medical care: trends and prospects. American Journal of Public Health, 1978, 68: 1125–1131.        

17. Buchmueller T, Valletta R. The effect of health insurance on married female labour supply. Federal Reserve Bank of San Francisco, 1996 (paper 96–09).        

18. Gruber J, Madrian B. Health insurance and the labour market. Cambridge MA, National Bureau of Economic Research, 1998 ( NBER WP N.6762).        

19. Sacks J, Gallup J. Geography and economic growth. Cambridge MA,Consulting Assistance on Economic Reform, Harvard Institute for International Development, 1999 (CAER II Discussion Paper No. 39).        

20. Bhargava A, Jamison DT, Lau LJ. Modelling the effects of health on economic growth. Geneva, World Health Organization (GPE Discussion Paper series, forthcoming).        

21. Donabedian A. Models for organizing the delivery of personal health services, and criteria for evaluating them. Milbank Memorial Fund Quarterly, 1972, 50 (4): 103–154.        

22. Pernick, M. Eugenics and public health in American history. American Journal of Public Health, 1997, 87 (11): 1767– 1772.        

23. Muller CA. Window on the past: the position of the client in twentieth century public health thought and practice. American Journal of Public Health, 1985, 75 (5): 470–476.        

24. Leavitt J. 'Typhoid Mary' strikes back: bacteriological theory and practice in early twentieth-century public health. Isis, 1995, 86 (4): 617–618.        

25. World Medical Association. Declaration of Lisbon, 1981 and 1995.        

26. The European Forum of Medical Associations and WHO. Principles of the rights of patients in Europe: a common framework. Copenhagen, World Health Organization Regional Office for Europe, 1996.        

27. Convention for the Protection of Human Rights and Dignity of the Human Being with Regard to the Application of Biology and Medicine: Convention on Human Rights and Biomedicine. Strasbourg, Council of Europe, 1996.        

28. Wainwright P. The observation of intimate aspects of care: privacy and dignity. In: Hunt G ed. Ethical issues in nursing. New York, Routledge, 1994: 38–54.        

29. Smith AM. Dignity and medical procedures. Journal of Medical Ethics, 1981, 7 (2): 88–90.        

30. Brock D. Life and death: philosophical essays in biomedical ethics. Cambridge, Cambridge University Press, 1993.        

31. US President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research.Making health care decisions: a report on the ethical and legal implications of informed consent in the patient-practitioner relationship. Washington, DC, Government Printing Office, 1982.        

32. Faden RR, Beauchamp TL, King NM. A history and theory of informed consent. Oxford, Oxford University Press, 1986.        

33. Rylance G. Privacy, dignity and confidentiality: interview study with structured questionnaire. British Medical Journal, 1999, 318: 301.        

34. Beauchamp T, Childress J. Principles of biomedical ethics. Third edition. Oxford, Oxford University Press, 1989.        

35. Bok S. Secrets. Oxford, Oxford University Press, 1986.        

36. Lowes M, Havens DMH. Privacy and confidentiality of health information. Journal of Pediatric Health Care, 1998, 12: 42–46.        

37. Reiman J. Privacy, intimacy, and personhood. Philosophy & Public Affairs, 1976, 6 (1): 26–44.        

38. Rachels J. Why privacy is important. Philosophy & Public Affairs, 1975, 4 (4): 323–333.        

39. Schoeman FD. Privacy and social freedom. Cambridge, Cambridge University Press, 1992.        

40. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Social Science and Medicine, 1997, 45 (12): 1829–1843.        

41. Ross CK, Steward CA, Sinacore JM. A comparative study of seven measures of patient satisfaction. Medical Care, 1995, 33 (4): 392–406.        

42. Thompson AGH, Sunol R. Expectations as determinants of patients' satisfaction: concepts, theory and evidence. International Journal for Quality in Health Care, 1995, 7 (2): 127–141.        

43. Carr-Hill RA. The measurement of patient satisfaction. Journal of Public Health Medicine, 1992, 13 (3): 236–249.        

44. Cohen G, Forbes J, Garraway M. Can different patient satisfaction survey methods yield consistent results? Comparison of three surveys. British Medical Journal, 1996, 313: 841–844.        

45. Bernhart MH et al. Patient satisfaction in developing countries. Social Science and Medicine, 1999, 48 (8): 989–996.        

46. Gilson L, Alilio M, Heggenhougen K. Community satisfaction with primary health care services: an evaluation undertaken in the Morogoro region of Tanzania. Social Science and Medicine, 1994, 39 (6): 767–780.        

47. Murray CJL et al. Defining and measuring fairness of financial contribution. Geneva, World Health Organization, 2000 (GPE Discussion Paper No. 24).        

48. Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly, 1996, 44: 166–203.        

49. Donabedian A. Explorations in quality assessment and monitoring .In: Griffith JR et al, eds. The definition of quality and approaches to its assessment. Ann Arbor, University of Michigan, Health Administration Press, 1980.        

50. Daniels N, Light D, Caplan R. Benchmarks of fairness for health care reform. New York, Oxford University Press, 1996.        

51. Bok S.Common values. Colombia, MO, University of Missouri Press, 1995.        

52. Human Development Report 1999. New York, Oxford University Press for the United Nations, 1999.        

53. Donabedian A, Wheeler JRC, Wyszenianski L. Quality, cost and health: an integrative model. Medical Care, 1982, 20 (10): 975–992.        

54. Boadway R, Bruce N. Welfare economics. Oxford, Basil Blackwell, 1984.        

55. Frenk J. Dimensions of health system reform. Health Policy, 1993, 27: 19–34.        

56. Londoño JL, Frenk J. Structured pluralism: towards an innovative model for health system reform in Latin America. Health Policy, 1997, 41: 1–36.        

57. Kutzin J. Towards universal health care coverage: a goal-oriented framework for policy analysis. Geneva, World Health Organization (GPE Discussion Paper, forthcoming).        

58. Ozanne L. How will medical savings accounts affect medical spending? Inquiry, 1996, 33 (3): 225–236.        

59. Scott WR. Some implications of organization theory for research on health services. Milbank Memorial Fund Quarterly, 1966, 44: 35–64.        

60. Frenk J, Ruelas E, Donabedian A. Staffing and training aspects of hospital management: some issues for research. Medical Care Review, 1989, 46 (2): 189–220.        

61. World Bank. Financing health services in developing countries: an agenda for reform. Washington, DC, The World Bank, 1987.        

Correspondence
Christopher J.L. Murray
World Health Organization
1211 Geneva 27, Switzerland

¹The ex post distribution is affected by the extent to which some households did not purchase care because they could not afford it. Ceteris paribus, we expect total expenditure to rise with increased financial risk pooling, because some purchases of health care that could not be afforded out of pocket can be prepaid. To the extent that households do not purchase health care because they cannot afford it, population health will be less than it could have been.

²More formally, production efficiency is based on the concept of a Pareto optimality, in which production factors are allocated such that it is not possible to reallocate factors to produce more of one good without producing less of another.