OPINIÃO OPINION

In July of 2000, the World Health Organization (WHO) published the World Health Report (2000 WHR). Its main objective was to compare the member countries in relation to the performance of their Health Systems. With this purpose, an index with 5 indicators was made. With this methodology, WHO proposed to begin a regular process to monitor the overall Health System Performance of the member countries.

The 2000 WHR represents an important contribution, putting on the agenda the responsibility to monitor the performance of the Health Systems of the member countries. Yet the document shows inconsistency, and a lack of transparency in the way it was done. With the utilization of questionable methods used for the evaluation, there is a need for a critical and detailed revision. There is a need to search for alternative solutions with regards to monitoring.

This response of the agency to the agenda of international organizations such as the World Bank, have been evident in the last decade (Wall, 1993). According to Richard Feachem, chief editor of the Bulletin of the World Health Organization, when WHO published the 2000 WHR, it was the first time it had ever assumed the role, which had been held by the World Bank throughout the 90's. This means an involvement in a more effective way in the formulation of the politics facing the Health Systems. According to the author, the administration assumed by WHO in 1998 has shown its intention of acting in non medical areas such as the economy, and the financing of the health area (Feachem, 2000).

The 2000 WHR was dedicated to analyze the systems of health services. The authors attempted to redefine the frontiers of the area and expand the "traditional" concept of health used by WHO so far. This choice shows a great innovation, as it reflects an awareness of WHO with the "new principles", stated by modernizing the debate by the contemporary sectional reforms.

The document draws attention to WHO's attachment to the agenda of sectorial reform. This redefines the role of the State in the provision of services and recommends changes in the public and private sector mix. WHO comes out in favor of the latter, thereby placing a question mark over policies which prescribe universal access to health services, considered utopian, and of limited effectiveness. It places emphasis on the ability and potential for health systems to protect poorer population sectors and promote social justice, and rejects more regressive financing models. The document defends the idea of gradual convergence towards what it calls the "new universalism", meaning the supplying of essential high quality services to the whole population, defined by the cost effectiveness criteria. This option implies the withholding in part of a number of specific technologies, leaving the private market to attend to more complex demands. According to the Report, this model would be more suitable for tackling the problems of inefficiency and inequality present to a greater or lesser extent in all health systems; it would also be an improvement on the restricted proposals focusing on the priority provisions of basic health care for the poor.

Seen from this angle, the document affirms the preferential learning of the WHO towards those health services which combine a mixture of public provision and regulated markets. Their need to meet consumer demand, and at the same time controlling supply, that are all subject to the cost-effectiveness criteria. Furthermore, "traditional" schemes would be abandoned in favor of "market-oriented reforms". In these respects, the Report undermines previous initiatives propounded by the WHO itself such as the Health for All in the Year 2000 and Primary Health Care in the 1970's.

The idea of health as "complete physical and mental well being", is opposed by the emphasis on new elements and thoughts which state, "To obtain from the system the best medical health level as possible, (goodness) with the least differentiation possible between individuals and the population groups" (fairness). The vital systems that need to function include: provision of services, resource generation, responsible management, and financing of the system (stewardship). This would be directed to obtain better levels of health and to overcome disparities (WHO, 2000). Along with this, the 2000 WHR affirms that the fundamental change in ideology is greater emphasis on the present choices and the individual responsibilities. Politically, this means to distinguish the expectations about what has to wait for the State, in terms of social benefits in the health area.

Within such a health system structure, the role of governments is seen as crucial in the funding of services, and in the formulation and regulation of policies. What is suggested is the convergence of systems in terms of public financing and regulation, but without this being necessarily accompanied by State distribution or a provision of the health services. The latter would be left to a combination of governments, health service providers and individuals to face and overcome the traditional shortcomings of both market and governments in this sector. Modifications in the incentives structures are also recommended.

The proposed range of new indicators would measure the performance of the health systems according to these coordinates and permit monitoring of the reform processes towards the above ends. Meanwhile, even these functions of the systems often fail to find an echo in the variables utilized for performance evaluation of health systems in Member Countries.

As for funding models and client definition in the public systems, WHO adopts the World Bank formulation which recommends developing countries to concentrate their actions on the "necessary and possible." The percentage of pre-payment in respect to total health expenditure is used as an indirect indicator of the system's coverage, but no attempt is made in the Report to show what percentage of the population is covered by some public or private health scheme, nor the amount and services covered by each of the schemes. It is indirectly assumed as a given premise that low out-of-pocket expenditures by households is associated with membership in some kind of health system; and that such health coverage implies having one's health requirements satisfied. Both assumptions do not necessarily correspond to the reality of the situation.

WHO has placed excessive confidence in the measurement of the effects of health systems by basing research on only five indicators. These indicators fail to take into account the profile of services supply; utilization of services in relation to health needs, as well as the different regulatory structures applied in each country. They also fail to account for the results of specific reform processes.

At the time this article was written, more detailed methodological assessment was not possible due to a lack of full and adequate reference throughout the document to the parameters used to estimate the indicators; and to the sources of information still under review. Nonetheless, given the preliminary status of this Report, the questionable validity of the indicators as comparative measures of health systems performance and the reduction of all the results to a single composite Index, the 2000 WHR cannot be used for briefing governments, economic agents and civil associations as proposed. The questionable results arising from the final classification, and the fact that this Index is out of line with the reality in different countries, confers upon the Document a political dimension which exposes its weaknesses. These weaknesses are described below in more detail.

Analysis of indicators

In the quantitative analysis of health indicators estimated for a number of countries for comparative purposes, a range of different objectives must be taken into account. The most relevant of these are: (a) to analyze whether the health indicator is actually succeeding in expressing quantitatively, what it is proposed to measure in its theoretical concept; (b) to understand the methodology behind the construction of the indicator, including in this the sources of information used for the calculation; (c) to verify whether the indicator is being constructed uniformly across the countries, with identical accuracy, and the same calculation methodology; (d) and to verify the feasibility of periodically estimating the indicator, so that monitoring remains viable over time.

The methodology employed in the comparative evaluation of health services performance of the Member Countries is based upon five indicators, as follows: Health Level (DALE), Health Distribution, Responsiveness Level, Responsiveness Distribution and Fairness in Financial Contribution, which are respectively weighted at, 25%, 25%, 12,5%, 12,5%, and 25%. Together they serve to comprise the Index of Overall Health System Performance.

The elements mentioned above were taken into consideration during our examination of each of the individual five indicators. First, however, it is worth drawing attention to the key problem presented in the 2000 WHR Statistical Annex: the lack of information available for carrying out the calculation of the indicators used for building the Index which classifies health system performance of Member Countries. Among 191 countries, only 5, (including Brazil) contained the complete data required to calculate all the five indicators.

Given the lack of health information which should have been available for calculating the indicators, estimates were obtained using data on poverty, education and income inequality. The net result of this was to make comparisons of health system performances based upon socioeconomic indicators. The most serious case refers to the estimate given for the indicator, Fairness in Financial Contribution (FFC). Of the 191 countries, only 21 were able to supply relevant information. The estimates for the remaining 170 countries were arrived at on the basis of predicted multiple regression values, using as independent variables the "fraction of health spending which is public" and "income distribution" (measured by Gini coefficient), and a binary variable indicating whether a given country had been Communist or not. This regression presented a very low adjustment, where only 26% of the total variation was explained by the three co-variables (R2 = 0.26). No other factor was sought in order to obtain a better adjustment. Neither was the low adjustment taken into account when the 191 Member Countries were classified via the FCC indicator, which basically measures a composite of the three co-variables employed in the regression.

Health Level and Health Distribution

As its name suggests, the Disability-Adjusted Life Expectancy at Birth (DALE) embraces the life table measurements of years lost through incapacity, by measuring life expectancy according to the number of healthy years. Although this is an interesting idea, the calculation of the percentage of years lost through incapacity in each country could not be made accurately, and has to be estimated for determined groups of countries. In other words, the percentage deduction through incapacity was taken as a constant in all the countries in a specific group which had similar levels of life expectancy at birth. Furthermore, the percentage of years deducted for incapacity is higher according to the increased degree of poverty existing in that country, inverse and linearly proportionate to life expectancy at birth. As a result, the DALE (2000 WHR, Table 5) and life expectancy (calculated as an average of life expectancies at birth for males and females - 2000 WHR, Table 2), has a very high correlation coefficient which is almost equal to 1 (r = 0.996). It can be concluded that the use of the traditional indicator, "life expectancy at birth" would produce practically the same results. In addition, the Report throws little light on the sources of information used to obtain data in each country. Nor is it clear about its calculation methodology, especially as regards those countries where vital statistics are incomplete.

As far as distribution of health is concerned, the WHO Report attempted to measure health inequality of the populations. In this respect, an indicator was used which expresses individual chances of survival in infancy, defined on the basis of indicators of inequality of health propounded by Gakidou et al. (2000). This indicator is expressed mathematically by:

where IID represents inter-individual differences, y represents health expectation and m the median value in the population. The parameters a and b are adjusted to give greater weighting to the variability of observations or for the mean. In this way, the higher the value of IID, the higher the health inequality. In the specific case of the calculation of the health inequality indicator, the values of a = 3 e b = 0.5, were used.

In spite of the innovative character of this type of indicator, which attempts to measure health inequalities of populations and not a central descriptive value of the average level of health in a particular country, it is necessary to point out that the use of this indicator to evaluate health systems performance is heavily constricted. Recalling that the IID calculation refers to the measurement of health inequality by means of the variability of individual information, this type of indicator incorporates in its estimate the social inequalities of the population, which signifies that the population has a significant influence on the IID estimate.

By way of example of the effect of the make-up of the population on the IID, it is worth considering the simulation presented in Table 1 below. The simulated population, in time t1, presents life expectancies at birth differentiated by social strata, numbered from 1 to 5, from the poorest to the richest. In time t1, the distribution of the population by social strata is set out in the second column of the same table. It is also worth observing that in time t2, the life expectancies increased by at least one year, and that these increases were distinguished by social strata. They were also higher (2 years) in the two poorest social strata. But let us suppose that in time t2 social inequality of the population actually increased, so that the make-up of the population was modified at both ends of the spectrum, as presented in the fourth column of Table 1. Calculating for t1 and t2, the mean values of life expectancy at birth, the standard variations and IID, it can be seen that for t2, the standard deviation increased as did the proposed Index of health inequality. In this case, undoubtedly the individual inequalities of the health level of the population increased due to the increase in social inequality and not as the result of health system performance, which supposedly performed better among the more disadvantaged sectors of the population.  

In the same context, in a given population in which there are large sectors living in poverty and the health services in these sectors perform poorly but they will be considered more homogeneous (and therefore will come out lower on the index) than a health system performing in a population with a lower concentration of poverty, since the health inequalities due to social conditions will have little weight in the calculation of the IID.

In Table 2, an example of two populations; A and B is given. Health inequality by social strata expressed by life expectancy at birth is evidently higher in population B. However, using a simulation of a population with a large concentration of poverty, based upon IID (3, 0.5), it can be inferred that population A has a higher degree of health inequality than population B. This means that the IID calculated in countries with a high incidence of poverty and a precarious performance of the health systems could be under-estimating inequality of the performance by social conditions, disguised in effect by the homogeneity of poverty.

It is interesting to note that in the absence of available health information for the calculation of health inequality indicators. The estimate is arrived at by means of the employment of data referring to poverty, level of educational attainment and income concentration as the Gini co-efficient. From this, it can be deduced that the indicators calculated in this way have one aim: which is to attract attention to existing social disparities in a given population, evidently associated with health inequality in the population, but not necessarily linked to inequality in the actual performance of the health system.

The question arises whether the indicators used to measure health inequalities in a population are truly appropriate for evaluating inequalities in the performance of actual health systems in a given country. As seen from the simulations presented previously, the composition of the population according to socioeconomic strata has had such an importance in the estimations of health inequality indicators, and as such, have little to do with the performance of the health system in the narrowing of social gaps.

Level and distribution of "responsiveness"

The Report defines responsiveness as the way the system functions in relation to the aspects that are not related to health. These include the satisfaction or dissatisfaction of the populations' expectations in relation to the treatment that should be done by the providers of the prevention service - healing and non-individualized - (WHO, 2000). The indicator gives privilege to the service of the requests to the health needs. The assumption that the service will attend to personal preferences as a fundamental condition in the utilization of health services is questionable. Although the personal preferences has an important role in the demand for service, the explanatory model is much more complex than a simple preference. It depends on factors associated with the services offered, of the profile of the health needs, and of the social composition of the population.

In the 2000 WHR, the study assessed the respect for dignity, confidentiality, and autonomy of the individual regarding the services. As variables are properly geared to the area of patient care, it evaluates the immediate care given in the case of emergencies, and in that of non-emergency situations, (such as length of wait for treatment) the comfort of surroundings where patients are cared for, access by family members and friends to the patients, and the ability to choose one's provider freely.

In the neoinstitutionalist analytic models (Przeworski, 1999), the responsiveness is usually defined as the capacity of governments to take decisions similar to the ones of the citizens who are fully informed. Reproducing the analytic schemes of the principal agents model, where the citizens are above the politics and bureaucrats in the system. In general the responsiveness can be focused in the regulatory political context. Considering the market flaws (presences of extremities, asymmetric information and the formation of monopolies) the State intervention, done by the regulatory politics is crucial for the accomplishment of the political functions mentioned before. In this way the capacity of servicing the citizens expectations (responsiveness), depends on the adequate control of the government's flaws (regulatory flaws). This happens usually because asymmetric information, and a total lack of knowledge of the transactional costs along with administrative and political inefficiency (Laffont & Tirole, 1994). The most adequate regulatory schemes combine a direct and normative State regulation with the group incorporation interested in the results of the politics, ("tripartism and multipartism") and the many levels of market delegation, (auto-regulation). The adaptation of regulatory politics to the characteristics of each economy sector and the varied markets configures schemes of responsive regulation (Ayers & Braithewaite, 1992). In the report, the concept is addressed mildly taking out of the State regulatory agenda the notion of Citizen's Rights.

This indicator represents an important contribution of WHO towards comparing systems, but it also involves a number of problems. These are of two types of problems. The first is the problem of the lack of consistent and comparable national data between countries. This has led to an exaggerated dependence upon responses to various questionnaires by consultants who for their part, do not have all the information required to make consistent deductions. As the Report stresses, the development of this indicator, given the fact of its relative newness, depends on the effective carrying out of national countrywide or micro-regional surveys. The second problem is external to the indicator, and arises from the non-utilization of resolution-sensitive measurements of the health needs in the comparisons performed. According to the Report, these factors would be picked up by the indicators of health results (level and distribution), which are questionable. Having recourse exclusively to the epidemiological indicators presented, it is inconsistent insofar as the outcomes of health care service utilization are not fully captured by these statistics.

The Report evaluates the level and distribution of responsiveness among different member countries, Brazil stayed in positions of 130^th and 131^st place for levels of responsiveness and 84^th and 85^th place for the distribution of responsiveness. Studies were made in only 35 countries, and projections were made for the other countries. There were 1,006 questionnaires used, (1/2 among the WHO staff) to define the score of these responsiveness items. The questionnaires were applied to groups of at least 50 main informants per country. A total of 1,791 surveys were completed with a punctuation of 0-10 per item, and the results obtained were corrected by sex, and government relation, and political freedom. Still, in relation to the methodology used, a research done with 1,006 informants of 125 countries, (half of the answers were given by WHO technicians) it defined a final weight of 25% to the items responsiveness in the Composed Indicator.

Many problems are observed in the way this indicator was estimated in the 2000 WHR. The main indicators are: the research with the main informants should have been a complement the composed data, the number of questionnaires answered was below the expectation in countries such as Brazil with 33 answered. Other factors include: compromising the representation related to the initial model, the survey utilized was too open to be used in countries of large territory, regional diversities, along with public and private automated, systems. This damaged the comparability of the results, the analysis does not take into consideration the political aspects, (relationships between groups, leaderships, boards and institutions) that interfere in the expectations between the countries reflected in the local, social, and political agenda. The study underestimates the importance of the results of opinion research, the problems are not solved in an advantageous way by the methodology utilized, it does not consider adequately the importance of the medical aspects, (diversity and complexity of demand, capacity of solving problems) reaching the expectations of the user in the responsiveness of the Systems.

Fairness in the financial contribution

For 2000 WHR, the financial system is fair when all homes use, in health, the same percentage of the available income, (with the exception of meal expenses) not considering your health conditions or utilization of a health system. This approach deviates from the "vertical equity". This concept assumes that a fair financing system is the one where the amount paid by the homes increases according to the income level (the greater the greater percentage to be paid). The indicator that measures fairness in the home contributors of financial health systems, was created according to the percentage of expenses with health of the homes in relation to the income available.

The available income was estimated to the gross income of homes, meal expenses not included. The disparities in this contribution were calculated using the variability of the percentage of expenses in each home in relation to the average percentage.

With the utilization of the available income as a base to estimate the percentage of health expenses in each home, health systems with resources obtained progressively will be evaluated as unfair, if there is important disparities in the income distribution.

Reasoning similar to that already discussed in relation to the indicators of health can also be applied to the indicator called Fairness of financial contribution (FFC). In this case, the estimate was effected on the basis of percentage household expenditure on health (Household contribution to the financing of the health system - HFC), related to the so-called "permanent income"(defined as the spending of family members minus that spent on food). The disparities of expenditure on health (FFC) measured by means of the sum of the variability of individual percentages in relation to the mean percentage expenditure, and is subject to significant modifications depending on the make-up of the population by social strata. In the same way as with IID, the FFC reflects basically the inequalities in society.

A further problem with the FFC indicator arises from the sources of information employed: in fact, in only 21 of the 191 Member Countries, (Table 7 in the Report) the FFC was calculated using information obtained from household surveys (in many of the cases, Standard of Living Surveys - PPV) containing questions referring to household spending. For the remaining countries, having data on only a fraction of the expenditure on health which is public, the Gini Index, and the fact of the country being or not being Communist was used. Furthermore, it is not known how distribution of the tax burden was estimated among households. (based on the slice of family expenditure related to the payment of taxes and various obligatory contributions).

It is also worth pointing out that while the WHO predicates a conceptual debate on the "fairness" of the system of sectorial funding, indicating the need to analyze among other variables the progressiveness or regressiveness of the tax system, this consideration was completely left to one side, and the FFC as outlined above was the only element used in the research.

Analysis of the progressiveness or regressiveness of the sectorial financing system, seen in the light of family or household budgets, becomes thoroughly distorted, since it is able to take into account only those taxes which form part of the family spending. In this way, when the tax burden is viewed from the angle of family spending, the progressive taxes incurred by companies or individuals are not taken into account, whereas households do pay in accordance with their contributory capacity. This is exactly the situation in Brazil as regards for the two main taxes which finance the public health system in our country: the Provisional Contribution on Financial Movements (CPMF) and the Contribution on Company Liquid Profits (CLLE). These are basically progressive taxes in principle and account for over 40% of the funding for the public health sector at the federal level, which provides more than 70% of the total funding for the Single Health System (SUS) throughout the country. Therefore, the most valid way to evaluate the level of progressiveness of the funding system for the health system would have been to carry out analyses based upon funding resources which effectively find their way into the health sector, in Brazil's case at the three levels of Government.

It is worth pointing out on the other hand that the FFC does not reflect inequalities in the use of the health services, nor does it insofar as the relationship between this use, show the levels of need in the population concerned. In this way, a single country can in fact present a good health service performance as measured by the FFC and at the same time exhibit huge inequalities in the utilization of the health services. The latter aspect does not appear to have been incorporated into any of the indicators which comprise the Composite Index, (Overall Health System Performance) insofar as the indicator which measures the state of the population's health, (DALE) and presents low specificity as a measurement of the health system's performance. For example, in Brazil, deaths resulting from the high murder rate among young males has impacted negatively on life expectancy rates at birth. Moreover, life expectancy figures are extremely sensitive to changes in income concentration.

The Report presents indicators which in spite of not being incorporated in the calculations for the Composite Index on health services performance, require comment in respect to the lack of clarity regarding the items raised, for example: private spending, out of pocket spending, private consumption, total consumption, total private consumption and so on. In Brazil's case, the high share attributed to private spending according to the Report represents 51.3% of the total expenditure on health and would basically comprise out of pocket spending (45.6%), while in reality private expenditure in Brazil related to the purchase of private health insurance and medical plans (pre-payment) absorbs a percentage approximately equal to the federal expenditure on health. In the light of this, out of pocket spending would appear to be over-estimated.

Composite Index (Overall Health System Performance)

As regards to the Overall Health System Performance Index, it is worth commenting on the manner in which it was calculated. In the first place, it is important to point out that the median level of health, as measured by the DALE method, is affected by the unequal distribution of the population, as recent studies have shown (Lynch, 2000). Attributing the weighing of 25% to the median health levels, 25% to the indicator of health distribution and 25% to the indicator of Fairness in Financial Contribution, which are considerably influenced by the effects of the population composite indicator which basically weights the social inequalities existing in the population. By doing so it over-estimates the role of income distribution to the detriment of a concrete evaluation of the true performance, and coverage of the health system as it affects the whole population.

In summary, the following are noted as the main problems with regard to the calculation of the indicators proposed in the 2000 WHR to evaluate the performance of health systems: (a) the choice of using inter-individual health distribution to assess health inequalities without taking into account the composition of the population is one reason why the Index does not adequately assess the inequalities of the health systems; (b) the over-emphasis on the role of social disparities in the estimates that, under-estimate the contribution made by the performance of the health system itself; (c) the absence of data for all the Member Countries, which has occasioned the use of estimates of indicators by methodologies that presents limitations and are controversial; (d) the comparison of countries by means of indicators calculated with different methodologies; (e) the lack of transparency in the methodological procedure used to calculate some of the indicators, such as that employed for estimating the Fairness of Financial Contribution.

In conclusion, because of the questions which need to be clarified in regards to the indicators employed, and the problems arising from the sources of information used along with the resulting need to calculate estimates for the majority of the countries, as well as criticism of the methodology applied to the elaboration of the estimates referred to and the resulting inaccuracy of the latter, it would seem inappropriate to list Member Countries in order of individual rankings.

References

AYRES, I. & BRAITHWAITE, J., 1992. Responsive Regulation: Transcending the Deregulation Debate. Oxford: Oxford University Press.         

FEACHEM, R. G. A., 1999. Health Systems: More evidence, more debate. Bulletin of the World Health Organization, 78:715.         

GAKIDOU, E. E.; MURRAY, C. & FRENK, J., 2000. Defining and measuring health inequality: An approach based on the distribution of health expectancy. Bulletin of the World Health Organization, 78:42-54.         

LYNCH, J. W.; SMITH, G. D.; KAPLAN, G. A. & HOUSE, J. S., 2000. Income inequality and mortality: Importance to health of individual income, psychosocial environment, or material conditions. BMJ, 320:1200-1204.         

LAFFONT, J.-J. & TIROLE, J., 1994. Theory of Incentives in Procurement and Regulation. Cambridge: MIT Press.         

PRZEWORKI, A.; STOKES, S. & MANIN, B., 1999. Democracy, Accountability, and Representation. Cambridge: Cambridge University Press.         

WALT, G., 1993. WHO under stress: Implications for the Health Policy. Health Policy and Planning, 24:125-144.         

WHO (World Health Organization), 2000. The World Health Report 2000: Health Systems - Improving Performance. Geneva: WHO.