BOOKS & ELECTRONIC MEDIA

Uwe E. Reinhardt^I; Tsung-mei Cheng^II

^IProfessor of Economics and Public Affairs, Woodrow Wilson School, Princeton University, Princeton NJ 08444–1013, USA
^IIPrinceton International Forum, Princeton, USA

Correspondence

The world health report 2000 – Health systems: improving performance

Geneva,World Health Organization, 2000. ISBN 924156198X, 215 pages, SFr 15, SFr 10.50 in developing countries

Here WHO attempts no less than to rank the vastly different health systems of 191 nations on two one-dimensional measures of performance: (a) ‘‘level of health,’’ represented by disability-adjusted life expectancy (DALE) and (b) an ‘‘index of overall health system performance’’. The latter is calculated as a weighted average of scores on five distinct dimensions: (1) the country’s DALE, (2) the ‘‘distribution of health’’ (based on child mortality distributions within countries), (3) the health system’s ‘‘responsiveness’’ to what people seek from it in terms of ‘‘prompt attention, dignity, autonomy, confidentiality,’’ and so on, (4) an index of the distribution of that ‘‘responsiveness’’ among socioeconomic classes, and (5) the degree of ‘‘fairness’’ with which the health system is financed. The weights for these five measures going into the ‘‘overall health system performance index’’ were culled from a survey of 1006 experts from 125 countries, about half of them on the staff of WHO.

The final rankings of countries on both of the two performance measures are not based on the actual values achieved by the nation, but on the ratios of the achieved values to the values that ought to have been achieved, given the country’s educational attainment and spending on health care. The denominator in this ratio was derived from an empirically estimated mathematical relationship that predicts, for any combination of national health spending and national educational attainment, the level of performance that would have been achieved by an efficiently run health system.

Because the ultimate rankings emerging from this study are the products of a whole series of inherently subjective analytic judgements on the specific measures of systems performance, on the weights to be attached to each measure and on the model used to compare actual with ideal performance, it is fair to query whether, on balance, so precarious an undertaking does more good than harm.

Before addressing that question in regard to the WHO report, it is well to keep in mind that the decision-makers in the socalled ‘‘real world’’ do prefer to have complex phenomena collapsed into one-dimensional indexes. Even professors at top universities despair of multi-line academic transcripts and prefer to see a student’s entire and often varied academic career collapsed into the single, highly dubious measure of the grade point average. Gross domestic product (GDP) is a similarly crude, flawed, one-dimensional indicator for national economic performance, as is quarterly earnings per share for a giant corporation. All of these simple measures are the products of whole hosts of precarious assumptions. Yet they are widely used, on the assumption that doing so does more good than harm. Can that assumption be made for the WHO report as well?

The chief virtue of the WHO report lies in the challenges it poses for its critics within the health services research community. Could these critics have done better? If so, precisely how? Or can these critics argue that quantitative assessments of this sort are never worth undertaking? In other words, are we stuck in a rut that allows physicians or politicians in every country to proclaim that theirs is ‘‘the best health system in the world’’ without being challenged by data? If that be the verdict of the research community, it would be good to have it flushed out into the open, and on paper.

On the other hand, there is reason to wonder whether more good than harm will have been done by the fanfare with which this report was injected into the public media and thence into the world of policy-making. Two requirements should have been met before the report was ready for a major media campaign.

First, the WHO research team should have been sure that their estimates are robust. Can they, in good conscience, make that claim? An artificially high ranking, for example, could take the wind out of the sails of desirable health-reform efforts. Similarly, an artificially low ranking could assign a bad grade to past reform efforts that were actually commendable. Rumour in the health services research community has it that France’s no.1 rank was driven in part by a flawed measure of national educational attainment. Under the methodology used by WHO, the more the level of educational attainment or of health spending is underestimated for a country, the higher will be the ratio of actual to ideal performance for that country and the higher will be the nation’s ranking.

Second, if the report is addressed to policy-makers, one must judge it poorly written. To be sure, it has a number of fascinating, if chatty, chapters; but these are only loosely connected to the actual work underlying this study. To see what was actually done, one must plough through the cryptic commentary that accompanies the tables in the Annex or dig up and read sundry sources cited in the references. Few policy-makers and even fewer journalists will go to that trouble.

To be useful as a policy analysis, the report ought to have started with the crisp executive summary that is now de rigueur among policy analysts, certainly in the United States. That summary would have presented the main conclusions emerging from the study and described, in layman’s terms, the methodology that was used to reach these conclusions. Most important of all, the executive summary should have contained the many caveats that must, in good conscience, accompany ambitious analyses of this sort.

Correspondence
Uwe E. Reinhardt
Woodrow Wilson School
Princeton University
Princeton NJ 08444–1013, USA
E-mail: reinhard@princeton.edu