Amenable mortality revisited: the AMIEHS study
Mortalidad tratable revisada: el estudio AMIEHS
Rasmus Hoffmanna, Iris Pluga, Bernadette Khoshabab, Martin McKeeb, Johan P. Mackenbacha and AMIEHS working group*
aDepartment of Public Health, Erasmus Medical Center, Rotterdam, The Netherlands
bLondon School of Hygiene & Tropical Medicine, London, United Kingdom
*The AMIEHS working groups consists of the authors and Grégoire Rey, Eric Jougla, Ragnar Westerling, Caspar W.N. Looman, José L Alfonso-Sánchez, Kersti Pärna,
Katrin Lang and Wolfgang Hellmeier.
The AMIEHS project-Amenable mortality in the European Union: towards better indicators for the effectiveness of health systems- has received funding from the European Union through the Executive Agency for Health and Consumers (EAHC) (agreement number 2007106).
Objectives: There is a renewed interest in health system indicators. In 1976 a measure of quality of healthcare, amenable mortality, was introduced by Rutstein. This indicator is based on the concept that deaths from certain causes should not occur in the presence of timely and effective healthcare. In the project "Amenable mortality in the European Union: toward better indicators for the effectiveness of health systems" (AMIEHS), we introduce a new approach to the selection of indicators of amenable mortality.
Methods: Based on predefined selection criteria and a broad review of the literature on the effectiveness of medical interventions, a first set of potential indicators of amenable mortality (causes of death) was selected. The timing of the introduction of medical innovations was established through reviews and questionnaires sent to national experts from seven participating European countries. The preselected indicators were then validated by a trend analysis that identified associations between the timing of innovations and cause-specific mortality trends and by a Delphi-procedure.
Results: After a short review of previous lists of amenable mortality indicators and a detailed description of the innovative procedure in the AMIEHS project we present a list of 14 causes of death that passed our selection criteria. We illustrate our empirical validation of these indicators using the examples of peptic ulcer and renal failure.
Conclusions: The innovation developed in the AMIEHS study is a rigorous new approach to the concept of amenable mortality that includes empirical validation. Only validated indicators can be successfully used to assess the quality of healthcare systems in international comparisons.
Key words: Avoidable mortality. Amenable mortality. International comparison. Healthcare assessment. Health system indicators. Healthcare effectiveness.
Objetivos: Actualmente hay un renovado interés por los indicadores para los sistemas de salud. En 1976, Rutstein propuso la "mortalidad tratable" como una medida de la calidad de la atención en salud, partiendo de que ciertas causas de muerte no deben presentarse si se cuenta con atención médica oportuna y eficaz. Un nuevo enfoque se presenta en el proyecto "Mortalidad tratable en la Unión Europea: en procura de mejores indicadores para los sistemas de salud" (AMIEHS, por sus siglas en inglés) para seleccionar indicadores de mortalidad tratable.
Métodos: Basándose en criterios predefinidos y en una extensa revisión de la literatura sobre la efectividad de las intervenciones médicas, se seleccionó un primer conjunto de indicadores potenciales de mortalidad tratable (causas de muerte). El momento de la introducción de las innovaciones médicas se fijó mediante revisiones y cuestionarios enviados a expertos de siete países europeos participantes. Se validaron los indicadores preseleccionados identificando la asociación entre la introducción de las innovaciones y el análisis de tendencias para causas de muerte específicas y usando un Delphi.
Resultados: Tras revisar anteriores listas de indicadores de mortalidad tratable y describir el procedimiento innovador en el proyecto AMIEHS, presentamos una lista de 14 causas de muerte que cumplen los criterios de selección. Luego ilustramos nuestra validación empírica usando como ejemplos la úlcera péptica y la insuficiencia renal.
Conclusiones: La innovación del estudio AMIEHS es una aproximación rigurosa a la nueva concepción de la mortalidad tratable que incluye validación empírica. Para evaluar la calidad de los sistemas de salud en comparación con otros países sólo pueden usarse con éxito indicadores validados.
Palabras clave: Mortalidad evitable. Mortalidad tratable. Comparación internacional. Evaluación de servicios de salud. Indicadores para los sistemas de salud. Efectividad de los servicios de salud.
The introduction of new pharmaceuticals and treatment methods during the late 20th century, and new and more effective ways of organizing care, mean that health care now contributes demonstrably to improved population health.1,2 However, as national health spending has tended to outstrip economic growth in most OECD countries in recent years, reaching 9% of GDP on average,3 it is important to quantify what this additional spending has achieved. The World Health Report 2000 identified health gain as a fundamental goal of health systems4 but was unable to assess the specific contribution of health care. It also stimulated an increasing interest in performance of health systems with improved tools being sought avidly by policy-makers seeking to determine whether they are getting value for money in health care systems.
In the early 1970s Rutstein introduced the concept of mortality amenable to medical care. His working group selected over 90 conditions as "sentinel health events" from which disease, disability or death "should not occur in the presence of timely and effective care".5 In 1977 and 1980, revisions of this list were undertaken.6,7 They have formed the basis for practically all subsequent studies on amenable mortality. Charlton was the first to apply the concept at the population level in England and Wales in 1974-78, also introducing the terms "avoidable deaths" and "conditions amenable to medical intervention".8 He narrowed the concept by excluding deaths that were not directly linked to medical care, e.g. deaths avoided by policies on tobacco control. The concept was developed further within the Health Services Research Program of the European Community in the 1980s. This collaborative action resulted in a European Community atlas of avoidable mortality in which the work of Charlton and colleagues was extended and the boundaries of health services were interpreted as encompassing primary care, hospital care and collective health services.9 In 2001 Tobias and Jackson produced an updated list of conditions derived from an expert consensus exercise in which the relative avoidability of death was distributed according to primary, secondary and tertiary actions.10
Although the concept underpinning amenable mortality has been widely accepted since its first introduction by Rutstein, it has also been criticized,11,12 mostly on the basis that the relation between health care interventions and amenable mortality has not been tested empirically. Critics argue that for amenable mortality to be established as a valid indicator of the outcome of health care, direct analyses of its effect on mortality are needed. This has been addressed in the existing literature by two means; first, in time series analyses, based on the implicit assumption that innovations in health care affect the trend in mortality and, second, geographically, by comparing mortality in regions with different levels of health care. Time series analyses show faster decline in mortality in recent decades from most of the conditions identified as amenable compared to non-amenable.13-15 This has been used to support the argument that at least part of the overall mortality decline is due to improvements in health care. Two studies of geographical variation also found mortality differences for amenable conditions correlated with availability of corresponding health care interventions among countries16 and regions.8Table 1 presents a summary of the progressive refinement of conditions amenable to health care.
Previous research applying amenable mortality to health system comparisons have differed in their choice of indicators (e.g. Rutstein, Charlton or Holland, sometimes with adaptations). They have also varied in how they define "health care". Finally, different upper age limits are used to define which deaths are considered amenable, which is justified because medical conditions may be amenable up to different ages, but which may also introduce a bias if different studies use different age limits. It has been noted that the lack of explicit criteria for selecting indicators of amenable mortality is an important shortcoming of the existing literature.12,14 The knowledge on indicators of amenable mortality is insufficient to believe that amenable mortality can serve as an indicator for the quality of health care because hardly any validation is done to reveal an empirical relation between the quality of health care and mortality outcome.
In the following method section we describe a systematic approach to select indicators of amenable mortality including an explicit list of initial selection criteria. We define health care as primary care, hospital care and personalized public health services (e.g. immunization and screening). The result section shows 14 causes of death that fulfilled the initial criteria, gives two examples how these causes of death were tested in an empirical validation analysis and presents the results of a Delphi procedure.
The AMIEHS project (Amenable Mortality in the European Union: towards better Indicators for the Effectiveness of Health Systems;http://amiehs.lshtm.ac.uk) has six aims, corresponding to the six levels in the project structure in figure 1. The outcome, a list of indicators of amenable mortality, should provide insights into quality of health care.
Criteria for the preliminary selection of causes of death
1. A substantial decline in mortality in a country with a well-developed health system.
2. The cause of a sufficient number of deaths so that rates are likely to be stable over time.
3. The existence of a distinct clinical innovation of proven effectiveness introduced in recent years.
Selection of the standards that must be achieved according to each of these criteria is inevitably arbitrary. Our choice of the magnitude of decline was 30%. We recognized that, among those conditions causing an appreciable number of deaths, even the best health care system could not prevent all deaths even if a highly effective treatment exists, so the figure should be less than 100%. Similarly, if the death rate has fallen by only a small amount in the presence of a generally well-performing health system, it is unlikely that health care is currently contributing, or has the potential to contribute much to the decline. For practical reasons a single country was used to check this criteria that should be of sufficient size to reduce the risk of spurious results arising from small numbers, be one where a continuous set of mortality data were available spanning over two decades, and be one where any changes in coding or other data artefacts were already known. We ultimately chose England and Wales. We must then specify the time period over which to measure any decline. Given the need to avoid problems from coding changes, we selected the period in which ICD-9 was in use (1979-2000). The choice of threshold for numbers of deaths takes account of the fact that, for many EU member states, the deaths would be likely to be substantially fewer than in England and Wales. After discussion, a threshold of 100 has been selected.
These criteria and standards were applied to each 3-digit ICD-9 code in 1979 and 1990 in England and Wales. Some adjustments are needed to take account of coding changes. One is a change in how perinatal deaths were coded; another is to include AIDS which was not in the initial version of ICD-9 but which, it was agreed from inspection of other data, would otherwise meet the criteria. Another modification was to group together the various forms of groin hernia, each having a separate code and falling below the threshold of 100 deaths, but clearly treated in a similar way.
Those conditions meeting the criteria were then inspected to determine inclusion in the next step. We excluded those that are: a) non-specific codes, such as "other disorders of the...", and b) conditions for which, on the basis of clinical knowledge, no specific intervention can be identified. For the rest, a series of systematic literature reviews was undertaken. It sought two types of evidence. First, evidence from well conducted observational studies (typically of populations or patients attending a facility or service) documenting a decline in mortality that could, with some confidence, be attributed to the intervention). Second were randomized controlled trials, again showing a decline in mortality of 30% or more. The evidence was graded on a four point scale: 4, evidence from systematic reviews or meta-analysis; 3, evidence from one or more randomized controlled trial; 2, evidence from observational studies; 1, consensus statements or expert opinions. We excluded innovations implemented before 1970 as being of little contemporary relevance in high-income countries.
Data collection on the timing of introduction of health care interventions related to pre-selected causes of death
Country specific information was obtained on the year of introduction and the process of diffusion within a country for each intervention. A questionnaire was developed and distributed in which representatives of the participating countries were asked to identify sources of information about the introduction of these interventions. This was answered by reference to a wide range of data sources such as national guidelines, committee reports, scientific papers, data on registration and sales of pharmaceuticals. By combining the data within a theoretical framework of "diffusion of innovation"17 further developed by Rogers,18,19 we determined when we would expect a favorable change in mortality at a population level in each country.
Validation of possible indicators of amenable mortality by mortality trend analysis
In order to relate the timing of innovations to changes in mortality a harmonized database was constructed for the period from 1970 to 2005 using mortality data from Estonia, France, Germany (East and West separately), The Netherlands, Spain, Sweden and the United Kingdom. This database comprises mortality data by year, 3-digit ICD-codes (revisions 8, 9 and 10), sex and 5-year age group, that were obtained from the statistical offices of each country. Known discontinuities due to changes in ICD-coding and recording were reviewed. Then, a new methodology was developed, based on the Polydect method,20 to identify abrupt changes in mortality, i.e. "jumps". Correction factors were applied to overcome the effect of these changes.21
Remaining (real) discontinuities in mortality trends were sought using joinpoint models based on linear spline regression (with the R software), which identifies "knots" where the trend changes significantly.22,23 This program also estimates the Percent Annual Change (PAC) for periods between knots. This enabled trends to be compared across countries and with the timing of introduction of innovations. First, we studied deaths at all ages and then limited the age range to ages from 0 to 74 in order to see in a sensitivity analysis whether this changed the results. To choose one age cut-off point for all causes of death is arbitrary, but we found two standard age limits in previous studies (65 and 75) and we choose the upper one because of rising life expectancy and improved effectiveness of medical treatments in higher ages. As we are interested in period effects, prior to running the analysis any possible cohort effects were sought by means of age-period-cohort (APC) analysis. We first examined whether favorable changes in mortality trends coincide with the period of expected mortality decline due to the introduction of a medical innovation. If a knot fell within this period, this was defined as a "match". Inevitably, some matches will occur by chance so we tested whether the number of matches found is significantly higher than chance. To draw a conclusion about associations between innovations and mortality, we counted the countries that show a match for a condition. If two thirds of all countries with valid data show the association, we interpreted this as evidence for an effect.
Despite taking a systematic and structured approach to the identification of potential indicators of amenable mortality, we recognize that a degree of judgment is still required to assess whether the conditions were useful in practice. To incorporate this judgment we used the Delphi method, in which individuals were asked their view of the appropriateness of the causes identified, supplemented by others that failed on one or a few criteria as well as conditions that, in the judgment of the investigators, are clearly not appropriate. We included 23 individuals from 16 countries who were either producers or users of evidence on health systems performance, seeking participation by both genders and health professionals and non-health professionals. The method involved two rounds and in each the respondents were asked to score the conditions on a point scale (1 = inappropriate, 9 = appropriate). Following previous research using nominal group methods to assess agreement, consensus was deemed to exist when all scores (except single outliers) were within a three point range. In the first round, individuals were presented with a summary, for each condition, of all information gathered in earlier steps. In the second round they were given the range of first round scores, anonymized except for their own.
Electronic atlas on amenable mortality
All indicators considered at different stages of the project, as well as indicators proposed by other projects on amenable mortality were made publically available through an interactive electronic atlas, constructed with the InstantAtlas software, documenting variations in amenable mortality (45 causes of death) between 31 European countries for the period 2001-2009 (http://survey.erasmusmc.nl/amiehs). Age-standardized mortality trends are presented for males and females separately using the European Standard Population.
Figure 2 shows the stepwise selection of causes of death based on the criteria outlined above. Of 644 three-digit ICD codes, 160 codes had 100 or more deaths in 2000 in the UK and, of those, 54 exhibited a decline in age-standardized mortality of 30% or more. The exclusion of 40 codes for different reasons shown in Figure 2 resulted in a preliminary set of 14 conditions and 18 related interventions which are shown in Table 2 and are taken forward to subsequent steps.
Figure 3 shows a graphical representation of the validation analysis for peptic ulcer and renal failure. In most countries mortality from peptic ulcer (Figure 3A) has decreased throughout the study-period. Only for men in France and for women in the UK there is a positive change in mortality that falls into the expected period of mortality decline, whereas for five countries we do not see this association. According to our criteria this example would lead to the conclusion that there is no sufficient evidence that population level mortality from peptic ulcer has been responsive to the introduction of Cimetidine. Mortality trends from renal failure (Figure 3B) are very diverse across the countries. We find increasing and decreasing trends, as well as trends that peak in the 1980s. Renal failure could not be studied in Estonia because of incomparable ICD coding. In the UK and Spain mortality declines in the expected period for both men and women. In Sweden we see this coincidence only for men. In three countries we do not see this match and therefore there is no sufficient evidence that population level mortality from renal failure has been responsive to the introduction of cyclosporin. The results of this mortality trend analysis for these two conditions serve as examples for the similar analysis of all 14 conditions with similar results: overall we have not found clear evidence for an association between innovations in health care and changes in mortality.
Table 3 shows the results of the Delphi procedure. Only for three conditions consensus could be reached. These were those where all scores (except single outliers) were within a three point range.
The present study is based on a rigorous process of constructing an up-to-date list of indicators of amenable mortality. We demonstrate how we got from all 644 three-digit ICD-9 codes to a preselection of 14 causes of death using explicit selection criteria. With two example conditions we illustrate the subsequent empirical validation analysis which shows that the association between medical innovations and changes in mortality is modest. Our Delphi exercise shows consensus among experts on three amenable conditions. This study uses empirical evidence to validate possible amenable mortality indicators, whereas in the past such indicators have often been selected by expert opinion only. The findings from this project help to assess and compare the performance of the health systems.
Some limitations need to be addressed: we focus on innovations introduced after 1970. Some major innovations like the introduction of antibiotics have taken place before and are therefore not considered. We also focus on medical innovations. Yet improvements in population health due to health care reflect three factors: innovations, improvement in the quality with which they are applied and expanded coverage. At the level of international comparisons over time it is not possible to take account of the other factors given the absence of adequate data. Third, we are looking at the effect of individual interventions, rather as if they are "magic bullets", although in reality many health gains are the consequence of incremental improvements. Fourth, in seeking evidence of the effect on mortality of specific innovations, we were limited as few trials have mortality as an end point, instead using intermediate measures. Fifth, we were constrained by the paucity of high quality observational studies attributing health gains to particular innovations. Sixth, although there is no alternative to using ICD codes at the three digit level, they are not designed for the purpose we are using them and the potential benefits of a therapeutic innovation may be distributed among many codes, such as those related to safer anesthesia. Finally, we adopt a narrow definition of the health system, excluding intersectoral actions. This is inevitable if using this method. A broader definition of the health care system may define more causes of death as potentially amenable. Not as a limitation of the amenable mortality approach, but as a factor that complicates the interpretation of observed trends in mortality, we want to mention that many factors other than innovations in health care influence mortality. Given the difficulty in precisely measuring these factors, it will remain a challenge to interpret mortality changes on the population level as a consequence of the particular change in health care. Socioeconomic differences in the trends of mortality and in the diffusion of medical innovation may hide socioeconomic differences in amenability. This could not be taken into account in this study. A recent study suggests that socioeconomic differences in amenable mortality do not reflect differences in access or quality of care.24
In the AMIEHS project the concept of amenable mortality is addressed in a completely different manner from previous studies by starting from all causes of death and working down to a limited number. Secondly, our selection of indicators is linked directly to changes in health care practice, addressing a previous criticism of the concept of amenable mortality. The indicators we identified help to highlight shortcomings in health care, but mortality rates from amenable causes must be used with caution as indicators of health care effectiveness in international comparisons. Elsewhere we suggest two possible alternative measures of amenable mortality25 and more in-depth analyses will be needed to identify the specific failures in health care systems. For policy makers the indicators identified in this study will offer the opportunity to put extra attention into specific causes of death. For researchers these indicators are hotspots giving rise to new investigations in the future.
R. Hoffmann and I. Plug are work package leaders in the project and drafted the manuscript. B. Khoshaba contributed to the analysis and to drafting the manuscript. M. McKee participated in the design of the study, performed analysis and contributed to the manuscript. J.P. Mackenbach participated in the design of the study and contributed to the manuscript. All authors read and approved the final manuscript.
Conflict of interests
1. Bunker JP, Frazier HS, Mosteller F. Improving health: measuring effects of medical care. Milbank Q. 1994; 72:225-58.
2. Mackenbach J. The contribution of medical care to mortality decline: McKeown revisited. J Clin Epidemiol. 1996; 49:1207-13.
3. OECD. Health system priorities when money is tight. OECD Health Ministerial Meeting. 7-8 October 2010. Paris: OECD; 2010. 205 p.
4. World Health Organization. The World Health Report 2000. Geneva: World Health Organization; 2000. 328 p.
5. Rutstein D, Berenberg W, Chalmers T, et al. Measuring the quality of medical care. N Engl J Med. 1976; 294:582-8.
6. Rutstein DD, Berenberg W, Chalmers TC, et al. Measuring the quality of medical care: revision of tables of indexes. N Engl J Med. 1977; 297:508.
7. Rutstein DD, Berenberg W, Chalmers TC, et al. Measuring the quality of medical care: second revision of tables of indexes. N Engl J Med. 1980; 302:1146.
8. Charlton JR, Hartley RM, Silver R, et al. Geographical variation in mortality from conditions amenable to medical intervention in England and Wales. Lancet. 1983; 26:691-6.
9. Holland W. European Community atlas of 'avoidable death' Oxford: Oxford University Press; 1988. (2nd edition 1993, 3rd edition 1997). 143 p.
10. Tobias M, Jackson G. Avoidable mortality in New Zealand, 1981-97. Aust NZ J Public Health. 2001; 25:12-20.
11. Carr-Hill RA, Hardman GF, Russell IT. Variations in avoidable mortality and variations in health care resources. Lancet. 1987; 1:789-92.
12. Westerling R. Studies of avoidable factors influencing death: a call for explicit criteria. Qual Health Care. 1996; 5:159-65.
13. Mackenbach JP, Looman CW, Kunst AE, et al. Post-1950 mortality trends and medical care: gains in life expectancy due to declines in mortality from conditions amenable to medical intervention in The Netherlands. Soc Sci Med. 1988; 27:889-94.
14. Mackenbach J, Bouvier-Colle M, Jougla E. Avoidable mortality and health services: a review of aggregate data studies. J Epidemiol Commun Health. 1990; 44:106-11.
15. Poikolainen K, Eskola J. The effect of health services on mortality: decline in death rates from amenable and non-amenable causes in Finland, 1969-1981. Lancet. 1986; 1:199-202.
16. Charlton JR, Velez R. Some international comparisons of mortality amenable to medical intervention. Br Med J. 1986; 292:295-301. (Clin Res Ed).
17. Ryan R, Gross N. The diffusion of hybrid seed corn in two Iowa communities. Rura Sociol. 1943; 8:237-52.
18. Rogers EM. Diffusion of innovations. London: Free Press of Glencoe; 1962.
19. Rogers EM. Diffusion of preventive innovations. Addict Behav. 2002; 27:989-93.
20. Zhang B, Su Z, Qiu P. On jump detection in regression curves using local polynomial kernel estimation. Pak J Stat. 2009; 25:505-28.
21. Rey G, Aouba A, Pavillon G, et al. Cause-specific mortality time series analysis: a general method to detect and correct for abrupt data production changes. Population Health Metric. 2011; 9:52.
22. Barendregt JJ, Looman CWN, Brønnum-Hansen H. Comparison of cohort smoking intensities in Denmark and the Netherlands. B World Health Organ. 2002; 80:26-32.
23. Otto SJ, Fracheboud J, Looman CW, et al, National Evaluation Team for Breast Cancer Screening. Initiation of population based mammography screening in Dutch municipalities and effect on breast-cancer mortality: a systematic review. Lancet. 2003; 361:1411-7.
24. Plug I, Hoffmann R, Artnik B, et al. Socioeconomic inequalities in mortality from conditions amenable to medical interventions: do they reflect inequalities in access or quality of health care?. BMC Public Health. 2012; 12:346.
25. Mackenbach JP, Hoffmann R, Khoshaba B, et al. Using 'amenable mortality' as indicator of health care effectiveness in international comparisons: results of a validation study. J Epidemiol Commun Health. 2012; in press.
26. Poikolainen K, Eskola J. Health services resources and their relation to mortality from causes amenable to health care intervention: a cross-national study. Int J Epidemiol. 1988; 17:86-9.
27. Westerling R. Trends in "avoidable" mortality in Sweden, 1974-1985. J Epidemiol Commun Health. 1992; 46:489-93.
28. Westerling R. Indicators of "avoidable" mortality in health administrative areas in Sweden 1974-1985. Scand J Soc Med. 1993; 3:176-87.
29. Westerling R, Gullberg A, Rosen M. Socioeconomic differences in 'avoidable' mortality in Sweden, 1986-1990. Int J Epidemiol. 1996; 25:560-7.
30. Simonato L, Ballard T, Bellini P, et al. Avoidable mortality in Europe 1955-1994: a plea for prevention. J Epidemiol Commun Health. 1998; 52:624-30.
31. Niti M, Ng TP. Temporal trends and ethnic variations in amenable mortality in Singapore 1965-1994: the impact of health care in transition. Int J Epidemiol. 2001; 30:966-73.
32. Nolte E, McKee M. Does health care save lives?. London: Nuffield Trust; 2004. 138 p.
Received 29 April 2012
Accepted 21 August 2012