Editorials

Epidemiological evidence: improving validity through consistency analysis

Colin D. Mathers,¹ Christopher J.L. Murray,² & Alan D. Lopez³

Information about the incidence and prevalence of diseases and injuries at the population level is frequently required for benchmarking, for advocacy of particular policies, to assist in setting funding priorities, for monitoring achievements towards internationally accepted goals and targets, and to guide technical strategies and responses. In this issue of the Bulletin (pp. 622–628), Kruijshaar et al. examine the use of incidence–prevalence–mortality (IPM) models to improve estimates of disease epidemiology.

The Global Burden of Disease (GBD) study (1, 2) developed explicit methods, including the DisMod software, to ensure internal consistency of epidemiological and mortality estimates for specific causes. WHO is now undertaking a new assessment of the GBD for the year 2000 (GBD 2000) and subsequent years (3). Explicit aims are to provide valid, internally consistent estimates of the incidence, prevalence, duration, and mortality for 135 disease and injury causes for major geographical regions, and to analyse the attributable burden of major physiological, behavioural, and social risk factors. The use of IPM models is crucial for achieving these objectives, and Murray & Barendregt have developed an improved software tool, DisMod II, for use in GBD 2000. This is available at no cost from WHO for use in other analyses (4).

A disease process can be described by a number of variables, such as incidence, prevalence, remission, case fatality, duration, and mortality. In principle, these can all be measured in populations, but with different degrees of difficulty. Mortality, for example, can be relatively easily measured using national vital registration systems, but the underlying cause of death can be misreported or misclassified. Nevertheless, many countries have cause-of-death statistics that usually constitute the most reliable and comparable source of disease data at population level.

Measuring incidence and prevalence of diseases, injuries, or impairments is usually much more difficult than measuring mortality. Data collection, when done, is often limited in time and geographical area; and problems of case definition abound. Not surprisingly, data are frequently incomplete, and their validity may be in doubt. In particular, given the different nature of the disease variables and the differences in the way the data are collected, it is inevitable that the observations are internally inconsistent. For example, when more incident cases than mortality are missed, the observed incidence will be too small for the observed mortality.

Both the GBD and national burden of disease studies have identified inconsistencies between incidence, prevalence, and mortality data for specific diseases of public health importance. Kruijshaar et al. give some examples in their paper. Using IPM analysis, the Australian Burden of Disease and Injury Study found that incidence estimates from a recent meta-analysis of incidence of dementia (5) were inconsistent with previous prevalence meta-analyses (6, 7) unless implausibly high case-fatality rates were assumed (8). Such inconsistencies in epidemiological evidence are common, and to maximize the validity and usefulness of such evidence for health policy, it is important to assess internal consistency.

Kruijshaar et al. conclude that use of IPM models does not permit determination of whether inconsistencies in empirical data are due to data inaccuracies or to past trends in incidence or mortality. To enable users to address this issue, both DisMod and DisMod II include an option to specify past trends in input parameters (which may be assessed from empirical evidence or expert opinion) in order to allow analysis of their effect on internal consistency for the time period of interest.

When IPM models are used to analyse empirical evidence, it is crucial to include excess mortality from other causes as well as direct mortality due to the cause of interest. For many diseases, and for injuries, there may be excess mortality due to a wide range of causes associated with common risk factors or with disease or injury sequelae or treatment. Thus, for example, for many cancers there is likely to be excess mortality from other causes such as cardiovascular disease, diabetes, and chronic respiratory disease, associated with common dietary risk factors and other behavioural, environmental, and genetic risk factors. Failure to include all excess mortality risk in the IPM model, as was the case with the analyses for the four cancers reported by Kruijshaar et al., will result in incorrect assessment of consistency between incidence and prevalence observations.

Health planning often proceeds on the basis of incomplete or biased epidemiological evidence that is not comparable across population groups. We argue that in all cases, health policy should be informed by valid and internally consistent epidemiological estimates. There may well be wide uncertainty around some estimates due to the lack of reliable information, but the uncertainty should be quantified and relayed to decision- makers to aid their planning. In this respect, IPM models provide an important tool to assist in the development of evidence for health policy.

1. Murray CJL, Lopez AD, editors. The Global Burden of Disease: a comprehensive assessment of mortality and disability from diseases, injuries and risk factors in 1990 and projected to 2020. Cambridge (MA); Harvard School of Public Health on behalf of the World Health Organization and the World Bank: 1996. Global Burden of Disease and Injury Series, Vol. 1.

2. Murray CJL, Lopez AD, editons. Global health statistics. Cambridge (MA); Harvard School of Public Health on behalf of the World Health Organization and the World Bank: 1996. Global Burden of Disease and Injury Series, Vol. 2.

3. Murray CJL, Lopez AD, Mathers CD, Stein C. The Global Burden of Disease 2000 project: aims, methods and data sources. Geneva,World Health Organization. 2001. GPE Discussion Paper No. 36. Available from: URL:&nbspwww.who.int/evidence/bod

4. DisMod II. Version 1.01. Geneva: World Health Organization. October 2001. Available for downloading at URL:&nbsphttp://www.who.int/evidence/dismod.

5. Jorm AF, Jolley D. The incidence of dementia: a meta-analysis. Neurology 1998;51:728–33.

6. Hofman A, Rocca WA, Brayne C, Breteler MM, Clarke M, Cooper B, et al. The prevalence of dementia in Europe: a collaborative study of 1980-1990 findings. International Journal of Epidemiology 1991;20:736-48.

7. Jorm AF, Korten AE, Henderson AS. The prevalence of dementia: a quantitative integration of the literature. Acta Psychiatrica Scandinavica 1987;76:465-79.

8. Mathers C, Vos T, Stevenson C . The burden of disease and injury in Australia. Canberra: Australian Institute of Health and Welfare; 1999.

¹Coordinator, Epidemiology and Burden of Disease, Global Program on Evidence for Health Policy, World Health Organization, 1211 Geneva 27, Switzerland (email:&nbspmathersc@who.int). Correspondence should be addressed to this author.

²Executive Director, Evidence and Information for Policy, World Health Organization, Geneva, Switzerland.

³Senior Science Advisor, Evidence and Information for Policy, World Health Organization, Geneva, Switzerland.

Ref. No. 02-0322