Résumé

Objectif

Examiner systématiquement la fiabilité des données hospitalières sur la cause des décès et encourager l'examen périodique de ces données en utilisant une méthode standard.

Méthodes

Nous avons utilisé Google Scholar, Pubmed et Biblioteca Virtual de la Salud pour rechercher des articles en anglais, en espagnol et en portugais, qui ont rapporté des études de validation des données sur la cause des décès. Nous avons analysé les résultats de 199 études qui ont utilisé des examens de dossiers médicaux pour valider la cause de la mort déclarée sur les certificats de décès ou par le système d'enregistrement de l'état civil.

Résultats

Les études passées au crible ont été publiées entre 1983 et 2013, et leurs résultats ont été publiés en anglais (n = 124), en portugais (n = 25) ou en espagnol (n = 50). Seules 29 études ont rempli nos critères d'inclusion. Parmi ces études, 13 ont étudié les schémas des causes de décès au niveau de la population – en vue de corriger les fractions de mortalité selon l'âge – alors que les 16 autres ont été entreprises pour identifier les contradictions dans le diagnostic des maladies spécifiques, avant et après l'examen du dossier médical. La plupart des études sélectionnées ont rapporté des erreurs importantes de diagnostic pour les causes de décès dans les hôpitaux. Il y avait une grande variation dans les méthodologies d'étude. De nombreuses études ne décrivaient pas suffisamment les méthodes utilisées pour pouvoir évaluer la reproductibilité ou la comparabilité de leurs résultats.

Conclusion

L'hypothèse selon laquelle les causes de décès sont déclarées de manière précise dans les hôpitaux est sans fondement. Pour améliorer la fiabilité et l'utilité des causes de décès déclarées, les gouvernements nationaux devraient effectuer des examens périodiques des dossiers médicaux afin de valider la qualité des données sur la cause des décès dans leurs hôpitaux, en utilisant une méthode standard permettant d'obtenir des résultats comparables au niveau international.

Resumen

Objetivo

Revisar sistemáticamente la fiabilidad de los datos de los hospitales sobre las causas de las muertes y promover revisiones periódicas de estos datos por medio de un método estándar.

Métodos

Se realizaron búsquedas de artículos en inglés, español y portugués en Google Scholar, Pubmed y la Biblioteca Virtual de la Salud sobre estudios que informaran acerca de la validación de datos sobre las causas de las muertes. Analizamos los resultados de 199 estudios que habían empleado revisiones de registros médicos para validar las causas de las muertes registradas en los certificados de defunción o por el sistema de registro civil.

Resultados

Los estudios seleccionados se habían publicado entre 1983 y 2013 y sus resultados aparecían en inglés (n = 124), portugués (n = 25) o español (n = 50). Solo 29 de los estudios cumplieron los criterios de inclusión. De estos, 13 habían examinado las causas de muerte en la población con el fin de corregir las fracciones de mortalidad por una causa concreta, mientras que los otros 16 pretendían identificar las discrepancias en el diagnóstico de enfermedades específicas antes y después de la revisión de los historiales médicos. La mayoría de los estudios seleccionados informaron acerca de diagnósticos sustancialmente incorrectos de las causas de muerte en los hospitales. Las metodologías de estudio mostraron una amplia variación. Muchos estudios no describieron los métodos utilizados de manera suficientemente detallada como para poder evaluar la reproducibilidad o la comparabilidad de sus resultados.

Conclusión

La suposición de que los hospitales notifican adecuadamente las causas de la muerte es infundada. A fin de mejorar la fiabilidad y la utilidad de las causas de muerte notificadas, los gobiernos nacionales deberían llevar a cabo revisiones periódicas de los historiales médicos con objeto de validar la calidad de los datos hospitalarios sobre las causas de muertes por medio de un método estándar para lograr resultados comparables a nivel internacional.

ملخص

الغرض إجراء استعراض منهجي لموثوقية بيانات المستشفيات الخاصة بسبب الوفاة وتشجيع إجراء استعراضات دورية لهذه البيانات باستخدام طريقة موحدة.

الطريقة تم إجراء بحث في Google Scholar وPubmed وBiblioteca Virtual de la Salud للحصول على الموضوعات المكتوبة باللغات الإنجليزية والإسبانية والبرتغالية التي أبلغت عن دراسات تحقق للبيانات بشأن سبب الوفاة. وقمنا بتحليل نتائج 199 دراسة استخدمت استعراضات السجل الطبي للتحقق من سبب الوفاة المبلغ عنه في شهادات الوفاة أو عن طريق نظام تسجيل الأحوال المدنية.

النتائج تم نشر الدراسات التي تم فحصها بين 1983 و2013 وتم الإبلاغ عن نتائجها باللغة الإنجليزية (العدد = 124) أو البرتغالية (العدد = 25) أو الإسبانية (العدد = 50). ولم تستوف سوى 29 دراسة معايير الإدراج الخاصة بنا. وتناولت 13 دراسة من هذه الدراسات سبب أنماط الوفاة على الصعيد السكاني - بغية تصحيح نسب معدل الوفيات المحدد بالأسباب - في حين تم إجراء الست عشرة دراسة الأخرى لتحديد أوجه التباين في تشخيص أمراض معينة قبل استعراض السجل الطبي وبعده. وأبلغت معظم الدراسات المحددة عن تشخيص خاطئ على نحو كبير لأسباب الوفاة في المستشفيات. وكان هناك تفاوت كبير في منهجيات الدراسة. ولم تصف العديد من الدراسات الطرق المستخدمة بتفصيل كاف يتسنى معه تقييم قابلية تكرار نتائجها أو مقارنتها.

الاستنتاج لا يوجد أساس لافتراض دقة الإبلاغ عن أسباب الوفاة في المستشفيات. ولتحسين موثوقية أسباب الوفاة المبلغ عنها وفائدتها، ينبغي أن تجري الحكومات الوطنية استعراضات دورية للسجلات الطبية للتحقق من جودة سبب بيانات الوفاة الذي يتم الإبلاغ به بواسطة المستشفيات، باستخدام طريقة موحدة لتحقيق نتائج يمكن مقارنتها على الصعيد الدولي.

摘要

目的

系统回顾医院死因数据的可靠性并鼓励使用标准方法定期回顾这些数据。

方法

我们搜索Google Scholar、Pubmed和Biblioteca Virtual de la Salud中报告死因数据验证研究的英语、西班牙语和葡萄牙语文章。我们分析了199个研究的结果,这些研究利用医疗记录回顾来验证死亡证明报告的死因或由生命登记系统报告的死因。

结果

筛选的研究发表在1983年和2013年之间,其研究结果以英语(n = 124)、葡萄牙语(n = 25)和西班牙语(n = 50)发表。只有29项研究满足我们的入选标准。这其中,13项研究在人口水平调查死因模式(旨在纠正死因别死亡率分数),而另外16项识别特定疾病的诊断在病历回顾前后的差异。大多数的所选择的研究报道了大量对医院死亡原因的错误诊断。研究方法存在很大差异。许多研究未详细描述所用的方法,不足以评估结果的再现性和可比性。

结论

医院准确报道死因的假设是没有根据的。为改善死因报告的可靠性和实用性,各国政府应该使用标准方法完成定期的病历回顾来验证医院死因数据的质量,以取得国际可比性的结果。

Резюме

Цель

Провести систематический обзор достоверности больничных данных о причинах смерти и способствовать проведению периодических обзоров таких данных, используя стандартный метод.

Методы

Был выполнен поиск статей на порталах Google Scholar, PubMed и Biblioteca Virtual de la Salud, написанных на английском, испанском и португальском языках, в которых приводились проверочные исследования данных о причинах смерти. Были проанализированы результаты 199 исследований, в которых были проведены обзоры медицинских записей для подтверждения причин смерти, указанных в свидетельствах о смерти или в записях актов гражданского состояния.

Результаты

Отобранные исследования были опубликованы в 1983-2013 гг. и их результаты были представлены на английском (n = 124), португальском (n = 25) или испанском (n = 50) языках. Только 29 из этих исследований соответствовали критериям включения. Из них в 13 исследованиях рассматривались причины смертей на популяционном уровне — с целью коррекции долей конкретных причин смертности, а еще 16 были проведены для выявления расхождений в диагностике конкретных заболеваний до и после проверки медицинской документации. В большинстве из выбранных исследований сообщалось о значительных ошибках в установлении причин смерти в больницах. Методологии исследований значительно различались между собой. Многие исследования не содержали достаточно подробного описания используемых методов, чтобы можно было оценить воспроизводимость или сопоставимость их результатов.

Вывод

Предположение, что в больницах ведется точная отчетность о причинах смерти, является необоснованным. Для повышения достоверности и полезности данных о сообщаемых причинах смерти национальные правительства должны проводить периодические обзоры медицинских записей с целью проверки качества данных о причинах смерти, предоставляемых больницами, используя стандартный метод для достижения международной сопоставимости результатов.

Introduction

The poor state of health information systems and, particularly, mortality statistics in many countries is widely documented in the literature and in country reports to the World Health Organization (WHO).¹1 Gaete-Darbó A. Appraisal of vital statistics in Latin America. Milbank Mem Fund Q. 1964;42(2):86–117. doi: http://dx.doi.org/10.2307/3348659 PMID: 5889230
https://doi.org/10.2307/3348659... ^–55 World Health Organization. Annual epidemiological and vital statistics 1939-1946. Part 1: vital statistics and causes of death. New York: Columbia University Press; 1952. However, health systems worldwide depend on reliable information about causes of mortality to be able to respond effectively to changing epidemiological circumstances. Such responses depend critically on accurate data to guide decision-making.⁶6 Health Metrics Network. Assessing national health information systems: an assessment tool. Geneva: World Health Organization; 2008. Within a health information system, accurate and timely data on the cause of death are fundamental for programme and policy development⁷7 Shibuya K, Scheele S, Boerma T. Health statistics: time to get serious. Bull World Health Organ. 2005;83(10):722. PMID: 16283042 and for measuring change in the magnitude and distribution of ill-health and disease in populations.⁸8 Ruzicka LT, Lopez AD. The use of cause-of-death statistics for health situation assessment: national and international experiences. World Health Stat Q. 1990;43(4):249–58. PMID: 2293493 Assessments of vital registration systems in low- and middle-income countries consistently reveal substantial weaknesses in the generation of cause-of-death statistics.⁹9 Strengthening civil registration and vital statistics for births, deaths and causes of death: resource kit. Geneva: World Health Organization; 2013. Available from http://www.uq.edu.au/hishub/docs/Resource%20Kit/CRVS_ResourceKIt_active_content.pdf [cited 2013 Aug 29].
http://www.uq.edu.au/hishub/docs/Resourc... ^–1111 Regional strategy for the improvement of civil registration and vital statistics systems. Cairo: Eastern Mediterranean Regional Office of the World Health Organization; 2013. Available from: http://applications.emro.who.int/docs/RC_Techn_paper_2013_10_15084_EN.pdf?ua=1 [cited 2014 Jul 18].
http://applications.emro.who.int/docs/RC... In a recent editorial, we drew attention to the fact that even hospital statistics on cause of death cannot be assumed to be correct – a fact that is not widely appreciated by governments and other users of these data.¹²12 Rampatige R, Mikkelsen L, Hernandez B, Riley I, Lopez AD. Hospital cause-of-death statistics: what should we make of them? Bull World Health Organ. 2014;92(1):3–3A. doi: http://dx.doi.org/10.2471/BLT.13.134106 PMID: 24391292
https://doi.org/10.2471/BLT.13.134106...

The gold standard for cause-of-death reporting is to have the cause certified by a medical practitioner using the rules and procedures of the International classification of diseases and related health problems (ICD), which is currently available in its tenth revision (ICD-10).¹³13 International statistical classification of diseases and related health problems. 10th revision. Geneva: World Health Organization; 2007. Available from: http://apps.who.int/classifications/apps/icd/icd10online [cited 2011 Jul 18].
http://apps.who.int/classifications/apps... Although most countries with statistical systems for cause of death now use the ICD classification for coding, not all countries have introduced the international standard certificate for reporting cause of death. Furthermore, physicians often do not receive adequate training in standard ICD death certification practices. It is, therefore, not surprising that comparative assessments commonly find that the quality of medical certification of the cause of death is poor.³3 Rao C, Lopez AD, Yang G, Begg S, Ma J. Evaluating national cause-of-death statistics: principles and application to the case of China. Bull World Health Organ. 2005;83(8):618–25. PMID: 16184281^,88 Ruzicka LT, Lopez AD. The use of cause-of-death statistics for health situation assessment: national and international experiences. World Health Stat Q. 1990;43(4):249–58. PMID: 2293493 Cause-of-death statistics of poor quality have limited policy utility and may even seriously mislead policy debates.¹⁴14 Burger EH, Groenewald P, Bradshaw D, Ward AM, Yudkin PL, Volmink J. Validation study of cause of death statistics in Cape Town, South Africa, found poor agreement. J Clin Epidemiol. 2012;65(3):309–16. doi: http://dx.doi.org/10.1016/j.jclinepi.2011.08.007 PMID: 22169084
https://doi.org/10.1016/j.jclinepi.2011....

In most developing countries, more than half of all deaths occur outside hospitals. Since out-of-hospital deaths are rarely medically certified, most of the physician-certified deaths come from hospitals.³3 Rao C, Lopez AD, Yang G, Begg S, Ma J. Evaluating national cause-of-death statistics: principles and application to the case of China. Bull World Health Organ. 2005;83(8):618–25. PMID: 16184281^,44 Polprasert W, Rao C, Adair T, Pattaraarchachai J, Porapakkham Y, Lopez AD. Cause-of-death ascertainment for deaths that occur outside hospitals in Thailand: application of verbal autopsy methods. Popul Health Metr. 2010;8(1):13. . doi: http://dx.doi.org/10.1186/1478-7954-8-13 PMID: 20482760
https://doi.org/10.1186/1478-7954-8-13... Can we automatically assume that the cause assigned to a death in hospital is accurate? Unfortunately, even in countries where hospital data are the only source of cause-of-death information, data quality is rarely evaluated. Research in different countries has repeatedly identified substantial misclassification of the cause of death of people who die in hospitals – with attendant implications for the use of cause-of-death data in informing policy.¹⁴14 Burger EH, Groenewald P, Bradshaw D, Ward AM, Yudkin PL, Volmink J. Validation study of cause of death statistics in Cape Town, South Africa, found poor agreement. J Clin Epidemiol. 2012;65(3):309–16. doi: http://dx.doi.org/10.1016/j.jclinepi.2011.08.007 PMID: 22169084
https://doi.org/10.1016/j.jclinepi.2011.... ^–2020 Carter K, Hufanga S, Rao C, Akauola S, Lopez AD, Rampatige R, et al. Causes of death in Tonga: quality of certification and implications for statistics. Popul Health Metr. 2012;10(1):4. doi: http://dx.doi.org/10.1186/1478-7954-10-4 PMID: 22390221
https://doi.org/10.1186/1478-7954-10-4...

To carry out a validation study of cause-of-death data collected in hospitals, we need a gold standard against which the hospital cause-of-death reports can be compared. While autopsy findings provide the ideal gold standard for cause-of-death evaluations, this approach is prohibitively expensive, rarely applied and likely to be based on a biased sample of deaths assigned to coroners. It would not be practical to carry out autopsies for all of the deaths occurring in a country – or even for all of the hospital deaths in a country.¹⁹19 Shojania KG, Burton EC, McDonald KM, Goldman L. Changes in rates of autopsy-detected diagnostic errors over time: a systematic review. JAMA. 2003;289(21):2849–56. doi: http://dx.doi.org/10.1001/jama.289.21.2849 PMID: 12783916
https://doi.org/10.1001/jama.289.21.2849... Instead, researchers have reviewed the medical records of people who have died in hospitals as a reference standard for validating the accuracy of the causes of deaths recorded by the hospitals. Although all hospitals have medical records for their patients, such records are rarely used for carrying out routine assessments of the extent and nature of any diagnostic misclassifications among hospital deaths. In part, this reflects a lack of awareness of the existence of such misclassification or – because there is no standard method and framework for carrying out routine evaluations of the quality of cause-of-death data – a lack of knowledge of how such misclassification might be identified. Here, we perform a systematic review of studies that used medical records to assess the quality of hospital cause-of-death data to ascertain the pattern and extent of diagnostic misclassification of the cause of death. We propose a standard method for the future use of medical record reviews for assessing the accuracy of hospital-based cause-of-death data.

Systematic review

Search strategy

We searched for published articles on studies that used medical record reviews to validate cause-of-death data for hospital deaths in English, Spanish or Portuguese, between 1983 and 2013. Search terms and databases are shown in (Fig. 1). All 199 studies identified from the initial investigation were screened for specific content and only the 29 studies that met our inclusion criteria were subjected to further scrutiny (Fig. 1, Appendix A available from: http://www.uq.edu.au/hishub/docs/Appendix%20A_final.pdf). Essentially, to be included in our systematic review, a study had to be primary research, published in a peer-reviewed journal and involve the validation of cause-of-death data originating from one or more hospitals against reference cause-of-death data obtained by review of the corresponding medical records.

Results

The 29 selected studies were categorized according to timing, geographical location, scope and basic study features, age at death and range of causes of death included (Table 1). Wherever possible, we also assessed methodological issues such as the characteristics of the reviewers, the quality of the medical records used and the diagnostic facilities available in the study hospitals. Since the methods, scopes and objectives of the selected studies were diverse, no review protocol was systematically applied and no comparative analysis of our findings was possible.

Only nine of the 29 selected studies were published after 2005. The number of deaths included in each study ranged from 23 in a study in Mexico¹⁵15 González-Medina A, Martínez-Natera O. Concordancia entre el diagnóstico de defunción y el diagnóstico anatomopatológico por necropsias en el Hospital Central Militar. Rev Sanid Milit. 2001;55(3):95–101. Spanish. to 3316 in a nationally representative study of cause-of-death accuracy in Thai hospitals.¹⁷17 Pattaraarchachai J, Rao C, Polprasert W, Porapakkham Y, Pao-In W, Singwerathum N, et al. Cause-specific mortality patterns among hospital deaths in Thailand: validating routine death certification. Popul Health Metr. 2010;8(1):12. doi: http://dx.doi.org/10.1186/1478-7954-8-12 PMID: 20482759
https://doi.org/10.1186/1478-7954-8-12... More than half of the selected studies (n = 18) were carried out in the Americas; Mexico contributed three studies and Brazil six. There was only one study from the Pacific – in Tonga²⁰20 Carter K, Hufanga S, Rao C, Akauola S, Lopez AD, Rampatige R, et al. Causes of death in Tonga: quality of certification and implications for statistics. Popul Health Metr. 2012;10(1):4. doi: http://dx.doi.org/10.1186/1478-7954-10-4 PMID: 22390221
https://doi.org/10.1186/1478-7954-10-4... – and one from Africa – in South Africa.¹⁴14 Burger EH, Groenewald P, Bradshaw D, Ward AM, Yudkin PL, Volmink J. Validation study of cause of death statistics in Cape Town, South Africa, found poor agreement. J Clin Epidemiol. 2012;65(3):309–16. doi: http://dx.doi.org/10.1016/j.jclinepi.2011.08.007 PMID: 22169084
https://doi.org/10.1016/j.jclinepi.2011.... Eighteen studies considered all causes of death while the remainder (n = 11) investigated selected causes only. The large majority (n = 24) covered all age groups. However, two studies included only adult deaths, one study focused on deaths in the elderly and another two studies focused on infant deaths (Table 1).

The main purpose of most of the medical record reviews was to identify the degree of misclassification of cause of death at the individual level – by comparing the hospital or vital registration diagnosis with a reference diagnosis based on a review of the dead patient’s medical records. A misclassification matrix is the primary outcome of several studies.³3 Rao C, Lopez AD, Yang G, Begg S, Ma J. Evaluating national cause-of-death statistics: principles and application to the case of China. Bull World Health Organ. 2005;83(8):618–25. PMID: 16184281^,1616 Hernández B, Ramírez-Villalobos D, Romero M, Gómez S, Atkinson C, Lozano R. Assessing quality of medical death certification: Concordance between gold standard diagnosis and underlying cause of death in selected Mexican hospitals. Popul Health Metr. 2011;9(1):38. doi: http://dx.doi.org/10.1186/1478-7954-9-38 PMID: 21816103
https://doi.org/10.1186/1478-7954-9-38... ^–1818 Rao C, Yang G, Hu J, Ma J, Xia W, Lopez AD. Validation of cause-of-death statistics in urban China. Int J Epidemiol. 2007;36(3):642–51. doi: http://dx.doi.org/10.1093/ije/dym003 PMID: 17329316
https://doi.org/10.1093/ije/dym003... ^,2020 Carter K, Hufanga S, Rao C, Akauola S, Lopez AD, Rampatige R, et al. Causes of death in Tonga: quality of certification and implications for statistics. Popul Health Metr. 2012;10(1):4. doi: http://dx.doi.org/10.1186/1478-7954-10-4 PMID: 22390221
https://doi.org/10.1186/1478-7954-10-4... ^,2222 Rampatige R, Gamage S, Peiris S, Lopez AD. Assessing the reliability of causes of death reported by the Vital Registration System in Sri Lanka: medical records review in Colombo. HIM J. 2013;42(3):20–8. PMID: 24067238^,2727 Khosravi A, Rao C, Naghavi M, Taylor R, Jafari N, Lopez AD. Impact of misclassification on measures of cardiovascular disease mortality in the Islamic Republic of Iran: a cross-sectional study. Bull World Health Organ. 2008 Sep;86(9):688–96. doi: http://dx.doi.org/10.2471/BLT.07.046532 PMID: 18797644
https://doi.org/10.2471/BLT.07.046532... An example of a misclassification matrix, based on empirical research carried out in Colombo, Sri Lanka,²²22 Rampatige R, Gamage S, Peiris S, Lopez AD. Assessing the reliability of causes of death reported by the Vital Registration System in Sri Lanka: medical records review in Colombo. HIM J. 2013;42(3):20–8. PMID: 24067238 is shown in Table 2. Selected findings from the misclassification matrices obtained in different countries are presented in Box 1. The misclassification matrix produced in a study that is nationally representative can be used to derive a series of correction factors that can be applied to routine cause-of-death data – from vital registration systems – to estimate the probable true cause-of-death pattern in the study country. In Thailand, for example, cause-specific mortality fractions that had been corrected in this manner were applied to the numbers of registered deaths in 2005 – which had been adjusted for underreporting – to estimate the probable true pattern of causes of death in the country in that year.⁴⁴44 Porapakkham Y, Rao C, Pattaraarchachai J, Polprasert W, Vos T, Adair T, et al. Estimated causes of death in Thailand, 2005: implications for health policy. Popul Health Metr. 2010;8(1):14. doi: http://dx.doi.org/10.1186/1478-7954-8-14 PMID: 20482761
https://doi.org/10.1186/1478-7954-8-14... For some causes of death, such as human immunodeficiency virus infection/acquired immunodeficiency syndrome and ischaemic heart disease, the corrected numbers of deaths in this study were 3- to 4-fold higher than the numbers recorded in the vital registration system – with huge implications for Thailand’s health policies. Fifteen of our selected studies¹⁶16 Hernández B, Ramírez-Villalobos D, Romero M, Gómez S, Atkinson C, Lozano R. Assessing quality of medical death certification: Concordance between gold standard diagnosis and underlying cause of death in selected Mexican hospitals. Popul Health Metr. 2011;9(1):38. doi: http://dx.doi.org/10.1186/1478-7954-9-38 PMID: 21816103
https://doi.org/10.1186/1478-7954-9-38... ^–1818 Rao C, Yang G, Hu J, Ma J, Xia W, Lopez AD. Validation of cause-of-death statistics in urban China. Int J Epidemiol. 2007;36(3):642–51. doi: http://dx.doi.org/10.1093/ije/dym003 PMID: 17329316
https://doi.org/10.1093/ije/dym003... ^,2020 Carter K, Hufanga S, Rao C, Akauola S, Lopez AD, Rampatige R, et al. Causes of death in Tonga: quality of certification and implications for statistics. Popul Health Metr. 2012;10(1):4. doi: http://dx.doi.org/10.1186/1478-7954-10-4 PMID: 22390221
https://doi.org/10.1186/1478-7954-10-4... ^–2323 Gotsens M, Marí-Dell’Olmo M, Rodríguez-Sanz M, Martos D, Espelt A, Pérez G, et al. [Validation of the underlying cause of death in medico-legal deaths]. Rev Esp Salud Publica. 2011;85(2):163–74. Spanish.doi: http://dx.doi.org/10.1590/S1135-57272011000200005 PMID: 21826379
https://doi.org/10.1590/S1135-5727201100... ^,2727 Khosravi A, Rao C, Naghavi M, Taylor R, Jafari N, Lopez AD. Impact of misclassification on measures of cardiovascular disease mortality in the Islamic Republic of Iran: a cross-sectional study. Bull World Health Organ. 2008 Sep;86(9):688–96. doi: http://dx.doi.org/10.2471/BLT.07.046532 PMID: 18797644
https://doi.org/10.2471/BLT.07.046532... ^,2828 Alpérovitch A, Bertrand M, Jougla E, Vidal J-S, Ducimetière P, Helmer C, et al. Do we really know the cause of death of the very old? Comparison between official mortality statistics and cohort study classification. Eur J Epidemiol. 2009;24(11):669–75. doi: http://dx.doi.org/10.1007/s10654-009-9383-2 PMID: 19728117
https://doi.org/10.1007/s10654-009-9383-... ^,3131 Nunes J, Koifman RJ, Mattos IE, Monteiro GT. [Reliability and validity of uterine cancer death certificates in the municipality of Belém, Pará, Brazil]. Cad Saude Publica. 2004;20(5):1262–8. Portuguese.doi: http://dx.doi.org/10.1590/S0102-311X2004000500020 PMID: 15486669
https://doi.org/10.1590/S0102-311X200400... ^–3333 Carvalho ML, Silver LD. [Reliability of the certification of the basic causes of neonatal deaths: implications for the study of preventable mortality]. Rev Saude Publica. 1995;29(5):342–8. Portuguese.doi: http://dx.doi.org/10.1590/S0034-89101995000500002 PMID: 8731273
https://doi.org/10.1590/S0034-8910199500... ^,3838 Laclé-Murray A. Causas de muerte en una cohorte de diabéticos tipo 2 de Costa Rica y la calidad de su certificado de defunción. Acta Med Costarric. 2012;54(1):23–30. Spanish.^,3939 Cordeiro R, Peñaloza ERO, Cardoso CF, Cortez DB, Kakinami E, Souza JJ, et al. [Validity of information on occupation and principal cause of death certificates in Botucatu, Sao Paulo]. Cad Saude Publica. 1999;15(4):719–28. Portuguese.doi: http://dx.doi.org/10.1590/S0102-311X1999000400006 PMID: 10633194
https://doi.org/10.1590/S0102-311X199900... ^,4444 Porapakkham Y, Rao C, Pattaraarchachai J, Polprasert W, Vos T, Adair T, et al. Estimated causes of death in Thailand, 2005: implications for health policy. Popul Health Metr. 2010;8(1):14. doi: http://dx.doi.org/10.1186/1478-7954-8-14 PMID: 20482761
https://doi.org/10.1186/1478-7954-8-14... provided information to correct cause-specific mortality fractions based on vital registration data – where these fractions were known or suspected to be incorrect.

The utility of any medical record review depends on the ability of the reviewers to identify the underlying cause of death correctly from the medical records.⁴⁵45 Maudsley G, Williams L. Death certification–a sad state of affairs. J Public Health Med. 1994;16(3):370–1. PMID: 7999397 The reviewers in most of our selected studies (n = 18) were physicians who had – reportedly – been trained in death certification practices (n = 15). One study simply reported that professionals had been used as reviewers, while nine studies provided no information about reviewer qualifications. A Brazilian study³⁴34 Niobey FM, Cascão AM, Duchiade MP, Sabroza PC. [The quality of the filling-in of death certificates of children below one year of age in the metropolitan area of Rio de Janeiro]. Rev Saude Publica. 1990;24(4):311–8. Portuguese.doi: http://dx.doi.org/10.1590/S0034-89101990000400009 PMID: 2103649
https://doi.org/10.1590/S0034-8910199000... that used both a physician and a researcher was classified as physician-review in this analysis. The number of reviewers per study depended on sample size and ranged from one in each of four studies to 84 in the Thai study.¹⁷17 Pattaraarchachai J, Rao C, Polprasert W, Porapakkham Y, Pao-In W, Singwerathum N, et al. Cause-specific mortality patterns among hospital deaths in Thailand: validating routine death certification. Popul Health Metr. 2010;8(1):12. doi: http://dx.doi.org/10.1186/1478-7954-8-12 PMID: 20482759
https://doi.org/10.1186/1478-7954-8-12...

The accuracy of the reference cause-of-death diagnosis depends on the quality of information available in the medical records. Ten of the reviewed studies included such an assessment – in which the records were categorized either simply, as adequate or inadequate, or, with more qualification, as excellent, good, average, weak or poor. One study excluded deaths when the corresponding medical records were judged to be incomplete.²³23 Gotsens M, Marí-Dell’Olmo M, Rodríguez-Sanz M, Martos D, Espelt A, Pérez G, et al. [Validation of the underlying cause of death in medico-legal deaths]. Rev Esp Salud Publica. 2011;85(2):163–74. Spanish.doi: http://dx.doi.org/10.1590/S1135-57272011000200005 PMID: 21826379
https://doi.org/10.1590/S1135-5727201100... The remainder provided no information about the quality of the medical records used.

In nine of our selected studies, standard diagnostic criteria were pre-determined – i.e. the clinical evidence required to classify a death as being due to a particular cause was specified in advance of the review.¹⁴14 Burger EH, Groenewald P, Bradshaw D, Ward AM, Yudkin PL, Volmink J. Validation study of cause of death statistics in Cape Town, South Africa, found poor agreement. J Clin Epidemiol. 2012;65(3):309–16. doi: http://dx.doi.org/10.1016/j.jclinepi.2011.08.007 PMID: 22169084
https://doi.org/10.1016/j.jclinepi.2011.... ^,1616 Hernández B, Ramírez-Villalobos D, Romero M, Gómez S, Atkinson C, Lozano R. Assessing quality of medical death certification: Concordance between gold standard diagnosis and underlying cause of death in selected Mexican hospitals. Popul Health Metr. 2011;9(1):38. doi: http://dx.doi.org/10.1186/1478-7954-9-38 PMID: 21816103
https://doi.org/10.1186/1478-7954-9-38... ^,2020 Carter K, Hufanga S, Rao C, Akauola S, Lopez AD, Rampatige R, et al. Causes of death in Tonga: quality of certification and implications for statistics. Popul Health Metr. 2012;10(1):4. doi: http://dx.doi.org/10.1186/1478-7954-10-4 PMID: 22390221
https://doi.org/10.1186/1478-7954-10-4... ^,2323 Gotsens M, Marí-Dell’Olmo M, Rodríguez-Sanz M, Martos D, Espelt A, Pérez G, et al. [Validation of the underlying cause of death in medico-legal deaths]. Rev Esp Salud Publica. 2011;85(2):163–74. Spanish.doi: http://dx.doi.org/10.1590/S1135-57272011000200005 PMID: 21826379
https://doi.org/10.1590/S1135-5727201100... ^,2525 Durruthy Wilson O, Sifontes Estrada M, Martínez Varona C, Olazábal Hernández A. [From the death certificate to the protocol of necropsy: basic causes of death]. Rev Arch Méd. 2011;15(3):542-52. Spanish.doi: http://dx.doi.org/10.1016/0895-4356(90)90094-6 PMID: 2254765
https://doi.org/10.1016/0895-4356(90)900... ^,3838 Laclé-Murray A. Causas de muerte en una cohorte de diabéticos tipo 2 de Costa Rica y la calidad de su certificado de defunción. Acta Med Costarric. 2012;54(1):23–30. Spanish.^,4040 Monteiro GT, Koifman RJ, Koifman S. [Reliability and accuracy of reported causes of death from cancer. II. Accuracy of stomach cancer reported in the municipality of Rio de Janeiro County, Brazil]. Cad Saude Publica. 1997;13 Suppl 1:53–65. Portuguese.doi: http://dx.doi.org/10.1590/S0102-311X1997000500006 PMID: 10886924
https://doi.org/10.1590/S0102-311X199700... ^,4141 Schnitman A. [Analysis of the reliability of the declaration of cancer as the basic cause of death in Salvador, Brazil]. Rev Saude Publica. 1990;24(6):490–6. Portuguese.doi: http://dx.doi.org/10.1590/S0034-89101990000600007 PMID: 2103070
https://doi.org/10.1590/S0034-8910199000... ^,4343 Moussa MAA, Shafie MZ, Khogali MM, el-Sayed AM, Sugathan TN, Cherian G, et al. Reliability of death certificate diagnoses. J Clin Epidemiol. 1990;43(12):1285–95. doi: http://dx.doi.org/10.1016/0895-4356(90)90094-6 PMID: 2254765
https://doi.org/10.1016/0895-4356(90)900... In case of diagnostic uncertainty, nine studies used a physician panel to discuss diagnostic problems. Three studies ordered verbal autopsies when the cause of death was in doubt. One study referred diagnostic problems to a second physician, and two others referred to external experts. Fourteen studies provided no information about the resolution of diagnostic uncertainty.

The diagnostic capacity of hospitals will clearly influence the accuracy of their cause-of-death diagnoses, both in vital registration systems and in medical record reviews. Only the Chinese study¹⁸18 Rao C, Yang G, Hu J, Ma J, Xia W, Lopez AD. Validation of cause-of-death statistics in urban China. Int J Epidemiol. 2007;36(3):642–51. doi: http://dx.doi.org/10.1093/ije/dym003 PMID: 17329316
https://doi.org/10.1093/ije/dym003... specifically mentioned that urban hospitals were selected to ensure that the study hospitals had adequate diagnostic facilities. None of the other studies included in our analysis referred to hospital diagnostic capacity.

The correct assignment of the underlying cause of death on the death certificate requires not only that the sequence and relationship of pathological conditions on the certificate are correctly certified but also that the underlying cause of death is correctly coded in terms of the ICD. Despite the critical role of coding, only three studies assessed coding accuracy.¹⁷17 Pattaraarchachai J, Rao C, Polprasert W, Porapakkham Y, Pao-In W, Singwerathum N, et al. Cause-specific mortality patterns among hospital deaths in Thailand: validating routine death certification. Popul Health Metr. 2010;8(1):12. doi: http://dx.doi.org/10.1186/1478-7954-8-12 PMID: 20482759
https://doi.org/10.1186/1478-7954-8-12... ^,2626 Fajardo S, Aerts DR, Bassanesi SL. [Accuracy of the Mortality Information System team in the specification of underlying cause of death in a State capital in southern Brazil]. Cad Saude Publica. 2009;25(10):2218–28. Portuguese.doi: http://dx.doi.org/10.1590/S0102-311X2009001000012 PMID: 19851621
https://doi.org/10.1590/S0102-311X200900... ^,3737 Granero R, Ortiz R, Gómez-Marín O, Isaacura C, Goyo A, Rodríguez J. [Validation of “myocardial infarction” as cause of death on death certificates in Barquisimeto, Lara state, Venezuela]. Cad Saude Publica. 1997;13(3):383–7. Spanish.doi: http://dx.doi.org/10.1590/S0102-311X1997000300013 PMID: 10886877
https://doi.org/10.1590/S0102-311X199700...

All studies reported some degree of diagnostic misclassification. Due to the use of different classification lists, the misclassification reported in these studies are not directly comparable. For the studies that have used ICD-10’s three-character codes for comparison of cause of death, the range of misclassification varied from 25²⁴24 Rodriguez Almada H, Ciriacos C, Piñeyrúa M, Logaldo R, González González D. Calidad del registro en el certificado de defunción en un hospital publico de referencia. Montevideo, Uruguay. Rev Med Urug (Montev). 2010;26:216–23. Spanish. to 62%.¹⁶16 Hernández B, Ramírez-Villalobos D, Romero M, Gómez S, Atkinson C, Lozano R. Assessing quality of medical death certification: Concordance between gold standard diagnosis and underlying cause of death in selected Mexican hospitals. Popul Health Metr. 2011;9(1):38. doi: http://dx.doi.org/10.1186/1478-7954-9-38 PMID: 21816103
https://doi.org/10.1186/1478-7954-9-38... The concordance at the level of ICD-10’s three-character codes was 41.4% in Sri Lanka²²22 Rampatige R, Gamage S, Peiris S, Lopez AD. Assessing the reliability of causes of death reported by the Vital Registration System in Sri Lanka: medical records review in Colombo. HIM J. 2013;42(3):20–8. PMID: 24067238 while concordance at the level of the Basic Tabulation List in the ninth revision of the ICD was found to be 77% in Sweden.³⁵35 Johansson LA, Björkenstam C, Westerling R. Unexplained differences between hospital and mortality data indicated mistakes in death certification: an investigation of 1,094 deaths in Sweden during 1995. J Clin Epidemiol. 2009;62(11):1202–9. doi: http://dx.doi.org/10.1016/j.jclinepi.2009.01.010 PMID: 19364635
https://doi.org/10.1016/j.jclinepi.2009.... In South Africa, concordance at the level of ICD-10’s mortality tabulation level was 55.3%.¹⁴14 Burger EH, Groenewald P, Bradshaw D, Ward AM, Yudkin PL, Volmink J. Validation study of cause of death statistics in Cape Town, South Africa, found poor agreement. J Clin Epidemiol. 2012;65(3):309–16. doi: http://dx.doi.org/10.1016/j.jclinepi.2011.08.007 PMID: 22169084
https://doi.org/10.1016/j.jclinepi.2011.... The pattern and extent of misclassification found in the studies were generally reported either as matrices or as percentages. Other metrics that were used included simple concordance-based measures of sensitivity and specificity, and measures that attempt to correct for the probability of getting a concordant diagnosis merely by chance, such as Kappa statistics and chance-corrected concordance.⁴⁶46 Murray CJL, Lozano R, Flaxman AD, Vahdatpour A, Lopez AD. Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies. Popul Health Metr. 2011;9(1):28. doi: http://dx.doi.org/10.1186/1478-7954-9-28 PMID: 21816106
https://doi.org/10.1186/1478-7954-9-28... More usefully, the analysis of misclassification matrices allows specific insight into the patterns of cause-of-death misallocation and is thus a critical input in activities designed to strengthen health information systems as well as in the correct interpretation of national mortality statistics. Common causes of death that were frequently misclassified included ischaemic heart disease, stroke, diabetes mellitus and external causes of injury. In a study based on a medical record review in the Islamic Republic of Iran, ill-defined causes of death were mainly reclassified to ischaemic heart disease (33.5%) and cerebrovascular accidents (17.1%)²⁷27 Khosravi A, Rao C, Naghavi M, Taylor R, Jafari N, Lopez AD. Impact of misclassification on measures of cardiovascular disease mortality in the Islamic Republic of Iran: a cross-sectional study. Bull World Health Organ. 2008 Sep;86(9):688–96. doi: http://dx.doi.org/10.2471/BLT.07.046532 PMID: 18797644
https://doi.org/10.2471/BLT.07.046532... while a study in Kuwait reported that original death certificates underestimated cerebrovascular disease by 69%, diabetes mellitus by 60% and ischaemic heart disease by 33%.⁴³43 Moussa MAA, Shafie MZ, Khogali MM, el-Sayed AM, Sugathan TN, Cherian G, et al. Reliability of death certificate diagnoses. J Clin Epidemiol. 1990;43(12):1285–95. doi: http://dx.doi.org/10.1016/0895-4356(90)90094-6 PMID: 2254765
https://doi.org/10.1016/0895-4356(90)900... Surprisingly, in our selected studies, ill-defined causes of death were frequently reported even in hospital settings. The policy impact of allocating these non-specific codes to more definitive causes are reported in two studies;²¹21 de Carvalho MH, Álvarez-Hernández G, Denman C, Harlow SD. Validity of underlying cause of death statistics in Hermosillo, Mexico. Salud Publica Mex. 2011;53(4):312–9. doi: http://dx.doi.org/10.1590/S0036-36342011000400005 PMID: 21986787
https://doi.org/10.1590/S0036-3634201100... ^,4444 Porapakkham Y, Rao C, Pattaraarchachai J, Polprasert W, Vos T, Adair T, et al. Estimated causes of death in Thailand, 2005: implications for health policy. Popul Health Metr. 2010;8(1):14. doi: http://dx.doi.org/10.1186/1478-7954-8-14 PMID: 20482761
https://doi.org/10.1186/1478-7954-8-14... key findings from a meta-analysis of misclassification patterns can be found in the article by Rampatige et al.⁴⁷47 Rampatige R, Mikkelsen L, Hernandez B, Riley I, Lopez AD. Assessing the reliability of hospital-based cause-of-death statistics: evidence-based guidelines for country application [Working paper 37]. Brisbane: University of Queensland; 2013. Available from: http://www.uq.edu.au/hishub/wp37 [cited 2013 August 30].
http://www.uq.edu.au/hishub/wp37...

Framework for reviewing medical records

Given the importance and cost of accurate cause-of-death statistics for health policy and priority setting – and the effort and cost of undertaking medical record reviews – evaluation studies using this method could benefit enormously from some simple, evidence-based lessons. Based on this systematic review and experience with our own field research, we provide a synthesis of some of the key issues that should guide national applications of medical record reviews.¹⁶16 Hernández B, Ramírez-Villalobos D, Romero M, Gómez S, Atkinson C, Lozano R. Assessing quality of medical death certification: Concordance between gold standard diagnosis and underlying cause of death in selected Mexican hospitals. Popul Health Metr. 2011;9(1):38. doi: http://dx.doi.org/10.1186/1478-7954-9-38 PMID: 21816103
https://doi.org/10.1186/1478-7954-9-38... ^–1818 Rao C, Yang G, Hu J, Ma J, Xia W, Lopez AD. Validation of cause-of-death statistics in urban China. Int J Epidemiol. 2007;36(3):642–51. doi: http://dx.doi.org/10.1093/ije/dym003 PMID: 17329316
https://doi.org/10.1093/ije/dym003... ^,2020 Carter K, Hufanga S, Rao C, Akauola S, Lopez AD, Rampatige R, et al. Causes of death in Tonga: quality of certification and implications for statistics. Popul Health Metr. 2012;10(1):4. doi: http://dx.doi.org/10.1186/1478-7954-10-4 PMID: 22390221
https://doi.org/10.1186/1478-7954-10-4... ^,2222 Rampatige R, Gamage S, Peiris S, Lopez AD. Assessing the reliability of causes of death reported by the Vital Registration System in Sri Lanka: medical records review in Colombo. HIM J. 2013;42(3):20–8. PMID: 24067238^,2727 Khosravi A, Rao C, Naghavi M, Taylor R, Jafari N, Lopez AD. Impact of misclassification on measures of cardiovascular disease mortality in the Islamic Republic of Iran: a cross-sectional study. Bull World Health Organ. 2008 Sep;86(9):688–96. doi: http://dx.doi.org/10.2471/BLT.07.046532 PMID: 18797644
https://doi.org/10.2471/BLT.07.046532...

Our systematic review confirms that a variety of approaches have been used to assess the accuracy of cause-of-death data. Rather than trying to evaluate whether there is one approach that is better than the others, we have tried to draw lessons from each study about the specific steps that added value to the study and could be repeated elsewhere. Building on these findings, we propose a framework that addresses the critical choices that need to be carefully considered when using medical record reviews for assessing cause-of-death data quality (Box 2). Further detail on each of the topics included in our framework has been published elsewhere – with specific guidance on the selection of hospitals and cases, the development and use of diagnostic criteria, interpreting misclassification matrices, evaluating the quality of the source documentation, and the choice and training of the reviewers used.⁴⁷47 Rampatige R, Mikkelsen L, Hernandez B, Riley I, Lopez AD. Assessing the reliability of hospital-based cause-of-death statistics: evidence-based guidelines for country application [Working paper 37]. Brisbane: University of Queensland; 2013. Available from: http://www.uq.edu.au/hishub/wp37 [cited 2013 August 30].
http://www.uq.edu.au/hishub/wp37...

Discussion

Medical record reviews can and have been carried out to serve many different purposes. Perhaps the most common application is the independent assessment of the reliability of hospital cause-of-death data – particularly when expert review or other use of mortality data has revealed these to be deficient. Another common purpose is to assess whether deaths from specific causes – e.g. various cancers or traffic accidents – are being reliably recorded in hospital settings. If such an assessment involves a reasonably representative national sample of hospital deaths, correction factors can be calculated from the misclassification matrices and applied to national cause-of-death data to estimate the likely true cause-specific mortality pattern in the population.

Despite an extensive literature review, we were only able to identify a small number of peer-reviewed studies that had used medical records to investigate the quality of cause-of-death data (n = 29). Given the critical role of cause-of-death data in informing debates around national health policies, this is both surprising and disappointing. Our main observation – of extensive misclassification of causes of death in hospitals wherever studies have been carried out – indicates that inaccurate cause-of-death data are pervasive. However, it is important to consider potential biases that might have affected the limited number of studies that we were able to analyse. First, we were only able to review articles that had been published in a language in which at least one member of our review team was competent – i.e. English, Portuguese or Spanish. It is possible that articles published in other languages might have led to different conclusions about the pattern and extent of cause-of-death misclassification in hospitals. More comprehensive studies covering other major language groups would be welcome but there is no reason to suspect that these would not reinforce our main findings – which seemed fairly consistent irrespective of the language of the article involved. Thus, our conclusions and recommendations for policy action should still be valid. It is also possible that the literature we screened was affected by selective biases – such as publication bias – but we suspect that this would only operate to suppress the publication of more extreme findings about misclassification. Finally, the studies we selected had diverse aims and the set of causes of death investigated in each study was presumably biased towards the priorities of the interest groups involved. We found only one study that involved the comprehensive investigation of cause-of-death misclassification across all major causes of death. It is therefore quite likely that the extent of cause-specific misclassification across all countries is different to that indicated by the studies we selected. Again, however, that does not alter our fundamental conclusion that hospital statistics on causes of death – whatever the universe of causes investigated – are likely to be grossly inaccurate in many countries and ought to be the focus of an immediate policy response.

All of the studies we selected for review assessed the quality of the hospital-reported cause-of-death data by comparing them with cause-of-death data derived by reviewing medical records. However, the lack of a standard framework or method for conducting and reporting the findings of medical record reviews has resulted in substantial variation in the approaches used and difficulties in interpreting and comparing results. Building on these findings and our own empirical experience, we propose a series of steps to guide future studies on the accuracy of cause-of-death data that involve the review of medical records. By following these steps, it should be possible to validate the quality of cause-of-death data reported by hospitals and vital registration systems more reliably.

The synthesis of lessons learnt, summarized in Box 2, outlines a clear and tested method and identifies the most important steps in the process – e.g. the design of the study, development of diagnostic criteria, selection of the sample, review of medical records, and the analysis and interpretation of findings. However, the framework that we have proposed – like all frameworks – has limitations. In particular, it would not be feasible to apply our framework in settings where medical records are poorly kept or where the diagnostic capacity of hospitals is generally poor. An understanding – by hospital and health administrators – of the importance of studies based on our framework for policy-makers is crucial, as is a commitment, of those responsible for the development of national health information systems, to implement any resultant recommendations.

While the proposed framework needs to be tested in its entirety in further empirical studies, we consider it sufficiently robust to be applied in its current state in different settings. We recommend its periodic application in all countries, to identify deficiencies in national mortality statistics. Using the misclassification matrix that the study method yields, health authorities should be able to identify the diseases that are commonly misclassified as causes of death and also ascertain whether the causes of death that are being recorded in hospitals are of sufficient quality to be fit for purpose. This knowledge can and should be used to guide strategies for the strengthening of health information systems – e.g. increasing awareness among medical associations about the value of doctors correctly certifying deaths, improving the keeping of medical records in hospitals, and ensuring that cause-of-death coders are appropriately trained.

Systems for the determination and recording of the causes of hospital deaths represent a purposeful and costly investment that countries make. It is imperative that such systems perform to the standard required to support good public policy.

Funding:

Competing interests:

References

Publication Dates

History