Abstracts
Studies of adverse drug events (ADEs) are important in order not to jeopardize the positive impact of pharmacotherapy. These events have substantial impact on the population morbidity profiles, and increasing health system operating costs. Administrative databases are an important source of information for public health purposes and for identifying ADEs. In order to contribute to learning about ADE in hospitalized patients, this study examined the potential of applying ICD-10 (10th revision of the International Classification of Diseases) codes to a national database of the public health care system (SIH-SUS). The study comprised retrospective assessment of ADEs in the SIH-SUS administrative database, from 2008 to 2012. For this, a list of ICD-10 codes relating to ADEs was built. This list was built up by examining lists drawn up by other authors identified by bibliographic search in the MEDLINE and LILACS and consultations with experts. In Brazil, 55,604,537 hospital admissions were recorded in the SIH-SUS, between 2008 and 2012, of which 273,440 (0.49%) were related to at least one ADE. The proportions and rates seem to hold constant over the study period. Fourteen out of 20 most frequent ADEs were identified in codes relating to mental disorders. Intoxications figure as the second most frequently recorded group of ADEs in the SIH-SUS, comprising 76,866 hospitalizations. Monitoring of ADEs in administrative databases using ICD-10 codes is feasible, even in countries with information systems under construction, and can be an innovative tool to complement drug surveillance strategies in place in Brazil, as well as in others countries.
Keywords:
Drug-Related Side Effects and Adverse Reactions; Medication Errors; Hospital Information Systems; International Classification of Diseases
Os estudos sobre eventos adversos a medicamentos (EAMs) são importantes para evitar de prejudicar o efeito positivo da farmacoterapia. Os EAMs têm impacto substancial nos perfis de morbidade da população e no aumento dos custos operacionais do sistema de saúde. As bases de dados administrativos representam uma fonte de informação importante para fins de saúde pública em geral e especificamente para identificar os EAMs. No intuito de contribuir para o conhecimento sobre EAMs em pacientes hospitalizados, o estudo examinou a aplicabilidade dos códigos da CID-10 (10ª revisão da Classificação Internacional de Doenças) ao Sistema de Informações Hospitalares do Sistema Único de Saúde (SIH-SUS). O estudo integrou uma avaliação retrospectiva de dados administrativos do SIH-SUS referentes aos anos de 2008 a 2012. Para tanto, foi elaborada uma lista de códigos da CID-10 relacionados a EAMs. A lista foi produzida a partir de uma consulta às listas projetadas por outros autores e identificadas através de uma busca em MEDLINE e LILACS e consultas com especialistas. No Brasil, foram registradas 55.604.537 internações hospitalares no SIH-SUS entre 2008 e 2012, das quais 273.440 (0,49%) estiveram relacionadas a pelo menos um EAM. As proporções e taxas de EAMs permaneceram constantes ao longo do período estudado. Quatorze dos vinte EAMs mais frequentes foram identificados através de códigos relacionados a transtornos psiquiátricos. As intoxicações figuram como o segundo grupo mais frequente de EAMs registrados no SIH-SUS, com 76.866 internações. O monitoramento dos EAMs com o uso dos códigos da CID-10 mostrou ser uma metodologia viável, mesmo em países com sistemas de informação ainda incompletos, e pode ser uma ferramenta inovadora para complementar as estratégias atuais de vigilância farmacológica no Brasil, assim como, em outros países.
Palavras-chave:
Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos; Erros de Medicação; Sistemas de Informação Hospitalar; Classificação Internacional de Doenças
Los estudios sobre eventos adversos por medicamentos (EAMs) son importantes, para no comprometer el impacto positivo de la farmacoterapia. Estos eventos tienen un impacto sustancial en los perfiles de morbilidad de la población e incrementan los costes operativos del sistema de salud. Las bases de datos administrativas son una importante fuente de información por motivos de salud pública y para identificar EAMs. Con el fin de contribuir al aprendizaje sobre EAM en pacientes hospitalizados, este estudio examinó el potencial de aplicar códigos CIE-10 (10ª revisión de la Clasificación Internacional de Enfermedades) a una base datos nacional del sistema de salud público (SIH-SUS). Este estudio estuvo constituido por una evaluación retrospectiva de EAMs, en la base de datos administrativa del SIH-SUS, desde 2008 a 2012. Para ello, se desarrolló la lista de la CIE-10 con códigos para EAMs. Esta lista se creó examinando listas diseñadas por otros autores, identificados a través de búsquedas bibliográficas en MEDLINE, LILACS y consultas con expertos. En Brasil, se registraron 55.604.537 admisiones hospitalarias en el SIH-SUS, entre 2008 y 2012, de las cuales 273.440 (0,49%) sufrieron al menos un EAM. Los porcentajes y tasas parecieron mantenerse constantes durante el periodo de estudio. Catorce de los 20 más frecuentes EAMs fueron identificados con códigos relacionados con enfermedades mentales. Las intoxicaciones fueron el segundo grupo registrado más común de EAMs en el SIH-SUS, representando 76.866 hospitalizaciones. La supervisión de EAMs en las bases de datos administrativas, usando los códigos CIE-10, es factible, incluso en países cuyos sistemas de información se encuentran en proceso de construcción, y puede ser un herramienta innovadora como complemento de las actuales estrategias de supervisión sobre medicamentos en Brasil, así como en otros países.
Palabras-clave:
Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos; Errores de Medicación; Sistemas de Información en Hospital; Clasificación Internacional de Enfermedades
Introduction
Studies of adverse drug events (ADEs) are important in order not to jeopardize the positive impact of pharmacotherapy. Such events include, in addition to medication errors, those associated with risks intrinsic to the drugs even when used appropriately 11. World Health Organization. The conceptual framework for the international classification for patient safety. http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf (accessed on 04/Nov/2017).
http://www.who.int/patientsafety/taxonom... . Hospital studies have identified high incidence of ADEs, which can affect 1.6 to 28.3% of inpatients 22. Martins AC, Giordani F, Rozenfeld S. Adverse drug events among adult inpatients: a meta-analysis of observational studies. J Clin Pharm Ther 2014; 39:609-20. and up to 5.8% of admissions 33. Atiqi R, Cleophas TJ, Van Bommel E, Zwinderman AH. Meta-analysis of recent studies on patients admitted to hospital due to adverse drug effects. Int J Clin Pharmacol Ther 2009; 47:549-55.. These events have substantial impact on the population morbidity, not only causing deaths, but increasing costs 22. Martins AC, Giordani F, Rozenfeld S. Adverse drug events among adult inpatients: a meta-analysis of observational studies. J Clin Pharm Ther 2014; 39:609-20.,44. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163:1409-16.. This phenomenon is a conspicuous public health problem, to which responses have been made at the international and national levels 55. Ministério da Saúde. Portaria nº 529, de 1º de abril de 2013. Institui o Programa Nacional de Segurança do Paciente (PNSP). Diário Oficial da União 2013; 2 abr.,66. Organização Pan-Americana da Saúde. Aliança mundial para segurança do paciente. Boletim informativo da OMS para segurança do paciente. http://www.paho.org/bra/index.php?option=com_content&view=article&id=931&Itemid=0 (accessed on 04/Nov/2017)
http://www.paho.org/bra/index.php?option... .
The importance of studying ADEs stems from the need to learn their frequency, characteristics and magnitude, identify vulnerable groups and, particularly, take measures to prevent them. Considering only events associated with hospital admissions or those that lead to emergency service care, half the events are regarded as avoidable 77. Hakkarainen KM, Hedna K, Petzold M, Hägg S. Percentage of patients with preventable adverse drug reactions and preventability of adverse drug reactions: a meta-analysis. PLoS One 2012; 7:e33236..
Efforts to gauge the magnitude of the risks apply several methodological approaches, singly or in combination, and various different data sources. The scope and accuracy of each approach will affect the estimates of such events 22. Martins AC, Giordani F, Rozenfeld S. Adverse drug events among adult inpatients: a meta-analysis of observational studies. J Clin Pharm Ther 2014; 39:609-20.,88. Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates D. Adverse drug events and medication errors: detection and classification methods. Qual Saf Health Care 2004; 13:306-14..
In Brazil, data are available from the national health surveillance notification system 99. Agência Nacional de Vigilância Sanitária. Relatórios Notivisa: evolução das notificações 2006 a 2013. http://www.anvisa.gov.br/hotsite/notivisa/relatorios/index.htm (accessed on 12/Nov/2017)
http://www.anvisa.gov.br/hotsite/notivis... and also from ad hoc research into ADEs at hospitals with prospective 1010. Santos DB, Coelho HL. Adverse drug reactions in hospitalized children in Fortaleza, Brazil. Pharmacoepidemiol Drug Saf 2006; 15:635-40.,1111. Camargo AL, Cardoso Ferreira MB, Heineck I. Adverse drug reactions: a cohort study in internal medicine units at a university hospital. Eur J Clin Pharmacol 2006; 62:143-9.,1212. Passarelli MCG, Jacob-Filho W. Reações adversas a medicamentos em idosos: como prevê-las? Einstein (São Paulo) 2007; 5:246-51., retrospective 1313. Roque KE, Melo ECP. Adaptação dos critérios de avaliação de eventos adversos a medicamentos para uso em um hospital público no Estado do Rio de Janeiro. Rev Bras Epidemiol 2010; 13:607-19.,1414. Giordani F, Rozenfeld S, Martins M. Adverse drug events identified by triggers at a teaching hospital in Brazil. BMC Pharmacol Toxicol 2014; 15:71. and cross-sectional 1515. Varallo FR, Capucho HC, Planeta CDS, Mastrolannin PC. Possible adverse drug events leading to hospital admission in a Brazilian teaching hospital. Clinics 2014; 69:163-7. monitoring.
Administrative databases are an important source of information for public health purposes and for identifying ADEs. These have permitted low-cost studies with considerable geographical and time coverage, involving large numbers of patients. They enable outcome frequencies to be calculated with reference to populations of local regions, to extract national estimates and to compare countries, because they share the same coding of health disorders and conditions, the International Classification of Diseases, currently in its tenth version (ICD-10) 1616. Rozenfeld S. Agravos provocados por medicamentos em hospitais do Estado do Rio de Janeiro, Brasil. Rev Saúde Pública 2007; 41:108-15.,1717. Hougland P, Xu W, Pickard S, Masheter C, Williams SD. Performance of International Classification of Diseases, 9th revision, linical modification codes as an adverse drug event surveillance system. Med Care 2006; 44:629-36.,1818. Miguel A, Marques B, Freitas A, Lopes F, Azevedo L, Pereira AC. Detection of adverse drug reactions using hospital databases: a nationwide study in Portugal. Pharmacoepidemiol Drug Saf 2013; 22:907-13..
In Brazil, today, the scenario for study of adverse events has been favourable since the Ministry of Health introduced the national patient safety programme 55. Ministério da Saúde. Portaria nº 529, de 1º de abril de 2013. Institui o Programa Nacional de Segurança do Paciente (PNSP). Diário Oficial da União 2013; 2 abr.. In order to contribute to learning about ADE in hospitalized patients, this study examined the potential of applying ICD-10 codes to a national database of the public health care system (Hospital Information Systems of the Brazilian Unified National Health Syustem - SIH-SUS).
Methods
Study design and data sources
The study comprised retrospective assessment of ADEs in the SIH-SUS administrative database, from 2008 to 2012, which were identified by way of a listing of ICD-10 codes. This public-access database covers more than 11 million hospitalizations yearly in a universal system where treatment is free of charge. The data are submitted electronically by hospitals after patient discharge, consolidated at the national level, anonymized and posted on the Ministry of Health website (Brazilian Health Informatics Department. http://www2.datasus.gov.br/DATASUS/index.php, accessed on 04/Nov/2017).
The World Health Organization’s (WHO) ICD is a health care classification system used as a standard research tool. Otherwise studies of ADEs have shown differences in regard to: choice of system (code list and ADE-related definitions); mechanisms or processes necessary to build up a study code list; research objectives; criteria for delimiting the research boundaries. A standardized set of codes for all countries would be more useful for international comparisons. Meanwhile, codes were selected following objective criteria, so as to guarantee some comparability with others studies.
The database contains variables relating to the treating facility and patient age and sex, together with fields for recording the principal diagnosis, secondary diagnosis, associated diagnosis and cause of death, by ICD-10 codes. In addition, the database records the procedures performed and amounts paid 1919. Instituto Brasileiro de Geografia e Estatística. Sistema de Informações Hospitalares do SUS - SIH/SUS. http://ces.ibge.gov.br/base-de-dados/metadados/ministerio-da-saude/sistema-de-informacoes-hospitalares-do-sus-sih-sus.html (accessed on 04/Nov/2017)
http://ces.ibge.gov.br/base-de-dados/met... .
The mandatory fields, “principal diagnosis” and “ICD causes of death”, were completed in 100% of cases, but the secondary diagnosis and associated causes fields were completed in only 12.6% and 1.1% of instances, respectively.
ADE definition and identification by ICD-10 codes
For this study, ADE was considered to be “any incident in which the use of a medication (drug or biologic) at any dose, a medical device, or a special nutritional product (e.g., dietary supplement, infant formula, medical food) may have resulted in an adverse outcome in a patient” 11. World Health Organization. The conceptual framework for the international classification for patient safety. http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf (accessed on 04/Nov/2017).
http://www.who.int/patientsafety/taxonom... . That definition was used to take in events relating to medications and vaccines, as well as intoxications by medicines, although the study did not include medical devices or special nutritional products.
The listing of ADE-related ICD-10 codes was built up by examining lists drawn up by other authors identified by bibliographical search in the MEDLINE and LILACS electronic bases. Selection of the articles considered only those that included a complete, comprehensive listing of ADE-related ICD-10 codes. The information extracted from the articles included: definition of adverse event; procedures for identifying ICD-10 codes; inclusion and exclusion criteria; and code categories, by their sensitivity in indicating an event. As a result, four articles were selected 1616. Rozenfeld S. Agravos provocados por medicamentos em hospitais do Estado do Rio de Janeiro, Brasil. Rev Saúde Pública 2007; 41:108-15.,2020. Lugardon S, Desboeuf K, Fernet P, Montastruc JL, Lapeyre-Mestre M. Using a capture-recapture method to assess the frequency of adverse drug reactions in a French university hospital. Br J Clin Pharmacol 2006; 62:225-31.,2121. Wu TY, Jen MH, Bottle A, Molokhia M, Aylin P, Bell D, Majeed A. Ten-year trends in hospital admissions for adverse drug reactions in England 1999-2009. J R Soc Med 2010; 103:239-50.,2222. Stausberg J, Hasford J. Drug-related admissions and hospital-acquired adverse drug events in Germany: a longitudinal analysis from 2003 to 2007 of ICD-10-coded routine data. BMC Health Serv Res 2011; 11:134..
From those articles, a preliminary list of 860 codes was built up, of which only those appearing in at least two articles were retained, resulting in 465 codes. Another 128 were included as comprising special situations, although cited in only a single article. Codes excluded were those selected in a single article (224 codes), as well as those classified in previous studies as “possible” 1616. Rozenfeld S. Agravos provocados por medicamentos em hospitais do Estado do Rio de Janeiro, Brasil. Rev Saúde Pública 2007; 41:108-15. or in categories C or D 2222. Stausberg J, Hasford J. Drug-related admissions and hospital-acquired adverse drug events in Germany: a longitudinal analysis from 2003 to 2007 of ICD-10-coded routine data. BMC Health Serv Res 2011; 11:134., because the association between drug and ADEs was regarded as weak. Three additional codes (G21.2; L27.8 and L27.9) were included. The uncertainties regarding the inclusion of 14 codes were discussed with experts and assessed against one recent systematic review 2323. Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc 2014; 21:547-57..
The final list was subjected to preliminary analysis against the database. On the basis of those results, the code D70 (neutropenia) was excluded: although it met the inclusion criteria, having been cited by two authors 1616. Rozenfeld S. Agravos provocados por medicamentos em hospitais do Estado do Rio de Janeiro, Brasil. Rev Saúde Pública 2007; 41:108-15.,2020. Lugardon S, Desboeuf K, Fernet P, Montastruc JL, Lapeyre-Mestre M. Using a capture-recapture method to assess the frequency of adverse drug reactions in a French university hospital. Br J Clin Pharmacol 2006; 62:225-31., neutropenia was associated with a diagnosis of cancer in more than 70% of cases, thus indicating low specificity in detecting ADEs.
The final list developed for this study thus comprised 595 ICD-10 codes, classified into: “Adverse Effects - Chapter XX” (drugs, medicaments and biological substances in therapeutic use); “Adverse Effects - Other Chapters”; “Vaccines”; and “Intoxications” (Box 1).
Cases of ADEs were considered to be admissions with at least one code on the list, when recorded in one of the four database input fields, viz., principal diagnosis secondary diagnosis, associated causes and cause of death.
List of the 10th revision of the International Classification of Diseases (ICD-10) codes related to adverse drug events (ADEs).
Statistical analysis
The indicators of ADE frequency were “proportion of patients with ADEs” (number of patients with at least one ADE per 1,000 admissions) and “ADEs rate” (number of ADE per 1,000 admissions), stratified by year.
The database management system used was PostGreSQL version 9.3.3 (https://www.postgresql.org/). The study was conducted exclusively using secondary data from public-access databases and, was authorized by the Research Ethics Committee of the Sergio Arouca National School of Public Health, Oswaldo Cruz Foundation (Opinion n. 15/2013 - CEP-ENSP).
Results
In Brazil, 55,604,537 hospital admissions were recorded in the SIH-SUS, between 2008 and 2012, of which 273,440 (0.49%) were related to at least one ADE and ranging from 0.47 to 0.53, according to the year studied. Of them, 2,528 (0.92%) died. All of them had an ICD code of ADE as a cause of deaths.
The deaths corresponded to 235 different codes, but about a quarter of the cases were concentrated in the following codes: poisoning by other and unspecified drugs, medicaments and biological substances (T509) - 234 deaths; cardiomyopathy due to drugs and other external agents (I427) - 230 deaths; diseases of liver (K719, K717, K710) - 296 deaths; and poisoning by penicillins (T360) - 77 deaths. The other causes of death are scattered among the remaining codes.
Of total admissions associated with ADEs, 15,475 (5.7%) involved two different events and 62 (0.02%), three, bringing total events to 289,039 and resulting in a rate of 5.20 ADEs per 1,000 admissions. In spite of fluctuations, the proportions and rates seem to hold constant over the study period (Table 1). Most of the ADEs identified in the study period were those classified as “Adverse Effects - Other Chapters” and “Intoxications”, at rates of 3.56 and 1.38 per 1,000 admissions, respectively. The ADEs identified as in the “Adverse Effects - Chapter XX” and “Vaccines” groups resulted in rates of 0.23 and 0.02 per 1,000 admissions, respectively (Table 2).
Table 3 shows the 20 most frequent ADEs, which accounted for approximately 70% of the total. Fourteen of them were identified in codes relating to mental disorders. All the 20 most frequent ADEs were identified in codes classified in this study as “Adverse Effects - Other Chapters” and “Intoxications”.
Table 4 shows the five most frequent ADEs by category. In addition to the “Adverse Effects - Other Chapters” and “Intoxications”, which are among the 20 most frequent ADEs, as described above, also prominent were those classified as “Adverse Effects - Chapter XX”, associated with specific drugs such as oestrogens, progestogens and anticoagulants. In the “Vaccines” category, the most frequent events were associated with complications resulting from medical care.
Discussion
The study detected ADEs that had occurred in patients admitted to Brazil’s public hospital system between 2008 and 2012. For that purpose, it used the database of the SIH-SUS, to which a list of 595 codes of the ICD-10 was applied in order to identify cases. There were 273,440 hospitalizations associated with at least one ADE. As a frequency, 0.49% of total patients admitted were patients with events. In these, 289,039 events occurred, resulting in an ADE rate of 5.20 per 1,000 admissions. About 1% of the hospitalizations with ADE had an ICD code in the field “cause of death” which has significant impact to public health purpose. Although Chapter XX (drugs, medicaments and biological substances in therapeutic use) relates specifically to the problem studied here, the codes that returned most information were scattered across the various chapters of the ICD-10, mostly as relating to mental disorders.
Similar studies performed with databases in England for the period from 1996 to 2008 resulted in percentages from 0.35% to 0.9% 2121. Wu TY, Jen MH, Bottle A, Molokhia M, Aylin P, Bell D, Majeed A. Ten-year trends in hospital admissions for adverse drug reactions in England 1999-2009. J R Soc Med 2010; 103:239-50.,2424. Waller P, Shaw M, Ho D, Shakir S, Ebrahim S. Hospital admissions for "drug-induced" disorders in England: a study using the Hospital Episodes Statistics (HES) database. Br J Clin Pharmacol 2005; 59:213-9.,2525. Patel H, Bell D, Molokhia M, Srishanmuganathan J, Patel M, Car J, et al. Trends in hospital admissions for adverse drug reactions in England: analysis of national hospital episode statistics 1998-2005. BMC Clin Pharmacol 2007; 7:9. . Nonetheless, the English data do not include intoxications. Taking the proportion of hospitalizations in the SIH-SUS, less intoxications, 3.8 ADEs were identified per 1,000 hospitalizations, a value similar to the English data for the period 1996 to 2000 2424. Waller P, Shaw M, Ho D, Shakir S, Ebrahim S. Hospital admissions for "drug-induced" disorders in England: a study using the Hospital Episodes Statistics (HES) database. Br J Clin Pharmacol 2005; 59:213-9.. However, estimates by more recent European studies are higher than those for Brazil. One French study, which also excluded intoxications, found a proportion of 0.6% 2020. Lugardon S, Desboeuf K, Fernet P, Montastruc JL, Lapeyre-Mestre M. Using a capture-recapture method to assess the frequency of adverse drug reactions in a French university hospital. Br J Clin Pharmacol 2006; 62:225-31.; a German study, which used a listing of codes similar to the one used here, estimated 0.63% of hospitalizations with ADEs 2222. Stausberg J, Hasford J. Drug-related admissions and hospital-acquired adverse drug events in Germany: a longitudinal analysis from 2003 to 2007 of ICD-10-coded routine data. BMC Health Serv Res 2011; 11:134.. The differences encountered may be explained, not only by characteristics of the health systems, but by other factors, such as the set of codes selected to identify cases and the number of database input fields available for recording secondary diagnoses. However, the strategy used in this study allows us to identify smaller estimates than other methods 22. Martins AC, Giordani F, Rozenfeld S. Adverse drug events among adult inpatients: a meta-analysis of observational studies. J Clin Pharm Ther 2014; 39:609-20.,33. Atiqi R, Cleophas TJ, Van Bommel E, Zwinderman AH. Meta-analysis of recent studies on patients admitted to hospital due to adverse drug effects. Int J Clin Pharmacol Ther 2009; 47:549-55., since events are recorded by health professionals during providing health care.
Identification of ADEs by ICD-10 codes in administrative databases can be a complementary drug surveillance tool for monitoring large populations over time. It would be useful to develop a systematic, prospective use of the two systems, in order to compare and integrate their contribution, also in terms of focusing on the differences, if any, of their origin from the different settings of the health system.
In the countries mentioned above, more secondary diagnosis input fields are available than in the SIH-SUS and the mean number of secondary diagnoses entered was greater than 2.7 2626. Drösler S, Romano P, Wei L. Health care quality indicators project: patient safety indicators report 2009. Paris: Organisation for Economic Co-operation and Development; 2009. (OECD Health Working Papers, 47).. In Brazil, only one secondary diagnosis field is available and, even so, completion rates are low. In this study, 87% of secondary diagnosis fields were not completed in the SIH-SUS database. However, in recent years, two additional diagnosis fields (“associated causes” and “cause of death-related ICD code”) have been introduced into the database, extending the scope for recording ADEs.
The list of 595 ICD-10 codes drawn up from the literature review and by consulting experts was originally based on four articles that examined only administrative and hospital databases. A recent review of studies that used the ICD-10 2323. Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc 2014; 21:547-57. was examined with a view to validating the list. No substantial discrepancies were observed.
The intention was to build a comprehensive list, while endeavouring not to include codes that might result in an accumulation of false positives. Accordingly, codes expressing very frequent diagnoses that may or may not be associated with drugs were excluded. Although these may possibly be drug-related, other, highly prevalent causes cannot be discarded. “Gastrointestinal haemorrhage, unspecified” (K92.2) and “Neutropenia” (D70) are examples of this situation. The decision to exclude very frequently used codes is based on clinical and statistical criteria. For instance, gastrointestinal haemorrhage (GIH) (K92.2) is associated with widely used drugs, such as nonsteroidal anti-inflammatories. However, it may also be related to diagnoses such as ulcers, varices, esophagitis, gastritis, tumours, alcohol, vascular disease, infection and so on. The proportion of drug-related cases of GIH is relatively low (about 30%) compared with the proportion of cases of GIH related to other diseases.
Few studies offered validity analyses of the selected codes. Hohl et al. 2323. Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc 2014; 21:547-57., comparing data from a prospective study of ADEs with cases identified in an emergency service database, found sensitivity of 28%. Wu 2727. Wu C. Adverse drug reactions in the emergency department population in Ontario: analysis of national ambulatory care reporting system and discharge abstract database 2003-2007 [Masters Thesis]. Toronto: University of Toronto; 2009., meanwhile, comparing data from emergency electronic patient records with chart review, found sensitivity of 45% and specificity of 100%.
The most frequent ICD-10 codes present in the “Adverse Effects - Other Chapters” group reflect predominantly events relating to mental disorders and to psychotropic drugs. Even though some of these codes are non-specific, these findings are useful in characterizing problems with drugs in the field of psychiatric disorders. The inappropriate use or abuse of psychotropic drugs, together with the reduced use of other therapeutic resources, has been discussed in the Brazilian and international literature 2828. Cooke A. Understanding psychosis and schizophrenia. London: British Psychological Society; 2014.,2929. Almeida OP, Ratto L, Garrido R, Tamai S. Fatores preditores e consequências clínicas do uso de múltiplas medicações entre idosos atendidos em um serviço ambulatorial de saúde mental. Rev Bras Psiquiatr 1999; 21:152-7.,3030. Campos GWS, Onocko-Campos RT, Del Barrio LR. Políticas e práticas em saúde mental: as evidências em questão. Ciênc Saúde Coletiva 2013; 18:2797-805.. The use of benzodiazepine drugs for long periods, for non-specific conditions and in populations of older adults, for example, has been associated with several adverse outcomes, including dependence, cognitive decline, falls and even death 3131. Brett J, Murnion B. Management of benzodiazepine misuse and dependence. Aust Prescr 2015; 38:152-5.,3232. Kurko TA, Saastamoinen LK, Tähkäpää S, Tuulio-Henriksson A, Taiminen T, Tiihonen J, et al. Long-term use of benzodiazepines: definitions, prevalence and usage patterns - a systematic review of register-based studies. Eur Psychiatry 2015; 30:1037-47.. However, use of this class of drugs is widespread. In Europe, benzodiazepine prescription rates are from 570 to 1,700 per 10,000 person-years 3333. Huerta C, Abbing-Karahagopian V, Requena G, Oliva B, Alvarez Y, Gardarsdottir HG, et al. Exposure to benzodiazepines (anxiolytics, hypnotics and related drugs) in seven European electronic healthcare databases: a cross-national descriptive study from the PROTECT-EU Project. Pharmacoepidemiol Drug Saf 2016; 25 Suppl 1:56-65.. In Brazil, consumption of these drugs has been increasing. In 2012, 45 in every 10,000 residents of Brazil’s state capitals used one dose of benzodiazepine drugs every day of the year 3434. Azevedo AJ P, Araújo A A, Ferreira MAF. Consumo de ansiolíticos benzodiazepínicos: uma correlação entre dados do SNGPC e indicadores sociodemográficos nas capitais brasileiras. Ciênc Saúde Coletiva 2016; 21:83-90..
In this study, “Intoxications” figure as the second most frequently recorded group of ADEs in the SIH-SUS, comprising 76,866 hospitalizations. These estimates represent a significant portion of notifications of drug-related intoxications compiled by the Brazilian poisoning information system, which recorded 138,376 intoxications over the same period (Sistema Nacional de Informaçãoes Tóxico-Farmacológicas. Estatística anual de casos de intoxicação e envenenamento, Brasil 2008- 2012. http://sinitox.icict.fiocruz.br/dados-nacionais, accessed on 04/Nov/2017). Prominent among drug classes related to intoxications are antibiotics (systemic-use and penicillins). Neurotoxic effects from several groups of antibiotics are common among vulnerable individuals 3535. Grill MF, Maganti RK. Neurotoxic effects associated with antibiotic use: management considerations. Br J Clin Pharmacol 2011; 72:381-93.. Age extremes, prior neurological disease and renal failure are factors that should be considered in adjusting doses in this population and preventing neurotoxicity associated with these drugs.
Among admissions evaluated in the SIH-SUS, benzodiazepines figured among the five most frequent intoxication-related causes recorded. Although widely prescribed for more than 50 years, the risk-benefit ratio of using benzodiazepines is still being discussed 3636. Lader M. Benzodiazepines revisited - will we ever learn? Addiction 2011; 106: 2086-109.. There is worldwide concern with the risks involved in the use of this drug class, as mentioned earlier, and the findings of this study corroborate the data of international studies 3737. Centers for Disease Control and Prevention. Unintentional and undetermined poisoning deaths: 11 states, 1990-2001. MMWR Morb Mortal Wkly Rep 2004; 53:233-8.,3838. Chan YC, Tse ML, Lau FL. Hong Kong Poison Information Centre: annual report 2014. Hong Kong Journal of Emergency Medicine 2015; 22:376-87..
Anticoagulants are among the drug classes most involved in ADEs relate to non-specific drugs among inpatients (“Adverse Effects - Chapter XX”) 3939. Runciman WB, Roughead EE, Semple SJ, Adams RJ. Adverse drug events and medication errors in Australia. Int J Qual Health Care 2003; 15 Suppl 1:i49-59.,4040. Classen DC, Jaser L, Budnitz DS. Adverse drug events among hospitalized Medicare patients: epidemiology and national estimates from a new approach to surveillance. Jt Comm J Qual Patient Saf 2010; 36:12-21.,4141. Morimoto T, Sakuma M, Matsui K, Kuramoto N, Toshiro J, Murakami J, et al. Incidence of adverse drug events and medication errors in Japan: the JADE Study. J Gen Intern Med 2010; 26:148-53.. They are considered high-alert medications, because they entail heightened risk of causing significant patient harm when administration is faulty 4242. Cohen MR, Smetzer JL, Tuohy NR, Kilo CM. High-alert medications: safeguarding against errors. In: Cohen MR , editor. Medication errors. 2nd Ed. Washington DC: American Pharmaceutical Association; 2006. p. 317-411.. In the USA, a large number of hospitalizations are associated with ADEs from the therapeutic use of anticoagulants, many of them regarded as medication errors and thus preventable 4343. Piazza G, Nguyen TN, Cios D, Labreche M, Hohlfelder B, Fanikos J, et al. Anticoagulation-associated adverse drug events. Am J Med 2011; 124:1136-42.. Studies in Brazil show that anticoagulants are among the classes most implicated in ADEs detected in inpatients by the trigger method 1414. Giordani F, Rozenfeld S, Martins M. Adverse drug events identified by triggers at a teaching hospital in Brazil. BMC Pharmacol Toxicol 2014; 15:71.,1616. Rozenfeld S. Agravos provocados por medicamentos em hospitais do Estado do Rio de Janeiro, Brasil. Rev Saúde Pública 2007; 41:108-15..
As regards vaccines, of the five most frequent codes, two related to complications from medical care: “Other complications following immunization, not elsewhere classified” (T88.1) and “Infection following immunization” (T88.0). In Northeast Brazil, a descriptive study using data from the National Vaccine Adverse Event Reporting System identified 667 adverse events in 402 records evaluated; adverse events occurred more commonly in bacterial (82.6%) than in viral vaccines 4444. Moura ADA, Costa AS, Braga AVL, ECSA Lima GG, Chaves ES. Vigilância de eventos adversos pós-vacinação no estado do Ceará, em 2011. Epidemiol Serv Saúde 2015; 24:155-60.. The Institute for Safe Medication Practices alerts to the risk of medication errors involving vaccines, which can precipitate severe events, including reported deaths 4545. Reis AM, Cassiani SH. Adverse drug events in an intensive care unit of a university hospital. Eur J Clin Pharmacol 2011; 67:625-32.. One US study, describing errors reported to the Vaccine Adverse Event Reporting System, from 2000 to 2013, received a total of 311,185 reports of adverse events following vaccination, of which 7% were vaccination error reports, and documented 21,843 errors. Reports increased from 10 per year in 2000 to 4,284 in 2013 4646. Hibbs BF, Moro PL, Lewis P, Miller ER, Shimabukuro TT. Vaccination errors reported to the Vaccine Adverse Event Reporting System (VAERS), United States, 2000-2013. Vaccine 2015; 33:3171-8..
The characteristics of the events and of the drugs, as well as the measures necessary to prevent them, may be different depending on whether they originate inside or outside the hospital. The ADE related to mental disorders and those related to complications following immunization illustrate the differences, the first one occurring inside or outside the hospital and the second one only in ambulatory setting, demanding distinct management of intervention and prevention.
Limitations in the reliability of the results presented here can be discussed at several levels. The small number of diagnosis input fields for Brazilian databases is certainly a factor of concern, which may reduce the usefulness of the list of codes for calculating ADE occurrence statistics. The SIH-SUS offers only two such input fields (principal and secondary diagnosis), while low completion rates for secondary diagnosis particularly limit information on events. However, these issues have not prevented the ICD-10 from being used, because they are not problems of Brazilian databases alone 1818. Miguel A, Marques B, Freitas A, Lopes F, Azevedo L, Pereira AC. Detection of adverse drug reactions using hospital databases: a nationwide study in Portugal. Pharmacoepidemiol Drug Saf 2013; 22:907-13..
Also in relation to the diagnosis fields, the quality and rules of completion entail important implications for the study of ADEs. In order to monitor, control and reduce the frequency with which adverse reactions and errors occur in drug indication, dispensing and administration processes, it is of interest to know if the event occurred at hospital admission or during the hospital stay. In Stausberg & Hasford 2222. Stausberg J, Hasford J. Drug-related admissions and hospital-acquired adverse drug events in Germany: a longitudinal analysis from 2003 to 2007 of ICD-10-coded routine data. BMC Health Serv Res 2011; 11:134., the event is considered present at admission if recorded in the principal diagnosis; if in the secondary diagnosis, it may have occurred during the hospital stay. However, that assumption has limitations. In Brazil, the guidelines for completing principal and secondary diagnosis fields are restricted to certain specific pathologies and procedures. Although in some cases, completion of the secondary diagnosis field is mandatory, as in the cases of disorders from external causes 4747. Coordenação-Geral de Sistemas de Informação, Departamento de Regulação, Avaliação e Controle, Secretaria de Atenção à Saúde, Ministério da Saúde. Manual técnico operacional do sistema de informação hospitalar do SUS. Brasília: Ministério da Saúde; 2014., it is possible that there is under-reporting.
One important shortcoming in Brazil’s national health care system, which compromises service evaluation and the quality of patient care, is the absence of information on drugs prescribed. Brazilian databases offer no information on medications prescribed or used, with the exception of a restricted group of (generally high-cost) drugs, which are reported as procedures in group six of the SIH-SUS schedule of procedures.
Studies using large databases encounter considerable theoretical and operational difficulties, which can affect event frequency statistics. The most prominent difficulties include the concept and diagnosis of ADEs and the identification of events among ICD codes. One pioneering study of methodological standardization, using the same ICD-10 codes and defining variables for adjusting rates, found adjusted prevalences of 5.64%, 4.78% and 3.22% for the USA, Germany and England, respectively. All the same, the comparison is limited, especially by differences in service structure and diagnosis coding processes 4848. Stausberg J. International prevalence of adverse drug events in hospitals: an analysis of routine data from England, Germany, and the USA. BMC Health Serv Res 2014; 14:125..
As there is no single, internationally-agreed list of codes, the list of codes built up for this study was based on lists provided by a number of authors, consultations with experts, the literature and objective selection criteria. This study points to a need to develop a single list suited to ADE research and monitoring. Efforts to achieve this goal are ongoing. In addition to the ICD-WHO code list there is the Medical Dictionary for Regulatory Activities (MedDRA) Terminology, an international medical terminology list. Members of the WHO and MedRA have attempted to harmonize these systems 4949. International Council of Harmonisation. MedDRA introductory guide version 20.0. https://www.meddra.org/sites/default/files/guidance/file/intguide_20_0_english.pdf (accessed on 04/Nov/2017).
https://www.meddra.org/sites/default/fil... .
In conclusion, although there is a similar previous study, restricted to the State of Rio de Janeiro 1616. Rozenfeld S. Agravos provocados por medicamentos em hospitais do Estado do Rio de Janeiro, Brasil. Rev Saúde Pública 2007; 41:108-15., the strength and originality of this study is that it analyses 11 million hospitalizations yearly and describes the main causes of adverse drug events.
The ICD-10 code listing proved applicable to Brazilian conditions, in spite of the small number of diagnosis input fields available in SIH-SUS administrative database. In future, validation studies using other databases or hospital patient records will be an important research strategy.
Monitoring of ADEs in administrative databases using ICD-10 codes is feasible and can be an innovative tool to complement drug surveillance strategies in Brazil, as well as in other countries.
Acknowledgments
The authors would like to thank the Professor Dr. Ruy Laurenti (in memory), from the Faculty of Public Health of the University of São Paulo, and Dra. Graças Mota Melo, from Oswaldo Cruz Foundation, for the collaboration in the definition of codes. The authors would like to thank INOVA-ENSP/Fiocruz for the financial support.
References
- 1World Health Organization. The conceptual framework for the international classification for patient safety. http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf (accessed on 04/Nov/2017).
» http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf - 2Martins AC, Giordani F, Rozenfeld S. Adverse drug events among adult inpatients: a meta-analysis of observational studies. J Clin Pharm Ther 2014; 39:609-20.
- 3Atiqi R, Cleophas TJ, Van Bommel E, Zwinderman AH. Meta-analysis of recent studies on patients admitted to hospital due to adverse drug effects. Int J Clin Pharmacol Ther 2009; 47:549-55.
- 4Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163:1409-16.
- 5Ministério da Saúde. Portaria nº 529, de 1º de abril de 2013. Institui o Programa Nacional de Segurança do Paciente (PNSP). Diário Oficial da União 2013; 2 abr.
- 6Organização Pan-Americana da Saúde. Aliança mundial para segurança do paciente. Boletim informativo da OMS para segurança do paciente. http://www.paho.org/bra/index.php?option=com_content&view=article&id=931&Itemid=0 (accessed on 04/Nov/2017)
» http://www.paho.org/bra/index.php?option=com_content&view=article&id=931&Itemid=0 - 7Hakkarainen KM, Hedna K, Petzold M, Hägg S. Percentage of patients with preventable adverse drug reactions and preventability of adverse drug reactions: a meta-analysis. PLoS One 2012; 7:e33236.
- 8Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates D. Adverse drug events and medication errors: detection and classification methods. Qual Saf Health Care 2004; 13:306-14.
- 9Agência Nacional de Vigilância Sanitária. Relatórios Notivisa: evolução das notificações 2006 a 2013. http://www.anvisa.gov.br/hotsite/notivisa/relatorios/index.htm (accessed on 12/Nov/2017)
» http://www.anvisa.gov.br/hotsite/notivisa/relatorios/index.htm - 10Santos DB, Coelho HL. Adverse drug reactions in hospitalized children in Fortaleza, Brazil. Pharmacoepidemiol Drug Saf 2006; 15:635-40.
- 11Camargo AL, Cardoso Ferreira MB, Heineck I. Adverse drug reactions: a cohort study in internal medicine units at a university hospital. Eur J Clin Pharmacol 2006; 62:143-9.
- 12Passarelli MCG, Jacob-Filho W. Reações adversas a medicamentos em idosos: como prevê-las? Einstein (São Paulo) 2007; 5:246-51.
- 13Roque KE, Melo ECP. Adaptação dos critérios de avaliação de eventos adversos a medicamentos para uso em um hospital público no Estado do Rio de Janeiro. Rev Bras Epidemiol 2010; 13:607-19.
- 14Giordani F, Rozenfeld S, Martins M. Adverse drug events identified by triggers at a teaching hospital in Brazil. BMC Pharmacol Toxicol 2014; 15:71.
- 15Varallo FR, Capucho HC, Planeta CDS, Mastrolannin PC. Possible adverse drug events leading to hospital admission in a Brazilian teaching hospital. Clinics 2014; 69:163-7.
- 16Rozenfeld S. Agravos provocados por medicamentos em hospitais do Estado do Rio de Janeiro, Brasil. Rev Saúde Pública 2007; 41:108-15.
- 17Hougland P, Xu W, Pickard S, Masheter C, Williams SD. Performance of International Classification of Diseases, 9th revision, linical modification codes as an adverse drug event surveillance system. Med Care 2006; 44:629-36.
- 18Miguel A, Marques B, Freitas A, Lopes F, Azevedo L, Pereira AC. Detection of adverse drug reactions using hospital databases: a nationwide study in Portugal. Pharmacoepidemiol Drug Saf 2013; 22:907-13.
- 19Instituto Brasileiro de Geografia e Estatística. Sistema de Informações Hospitalares do SUS - SIH/SUS. http://ces.ibge.gov.br/base-de-dados/metadados/ministerio-da-saude/sistema-de-informacoes-hospitalares-do-sus-sih-sus.html (accessed on 04/Nov/2017)
» http://ces.ibge.gov.br/base-de-dados/metadados/ministerio-da-saude/sistema-de-informacoes-hospitalares-do-sus-sih-sus.html - 20Lugardon S, Desboeuf K, Fernet P, Montastruc JL, Lapeyre-Mestre M. Using a capture-recapture method to assess the frequency of adverse drug reactions in a French university hospital. Br J Clin Pharmacol 2006; 62:225-31.
- 21Wu TY, Jen MH, Bottle A, Molokhia M, Aylin P, Bell D, Majeed A. Ten-year trends in hospital admissions for adverse drug reactions in England 1999-2009. J R Soc Med 2010; 103:239-50.
- 22Stausberg J, Hasford J. Drug-related admissions and hospital-acquired adverse drug events in Germany: a longitudinal analysis from 2003 to 2007 of ICD-10-coded routine data. BMC Health Serv Res 2011; 11:134.
- 23Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc 2014; 21:547-57.
- 24Waller P, Shaw M, Ho D, Shakir S, Ebrahim S. Hospital admissions for "drug-induced" disorders in England: a study using the Hospital Episodes Statistics (HES) database. Br J Clin Pharmacol 2005; 59:213-9.
- 25Patel H, Bell D, Molokhia M, Srishanmuganathan J, Patel M, Car J, et al. Trends in hospital admissions for adverse drug reactions in England: analysis of national hospital episode statistics 1998-2005. BMC Clin Pharmacol 2007; 7:9.
- 26Drösler S, Romano P, Wei L. Health care quality indicators project: patient safety indicators report 2009. Paris: Organisation for Economic Co-operation and Development; 2009. (OECD Health Working Papers, 47).
- 27Wu C. Adverse drug reactions in the emergency department population in Ontario: analysis of national ambulatory care reporting system and discharge abstract database 2003-2007 [Masters Thesis]. Toronto: University of Toronto; 2009.
- 28Cooke A. Understanding psychosis and schizophrenia. London: British Psychological Society; 2014.
- 29Almeida OP, Ratto L, Garrido R, Tamai S. Fatores preditores e consequências clínicas do uso de múltiplas medicações entre idosos atendidos em um serviço ambulatorial de saúde mental. Rev Bras Psiquiatr 1999; 21:152-7.
- 30Campos GWS, Onocko-Campos RT, Del Barrio LR. Políticas e práticas em saúde mental: as evidências em questão. Ciênc Saúde Coletiva 2013; 18:2797-805.
- 31Brett J, Murnion B. Management of benzodiazepine misuse and dependence. Aust Prescr 2015; 38:152-5.
- 32Kurko TA, Saastamoinen LK, Tähkäpää S, Tuulio-Henriksson A, Taiminen T, Tiihonen J, et al. Long-term use of benzodiazepines: definitions, prevalence and usage patterns - a systematic review of register-based studies. Eur Psychiatry 2015; 30:1037-47.
- 33Huerta C, Abbing-Karahagopian V, Requena G, Oliva B, Alvarez Y, Gardarsdottir HG, et al. Exposure to benzodiazepines (anxiolytics, hypnotics and related drugs) in seven European electronic healthcare databases: a cross-national descriptive study from the PROTECT-EU Project. Pharmacoepidemiol Drug Saf 2016; 25 Suppl 1:56-65.
- 34Azevedo AJ P, Araújo A A, Ferreira MAF. Consumo de ansiolíticos benzodiazepínicos: uma correlação entre dados do SNGPC e indicadores sociodemográficos nas capitais brasileiras. Ciênc Saúde Coletiva 2016; 21:83-90.
- 35Grill MF, Maganti RK. Neurotoxic effects associated with antibiotic use: management considerations. Br J Clin Pharmacol 2011; 72:381-93.
- 36Lader M. Benzodiazepines revisited - will we ever learn? Addiction 2011; 106: 2086-109.
- 37Centers for Disease Control and Prevention. Unintentional and undetermined poisoning deaths: 11 states, 1990-2001. MMWR Morb Mortal Wkly Rep 2004; 53:233-8.
- 38Chan YC, Tse ML, Lau FL. Hong Kong Poison Information Centre: annual report 2014. Hong Kong Journal of Emergency Medicine 2015; 22:376-87.
- 39Runciman WB, Roughead EE, Semple SJ, Adams RJ. Adverse drug events and medication errors in Australia. Int J Qual Health Care 2003; 15 Suppl 1:i49-59.
- 40Classen DC, Jaser L, Budnitz DS. Adverse drug events among hospitalized Medicare patients: epidemiology and national estimates from a new approach to surveillance. Jt Comm J Qual Patient Saf 2010; 36:12-21.
- 41Morimoto T, Sakuma M, Matsui K, Kuramoto N, Toshiro J, Murakami J, et al. Incidence of adverse drug events and medication errors in Japan: the JADE Study. J Gen Intern Med 2010; 26:148-53.
- 42Cohen MR, Smetzer JL, Tuohy NR, Kilo CM. High-alert medications: safeguarding against errors. In: Cohen MR , editor. Medication errors. 2nd Ed. Washington DC: American Pharmaceutical Association; 2006. p. 317-411.
- 43Piazza G, Nguyen TN, Cios D, Labreche M, Hohlfelder B, Fanikos J, et al. Anticoagulation-associated adverse drug events. Am J Med 2011; 124:1136-42.
- 44Moura ADA, Costa AS, Braga AVL, ECSA Lima GG, Chaves ES. Vigilância de eventos adversos pós-vacinação no estado do Ceará, em 2011. Epidemiol Serv Saúde 2015; 24:155-60.
- 45Reis AM, Cassiani SH. Adverse drug events in an intensive care unit of a university hospital. Eur J Clin Pharmacol 2011; 67:625-32.
- 46Hibbs BF, Moro PL, Lewis P, Miller ER, Shimabukuro TT. Vaccination errors reported to the Vaccine Adverse Event Reporting System (VAERS), United States, 2000-2013. Vaccine 2015; 33:3171-8.
- 47Coordenação-Geral de Sistemas de Informação, Departamento de Regulação, Avaliação e Controle, Secretaria de Atenção à Saúde, Ministério da Saúde. Manual técnico operacional do sistema de informação hospitalar do SUS. Brasília: Ministério da Saúde; 2014.
- 48Stausberg J. International prevalence of adverse drug events in hospitals: an analysis of routine data from England, Germany, and the USA. BMC Health Serv Res 2014; 14:125.
- 49International Council of Harmonisation. MedDRA introductory guide version 20.0. https://www.meddra.org/sites/default/files/guidance/file/intguide_20_0_english.pdf (accessed on 04/Nov/2017).
» https://www.meddra.org/sites/default/files/guidance/file/intguide_20_0_english.pdf
Publication Dates
- Publication in this collection
29 Nov 2018
History
- Received
23 Jan 2018 - Reviewed
05 Aug 2018 - Accepted
31 Aug 2018