THEME PAPERS

Monitoring signals for vaccine safety: the assessment of individual adverse event reports by an expert advisory committee

Surveillance de la sécurité des vaccins : évaluation des cas de réactions indésirables par un comité consultatif d’experts

Vigilancia de la seguridad de las vacunas: evaluación de las notificaciones de episodios adversos por un comité consultivo de expertos

J.-P. Collet^I; N. MacDonald^II; N. Cashman^III; R. Pless^IV; the Advisory Committee on Causality Assessment¹

^IDirector of the Randomized Clinical Trial Unit, Centre for Clinical Epidemiology and Community Studies, The Sir Mortimer B. Davis — Jewish General Hospital, 3755 avenue Côte Ste-Catherine, suite A-132, Montreal, Québec H3T 1E2, Canada

^IIChief, Division of Infectious Disease, Children’s Hospital of Eastern Ontario, Ontario, Canada; Professor, Departments of Pediatrics and Microbiology, University of Ottawa, Ottawa, Canada

^IIIProfessor, Department of Medicine, Centre for Research in Neurodegenerative Diseases, University of Toronto, Toronto, Canada

^IVActing Chief and Head, Vaccine Safety Surveillance Section, Division of Immunization, Laboratory Centre for Disease Control, Health Canada, Ottawa, Canada

Correspondence

ABSTRACT

Monitoring vaccine safety is a complex and shared responsibility. It can be carried out in many ways, one of which is the reporting of individual cases of adverse reactions thought to be due to vaccination. The task is difficult because ascribing causality to an individual case report is fraught with challenges. A standardized evaluation instrument — known as the causality assessment form — was therefore developed for use by an expert advisory committee to facilitate the process. By following the several sections in this form, the members of the committee are taken through series of points to establish causality. These points include the basic criteria for causation such as biological plausibility, the time elapsed between the vaccine administration and the onset of the adverse event, and whether other factors (drugs, chemicals or underlying disease) could account for the adverse symptoms. The form concludes with consensus assessment of causality, a commentary about the assessment, and advice for further study or follow-up. This method of assessing the more serious cases of adverse reaction reported to vaccination has proven useful in evaluating ongoing safety of vaccines in Canada. Through analyses such as this, new signals can be identified and investigated further.

Keywords: adverse drug reaction reporting systems, methods; Canada; drug utilization review, methods; pharmacy and therapeutics committee; vaccines, adverse effects

RÉSUMÉ

La surveillance de la sécurité des vaccins est une responsabilité complexe et partagée, qui peut prendre de nombreuses formes, notamment celle de la conduite de grands essais cliniques après homologation, d’études de regroupement des dossiers qui retracent les visites de santé ayant fait suite à la vaccination, ou d’études de suivi plus ciblées, par ex. à partir des carnets de santé. Toutefois, dans la plupart des pays, l’élément essentiel des systèmes de surveillance est le système de notification passive qui repose sur la vigilance des prestateurs de soins de santé. Au Canada, les cas de réactions indésirables à une vaccination sont soumis, par l’intermédiaire des ministères de la santé des provinces et territoires, au Programme de surveillance des effets indésirables des vaccins de la Division Vaccination, Bureau des Maladies infectieuses, Santé Canada, sur un formulaire de notification spécial utilisé (à quelques modifications mineures près) dans l’ensemble du pays. Tous ces systèmes de notification passive présentent les mêmes inconvénients, à savoir une sous-notification, une qualité variable et des notifications incomplètes. Ces dernières font qu’on se heurte à de multiples difficultés pour attribuer une causalité à un cas donné. Néanmoins, il faut effectuer une évaluation critique des dossiers relatifs à des cas individuels afin de déterminer s’il existe véritablement un lien plausible entre la vaccination et la réaction indésirable, ou s’il ne s’agit que d’une coïncidence dans le temps, l’effet indésirable représentant alors une maladie sous-jacente.

On a réuni un groupe d’experts (Comité consultatif sur l’évaluation de la causalité (ACCA)) pour analyser toutes les réactions indésirables graves signalées au Canada par le système de notification passive et par un système de surveillance active des cas pédiatriques en milieu hospitalier. Pour faciliter ce processus, l’ACCA a d’abord élaboré un instrument d’évaluation normalisé, basé sur des techniques d’évaluation de la causalité décrites dans la littérature. Ce questionnaire contient plusieurs rubriques qui amènent les membres du Comité à toute une série de points liés à l’évaluation de la causalité de façon à faciliter le recours à la technique d’évaluation par « introspection globale » pour parvenir à une conclusion au moyen des définitions de la causalité adoptées par l’OMS : « très probable », « probable », « possible », « peu probable » et « sans aucun rapport ». Les questions posées comportent les tests de base tels que la plausibilité biologique, les délais entre l’administration du vaccin et le début de la réaction indésirable, et la recherche d’autres facteurs qui pourraient expliquer les symptômes. Le formulaire est rempli et accompagné d’une évaluation par consensus, d’un commentaire et de conseils relatifs à une étude ou à un suivi ultérieurs.

Le Comité, qui s’est réuni deux fois par an depuis 1994, analyse 120 à 220 des 4000 à 5000 notifications de cas reçues chaque année. Il s’agit des réactions rapportées les plus graves, qui comprennent des catégories diagnostiques particulières et toutes les réactions ayant conduit à une hospitalisation ou ayant été mortelles. Des copies de chaque dossier, comprenant des photocopies du formulaire de notification d’une réaction indésirable à un vaccin rempli et tout autre document supplémentaire, par exemple les résumés de dossier rédigés à la sortie de l’hôpital, les notes prises lors des consultations ou les fiches cliniques, et les résultats de laboratoire, sont fournies aux membres du Comité avant chaque réunion – débarrassées de toute indication personnelle permettant d’identifier les patients.

Les évaluations de cas, outre le fait qu’elles servent à surveiller à la sécurité des vaccins, sont renvoyées aux autorités de santé publique responsables de chaque province ou de chaque territoire (copies) pour les informer en retour. Chaque envoi est accompagné d’une mention indiquant les limites du processus d’analyse.

Cette méthode d’évaluation des cas les plus graves de réactions indésirables à la vaccination s’est avérée utile pour évaluer la sécurité des vaccins au Canada. Outre le fait qu’elles sont rassurantes pour ce qui concerne la sécurité, ces évaluations de la causalité ont également permis de souligner le fait que seules environ 18% des réactions graves ont été considérées comme « en rapport » ou « probablement en rapport » avec la vaccination, alors que 24,7% ont été considérées comme « sans aucun rapport » avec elle. Toutefois, près de 25,8% des dossiers de cas ne contenaient pas suffisamment d’informations pour qu’on puisse effectuer une évaluation correcte, ce qui laisse à penser qu’il faut encore travailler à améliorer la qualité de la notification dans toute la mesure possible.

RESUMEN

La vigilancia de la seguridad de las vacunas es una responsabilidad compleja y compartida, y puede adoptar diversas formas, entre ellas la realización de amplios ensayos clínicos tras la aprobación del producto, estudios de relación de registros que permiten seguir las consultas médicas tras la vacunación, o estudios de seguimiento más focalizados, como los que utilizan diarios de salud. Sin embargo, la piedra angular de los sistemas de vigilancia en la mayoría de los países son mecanismos de notificación pasivos basados en la vigilancia de los agentes de salud. En el Canadá, los informes de episodios adversos postinmunización se remiten, a través de los ministerios provinciales y territoriales de salud, al Programa de vigilancia de episodios adversos asociados a vacunas, de la División de Inmunización, Oficina de Enfermedades Infecciosas, Health Canada, mediante un formulario especial de notificación de episodios adversos empleado (con pequeñas modificaciones) en todo el país. Todos estos sistemas de notificación pasivos adolecen de limitaciones similares, en particular la subnotificación, la desigual calidad y la fragmentariedad de los informes de los casos Esta última deficiencia explica que la tarea de establecer una relación causal en un determinado caso esté plagada de dificultades. No obstante, es necesario evaluar críticamente cada informe, a fin de determinar si en efecto existe una relación plausible entre la vacunación y el episodio adverso, o si se trata por el contrario de una mera coincidencia en el tiempo, reflejo de una enfermedad subyacente.

Se convocó a un grupo de expertos (Comité consultivo de evaluación de la causalidad) para que estudiara todos los episodios adversos graves definidos que se hubieran notificado en el Canadá a partir tanto del sistema de notificación pasiva como del sistema de vigilancia activa basado en hospitales pediátricos. Para facilitar este proceso, el Comité elaboró en primer lugar un instrumento de evaluación normalizada basado en técnicas de evaluación de la causalidad descritas en la literatura. El cuestionario contiene diversas secciones, a través de las cuales los miembros del Comité abordan una serie de puntos relacionados con la evaluación de la causalidad para poder aplicar más fácilmente la técnica de evaluación por «introspección global» y llegar a una conclusión utilizando las definiciones de causalidad de la OMS: «muy probable», «probable», «posible», «improbable» y «no relacionado». Las preguntas incluyen pruebas básicas para establecer la causalidad, como la plausibilidad biológica, el tiempo transcurrido entre la administración de la vacuna y el inicio del episodio adverso, y la existencia o no de otros factores que puedan explicar los síntomas. El formulario termina con una evaluación de consenso, reservándose espacio para añadir observaciones y posibles consejos para futuros estudios o actividades de seguimiento.

El Comité, que se ha reunido dos veces al año desde 1994, examina 120-220 de los 4000-5000 informes de casos recibidos anualmente. La cifra comprende los episodios notificados de mayor gravedad, lo que incluye categorías de diagnóstico particulares y todos los episodios que requirieron hospitalización o desembocaron en la defunción. Antes de cada reunión, se distribuyen a los miembros del Comité, desprovistas de toda identificación personal, copias de todos los informes de los casos, consistentes en fotocopias del formulario rellenado de notificación del episodio adverso asociado a la vacuna y cualquier otro documento adicional (como resúmenes de altas, notas de consultas u observaciones médicas, y resultados de laboratorio).

Las evaluaciones de los casos aportan información respecto a la seguridad de las vacunas, y se remiten a modo de retroinformación a las autoridades de salud pública de cada provincia o territorio. En cada envío se adjunta una declaración en la que se especifican las limitaciones del proceso de examen.

Este método de evaluación de los casos más graves de episodios adversos relacionados con vacunaciones ha demostrado ser eficaz para evaluar de forma continua la seguridad de las vacunas en el Canadá. Estas evaluaciones de la causalidad, además de tener un efecto tranquilizador en lo concerniente a la seguridad, han puesto de manifiesto que sólo un 18% de las reacciones graves estaba «relacionado» o «probablemente relacionado» con la vacunación, mientras que el 24,7% de los casos se consideraron no relacionados. Sin embargo, en el 25,8% de los casos la información facilitada era insuficiente para evaluar cabalmente la causalidad, lo que lleva a pensar que es preciso seguir trabajando para mejorar en la medida de lo posible la calidad de los informes.

Introduction

The continuous monitoring of the safety of drugs once they are licensed and in widespread use is a complex and shared responsibility between governments, industry, health care providers and patients. This responsibility is all the more important for vaccines, which are administered on a large scale to healthy individuals for anticipated benefits. Vaccines demand a very high degree of safety. Parents of infants receiving their routine immunizations need reassurance that vaccines are safe.

The safety monitoring of any drug licensed for use can take many forms, including large post-approval clinical trials, record linkage studies that track health care visits following vaccination, or more targeted follow-up studies such as those using health diaries. However, the cornerstone of surveillance systems in most countries is passive reporting schemes that rely on the vigilance of health care providers (1) to detect events that are felt to be due to the administration of a drug product (vaccines in this case), and their reporting of these cases to a regional or national authority or to the manufacturing company. In Canada the systems in place to monitor vaccine safety include active surveillance for serious reactions that would result in admission to a paediatric hospital, in addition to the voluntary (spontaneous) reporting system (2–4). Both these systems rely on the collection of case reports, and an intimate partnership between the provincial and territorial ministries of health and the Vaccine-Associated Adverse Events Surveillance Program in the Division of Immunization, Bureau of Infectious Diseases, at Health Canada. The majority of case reports are submitted by public health nurses, especially in those provinces where immunization delivery is mainly through public health clinics. Physicians report less than 10% of the cases, while the public is asked to report through their physician or public health unit rather than directly to improve not only the diagnosis and reporting per se, but also to ensure that the vaccine provider is made aware of the alleged event. Because the system is so well established through public health channels, manufacturers receive few case reports directly (less than 5%). Reporting is voluntary in all provinces except one, Ontario, where selected events are part of the same mandatory reporting legislation as reportable infectious diseases. Nevertheless, reporting rates are not improved by this requirement. There are a variety of reasons why physicians report so poorly overall, the most important being lack of awareness of the reporting system and its criteria, rather than the burden of reporting (5). This limitation is significant as the diagnosis of the most serious adverse events relies on reporting by physicians.

Data analysis is a complex undertaking, whose ultimate goal is the search for signals that may trigger immediate action to alter or suspend the use of a vaccine or vaccine lot, suggest changes to the product labelling, or lead to the initiation of formal clinical or epidemiological trials to confirm or refute the concern. To the best of our knowledge, a major intervention regarding the use of a vaccine has only been required twice in Canada. The only occasion that a safety concern arose was the withdrawal in 1987 of a measles–mumps–rubella (MMR) vaccine containing the Urabe strain of mumps vaccine which posed an excess risk of aseptic meningitis in the country (6). There was a later recall of an MMR product from the field on discovery of vials that reconstituted poorly and were discoloured. It turned out that a packaging unit had “caramelized” the product.

The information supplied in adverse-event case reports relating to vaccines varies in quality and completeness. Practitioners are encouraged to submit cases that are mere suspicions, and need not have a known causal relationship. As a result, a critical assessment of individual case reports is necessary to determine if some plausible relationship really exists between the vaccination and the adverse event described, or whether the link with vaccination was a coincidence while the adverse event was due to an underlying illness. This level of review is also important for reports obtained through active monitoring programmes, as these cases are identified using pre-determined clinical criteria regardless of causality. It is only by reporting suspicions, along with well-known adverse reactions, that new signals or concerns can be generated which may need further investigation. However, this approach casts a very broad net and makes the evaluation of many cases difficult, requiring the assistance of a panel of experts to evaluate whether or not the implicated vaccine could indeed be responsible. Application of a rigorous evaluation method for the causality assessments is required to standardize the procedures.

To determine causality, some details regarding the event (i.e. clinical description, natural incidence, etiological factors, etc.) and its timing in relation to vaccine exposure according to the pathophysiological mechanism of production, and the possible effect of other contributing factors (such as underlying disease, or administration of other drugs) have to be considered. Several methodological strategies have been proposed to collect and combine this type of information in a systematic manner (7). However, these methods have been developed for adverse drug reactions (ADR); and, despite the relative specificity of vaccine-associated adverse events (VAAE, a term used in Canada which is equivalent to the more common term, AEFI — adverse event following immunization), no such systematic approach to investigation has been published to evaluate causality for VAAE.

This article describes the assessment methods developed by the Advisory Committee on Causality Assessment (ACCA), an expert advisory group first convened in March 1994 by the Division of Immunization to review all reports of serious and unusual VAAE from both active and passive monitoring systems in Canada.

Methods

The vaccine-associated adverse events programme in Canada receives about 4000–5000 case reports each year. The great majority of these cases (over 95%) describe minor or well-known reactions related to vaccines, which are monitored under routine surveillance by the Laboratory Center for Disease Control (LCDC), e.g. febrile seizures after administration of MMR vaccine. However, an unusual change in the frequency of these reported reactions, or concerns expressed by an individual immunization clinic or region regarding the vaccines which they are administering, would trigger an immediate investigation and appropriate action taken if necessary. The most serious and unusual reactions requiring detailed review are submitted to ACCA; at each twice yearly meeting, between 60 and 110 cases are evaluated. Pressing issues can also be submitted by e-mail or teleconference for review by the committee. ACCA is composed of specialists in paediatrics, epidemiology, infectious diseases, immunology, neurology, pathology, adverse event surveillance, and microbiology and has been reviewing cases and refining its methodology since its inception.

In order to ensure that all the important case reports are reviewed, a set of severity criteria was developed, against which the case reports in the database are screened and selected for intensive review. Since the reporting form includes check boxes for diagnoses/categories (along with definitions) of adverse reactions, as well as space to describe other clinically significant reactions, these criteria were selected by the committee in order to capture the most serious events listed on the form (Table 1). Events traditionally considered serious, such as those that lead to hospitalization or are fatal, were also selected. As the other usual criteria for severity (e.g. events leading to disability) are not coded as such in the database, they were not automatically retrievable.

Copies of each case report, which consists of photocopies of the completed VAAE reporting form and any additional documentation, such as discharge summaries, consultation or clinic notes, and laboratory results, are provided to committee members in advance of each meeting for their individual (independent) pre-meeting preparation. The copies are subject to strict confidentiality provisions — they are carefully stripped of all personal identifiers including the identities of the patient, the health provider/reporter (the person submitting the case report) and the health unit.

Case review discussions

During each committee meeting, all cases are reviewed in a systematic stepwise manner and categorized on a specially designed causality assessment form (see below). The completeness of each case report, which is the key for an adequate causality assessment, is highly variable as reporters may or may not add detailed comments on the reporting form. The most complete reports are those submitted through active surveillance, whose more detailed forms capture more of the data required for an adequate assessment. The least complete are reports submitted electronically, the situation for two provinces, where usually only a final diagnosis or coding is given. Unless extensive “comments” are also recorded in the computer record, causality cannot be assessed . When the available information is not sufficient to draw a conclusion and the committee considers that the case is interesting or important enough to require a second review, supplementary information is requested through the reporting provincial public health authority. The case may then be reviewed more thoroughly at the following meeting, and scored on a specially designed follow-up form. Follow-up information is readily available for cases reported through active surveillance. For other cases, obtaining follow-up information is more difficult.

Case assessment

An assessment method was developed to maximize the efficiency of the work of the committee following a review of the literature on causality assessment. The range of methods are described elsewhere (7), but the committee initially considered options including “global introspection”, algorithms with and without scoring methods, and more complex systems. Regarding which method to favour, the number of cases that were to be evaluated was an important consideration, as was the need for standardized treatment of each case in order to be able to evaluate groups of adverse events that may comprise a signal. Although the broad range of professional expertise among members of ACCA could help them to evaluate each adverse event by discussion, the committee developed a guided series of questions on a form, the causality assessment form, to optimize efficiency and to enhance the standardization of their work.

Causality assessment form

The causality assessment form was refined in the course of several meetings. Important features were incorporated to take into account the specificity of vaccine reactions and the context within which they occur (i.e. paediatrics, neurological disorders, normal versus abnormal immunity). Given the significant value of follow-up or additional information about cases, the form also includes some evaluation of the completeness of the report, the necessity for a second review, and whether the case might be informative for educational purposes. The form has seven sections, which are described below.

Section 1 relates to the reason for reporting and whether the committee agreed with both the diagnosis that was made and the statement of severity. The review may be halted at this level if the committee feels that there is an error of coding such that the reaction being described no longer meets the severity criteria for review, or if there are not enough data to carry out an adequate evaluation using the questions in Section 2. In addition, some events that meet the severity criteria but are known to be unrelated to immunization, e.g. sudden infant death syndrome (SIDS) or infantile spasms, will also be rejected from detailed review if the diagnosis is felt to be correct (i.e. for SIDS, an autopsy confirmation is sought). This is done to save time for more detailed review of other cases.

Section 2, i.e. Questions 2.1–2.9, takes the evaluators through several important factors that have to be considered in assessing the causality between the reported adverse event and the vaccination:

Question 2.1. Frequency of occurrence of the adverse event. There are four possible answers according to the incidence of the event: common (> 5%), intermediate (1–5%), rare (< 1%), and not previously reported (NPR). When an event is reported for the first time in relation to a vaccine it could be a new entity that deserves special attention (see Question 3.2). This question specifically relates to the reaction as an entity on its own, whether or not it is related to vaccination (see Questions 2.2 and 2.3).

Question 2.2. Similar events known to occur with other diseases. This question is included to ensure that other possible etiological factors for the condition reported are considered.

Question 2.3. Event is known to be related to this vaccine. This question is focused more specifically on the role of the vaccine in the development of the adverse event. This information can be gleaned from the literature, product labelling or post-marketing surveillance databases.

Questions 2.4 and 2.5 relate to the pathophysiological mechanism of production of the event in relation to vaccine exposure.

Question 2.4. Event is explainable by the biological properties of the vaccine — either what is known about the vaccine itself, or even the infectious agent from which the vaccine was derived (especially for live vaccines).

Question 2.5. Vaccine–event interval compatible with the event. For example, anaphylaxis occurs usually within minutes of exposure to the allergen. If a report of “vaccine-associated” anaphylaxis after 24 hours is made, the interval would be judged incompatible. On the other hand, if the vaccine recipient was exposed to a potential trigger for anaphylaxis (e.g. bee sting) at precisely the same time as the vaccine, and anaphylaxis occurred, the interval between vaccination and the adverse event would still be deemed to be “typical” as a response to this question. When the event is reported for the first time, it is possible to answer “not applicable”, if no hypothesis can be postulated. This response is also used if there is no biological plausibility (rendering the compatibility of an interval meaningless).

Question 2.6, as well as Questions 2.7, 2.8 and 2.9, are aimed at focusing attention on the other possible etiological factors for the adverse event described. For all these questions it is possible to answer “unknown” if the information is missing in the report.

Question 2.6. The patient had similar symptoms in the past. For example, a history of a similar reaction to vaccination would make the present reaction more likely to be related to the vaccine. On the other hand, a similar event unrelated to vaccination would increase the likelihood that the current episode occurred by coincidence.

Question 2.7. Concomitant or preceding drug therapy. Inclusion of this question ensures that the committee has considered whether drug therapy may have had an impact on the relationship between the adverse event and the vaccination.

Question 2.8. Concomitant or preceding condition. This refers to an antecedent condition immediately related to or preceding the immunization (distinct from Question 2.6). This question gives the opportunity to check a “relevant” box when the event reported is thought to be due to the vaccine in the context of the underlying condition. For example, an elderly person with chronic cardiac failure might develop symptoms of cardiac decompensation after influenza vaccination due to a vaccine-caused elevation in temperature or stress from a local reaction at the site of vaccination. The vaccine is therefore considered to have contributed to cardiac failure in this specific situation only. This box allows the consideration of interactions (both biological and statistical) between the underlying condition and the vaccine which may lead to an adverse event.

Question 2.9. Other contributing factors. Other factors in place that could have affected the occurrence of the adverse event in relation to the vaccination, such as inappropriate administration (sterile abscess resulting from subcutaneous administration of a product that should have been given intramuscularly).

Section 3 relates to the assessment of causality. The definitions used for the different classes of probability are the causality assessment criteria used by WHO (Table 2). This section also allows the recording of two other pieces of information, namely the recognition of a potentially new “event” and the need for follow-up of the event.

Question 3.2 is checked when the event has never been reported before in relation to a vaccine. Similar reports will subsequently be flagged for special attention by the committee.

Question 3.3 is checked when the committee feels that there is not enough information to reach a more definite conclusion.

Question 3.4 is checked when the committee thinks that the case may benefit from a second review, should additional information become available.

Section 4 permits a brief summary of the case to be written, including important elements of the discussion which contributed to the final assessment of causality. If the final assessment should be altered by other information not available to the committee, it is noted here. The recipient of feedback from the committee may then be in a position to obtain or consider the additional information and “modify” the assessment of causality based on that information.

Section 5 permits recommendations for improving immunization delivery or case-reporting procedures to be written. For example, recommendations to continue a vaccine series can be made if the committee feels that the reporter has incorrectly decided to defer further immunization. Alternatively, the committee can recommend that further immunizations be deferred when appropriate. Such recommendations are based solely on the details provided in the case report — the final decision still rests with the recipients of the case assessment feedback. The committee may also request the results of additional investigations or suggest investigations that would be useful.

Section 6 considers whether the case could be useful for educational purposes. Here, education refers to information that can be widely disseminated (e.g. through publication of the reports) rather than specific “education” for the reporter of the case. This function might take the form of reporting specific unusual adverse events, a series of illustrative cases, or pointing out the misapplication of contraindications to immunization.

Section 7 considers whether the case could be useful for publication. This is similar to Section 6, but is a stronger message in favour of considering this case (and possibly other similar cases) for immediate publication.

Feedback

Copies of the case report and assessment form (including the occasional request for follow-up information) are returned to the responsible authorities in each province or territory. Each mailing is accompanied by a statement outlining the limitations of the review process, to ensure that the feedback is not misinterpreted or misrepresented (see Annex). The statement also serves as a quick reference regarding the methods used by the committee. The ACCA assessment form is the only verbatim information returned with the case report, because the committee is not responsible for passing judgement on any particular case. Of greatest utility to the recipient of the feedback are the sections describing the findings of the ACCA review, including any caveats to the committee’s findings and the summary of recommendations (if any).

Table 3 shows the distribution of causality assessment results after 4 years of ACCA review. Only about 18% of serious reactions were deemed “related” or “probably related” to vaccination, while 24.7% were felt to be unrelated. However, almost 25.8% of cases contained insufficient information for a proper causality assessment.

Conclusions

The evaluation of individual adverse event reports — to determine the likelihood that a vaccination was responsible for the event — is an integral part of the continuous monitoring of vaccine safety. Unlike the monitoring and evaluation of less severe and more common adverse events, which are less problematic, the assessment of serious adverse reactions is vital. As pointed out elsewhere (8), not only is the monitoring of serious adverse reactions a fundamental responsibility of public health authorities who distribute vaccines, but it is also critical to protect the reputation of vaccines that are increasingly under attack. Although some of the concerns have good intentions and serve as appropriate stimuli for vigilance by all those responsible for vaccination programmes, other attacks can be potentially destructive. False and misleading allegations appear increasingly in the press, in bookstores and on the Internet. Preliminary and unconfirmed research which calls into question the safety of vaccines is occasionally reported in the lay press and the scientific literature. Subsequent contradictory findings and failures to replicate this research are not always successful in reversing the negative messages left by the initial reports. One of the cornerstones of all such allegations and concerns is the confusion over causal versus temporal associations with vaccination. As is evident from the work of ACCA, many reports of severe reactions are either unrelated to the vaccination or have insufficient data to allow a proper assessment. To claim that all reported adverse events are caused by vaccination is misleading and erroneous, but the opponents of vaccination often still refuse to accept this.

Only by reviewing all individual case reports using a structured methodology to determine causality, and using these data to initiate appropriate follow-up action, can vaccine safety be properly monitored. Through these analyses, new or serious signals can be detected, and parents and practitioners who immunize can be reassured that they are doing the right thing. The use of standardized causality assessment by an expert multidisciplinary group is a step in that direction. Much work remains to be done to improve the quality of the data available for the assessments, but education and feedback to providers, as well as the lessons learned from the active surveillance network, should slowly improve the situation.

References

1. Rossi AC et al. The importance of adverse reaction reporting by physicians. Journal of the American Medical Association, 1988, 259 (8): 1203–1204.         

2. Pless RP. Vaccine-associated adverse events: surveillance and reporting. In: Krogh C, ed. Compendium of pharmaceuticals and specialities, 33rd edition. Ottawa, Canadian Pharmaceutical Association, 1998.         

3. Bentsi-Enchill A et al. Adverse events temporally associated with vaccines: 1992 report. Canada Communicable Disease Report, 1995, 21 (13): 117–128.         

4. Morris R et al. IMPACT monitoring network: a better mousetrap. Canadian Journal of Infectious Diseases, 1993, 4: 194–195.         

5. Duclos P et al. Reporting vaccine-associated adverse events. Canadian Family Physician, 1997, 43: 1551–1560.         

6. Furesz J, Contreras G. Vaccine-related mumps meningitis – Canada. Canadian Diseases Weekly Report, 1990, 16 (50): 253–254.         

7. Causality assessment of individual ADE’s. In: Avery W. Post-marketing drug safety management. International Journal of Risk and Safety in Medicine, 1994, 5: 125–144.         

8. Schiefele D. Point, counterpoint. Canadian Medical Association Journal, 1997, 157: 1705–1706.         

Correspondence
J.-P. Collet
Centre for Clinical Epidemiology and Community Studies, The Sir Mortimer B. Davis — Jewish General Hospital
3755 avenue Côte Ste-Catherine, suite A-132, Montreal, Québec H3T 1E2, Canada
E-mail: jpcollet@epid.jgh.mcgill.ca

¹ Members: Dr Neil Cashman (Chair); Dr Jean-Paul Collet, Dr Noni MacDonald; Dr Scott Halperin; Dr Monique Landry; Dr Laszlo Palkonyay; Dr Philippe Duclos; Dr Robert Pless; Dr Gina Mootrey; Dr Brian Ward; Dr Nicole LeSaux; and Dr Vito Caserta.

Annex

The role of an ‘‘Advisory Committee on Causality Assessment’’ (ACCA) in vaccine safety

Clarification statement for case review reports

Cases of adverse events related to immunization are reported by public health nurses and physicians as suspicions only — a definite causal link need not be established. It is by evaluating these cases on an aggregate, national level that “signals” of concern can be generated. In some cases where the adverse event is an expected one, the causal association with a vaccine can be easily determined and accepted. In other cases, even with known events, the causal association may be less obvious. Other events that are temporally associated with receipt of a vaccine may never have been reported before, and are therefore unexpected. In those situations, it may be very difficult to ascertain whether the vaccine was in fact responsible in some way, and the opinion of a group of experts to discuss the case in more detail can be very useful in evaluating whether the reported event is significant in terms of vaccine safety. It is crucial to be aware that ACCA’s review of a case is only based on the material presented. Any assessment reached by ACCA must necessarily be taken in this context and interpreted in the light of any other information which the reporting physician or public health authority is aware of, but which may not have been communicated to the committee. ACCA’s role is to review, in a systematic fashion, a pre-selected series of cases of adverse events related to immunization which have been reported to the Division of Immunization, Bureau of Infectious Diseases at the Laboratory Centre for Disease Control. The Division of Immunization has the mandate to undertake the postmarketing surveillance of all vaccine products in Canada and works closely with the Vaccine Division, Bureau of Biologics and Radiopharmaceuticals of the Therapeutic Products Directorate.

The cases are selected automatically from among all reported events according to severity criteria, and stripped of all identifiers (patient’s identity number, provider and health unit) except for the province of origin. Using these case reviews aggregated from across the country, and assessed using a standard causality algorithm which is applied uniformly to all cases, ACCA monitors the safety of vaccines on the market. On occasion, there may be information within individual cases that suggests a need for an educational message to health care providers regarding the use of vaccines or the diagnosis of an adverse event (such as when it may appear that contraindications are being misapplied or adverse events are being systematically misdiagnosed). Periodic publication of the aggregated results of these reviews and collection of educational messages serve to point out the issues of importance. However, neither ACCA nor the Division of Immunization have any mandate to provide feedback directly to the health unit or health care provider who reported the case. First, because all case reports are forwarded through public health authorities at the provincial or territorial ministries of health. And second, this mechanism is vital in order to protect confidentiality of the case reports, to maintain objectivity, and to make use of the results of causality assessment solely for the purpose of public health immunization programme monitoring and overall health professional education.

Similarly, ACCA does not review individual cases that do not have implications for the overall safety of vaccines in Canada. For example, cases of sudden infant death syndrome (SIDS) or infantile spasms, which have been demonstrated by epidemiological research to be unrelated to vaccination, are not assessed unless the diagnosis is in question. Also, special cases that the public health authorities would like ACCA to review on an individual basis are similarly not part of ACCA’s mandate. However, because of the systematic nature with which the more serious cases are selected, the Division of Immunization flags these cases and groups them for feedback to the reporting province or territory. Committee members, however, are blinded as to their “special status”. Despite accepting these special requests, ACCA is not a “tribunal” for judging individual cases. Feedback to the provincial or territorial public health authorities is provided as a courtesy. Should patients or parents wish an in-depth review of what is suspected as an adverse event following immunization, they should seek a medical opinion from their local physicians.