ABSTRACT

Objectives.

To estimate the prevalence of missed opportunities for vaccination (MOV) in Latin America and the effect of interventions targeting health systems, health workers, patients, and communities on MOV.

Methods.

Searches were conducted in MEDLINE, EMBASE, CINAHL, and LILACS electronic databases and relevant organizations were contacted, including the Pan American Health Organization (PAHO), to identify studies meeting eligibility criteria. A pair of reviewers identified 27 randomized and non-randomized studies quantifying the effectiveness of any intervention for reducing MOV and 5 studies assessing the rate of MOV in Latin America. Results are reported narratively when criteria to pool results were not met, and the certainty of this evidence was assessed using the GRADE approach.

Results.

Evidence suggests the rate of MOV in Latin America ranged from 5% to 37% with a pooled estimate of 17% (95% CI [9, 32]) (low certainty) and that monetary incentives to healthcare teams, training for healthcare teams on how to communicate with patients, and educational interventions for caregivers probably reduce MOV (moderate to very low certainty).

Conclusions.

There is insufficient evidence supporting the implementation of any intervention as policy based only on the potential reduction of MOV without considering several factors, including costs, feasibility, acceptability, and equity.

Keywords
Vaccination; vaccination coverage; immunization; Latin America

RESUMEN

Objetivos.

Estimar la prevalencia de las oportunidades perdidas de vacunación en América Latina y el efecto de las intervenciones dirigidas a los sistemas de salud, los trabajadores de salud, los pacientes y las comunidades.

Métodos.

Se realizaron búsquedas en las bases de datos electrónicas MEDLINE, EMBASE, CINAHL y LILACS y se estableció contacto con las organizaciones pertinentes, incluida la Organización Panamericana de la Salud (OPS), para identificar aquellos estudios que cumplieran con los criterios de admisibilidad. Un par de revisores identificaron 27 estudios aleatorizados y no aleatorizados que cuantificaban la efectividad de cualquier intervención para reducir las oportunidades perdidas de vacunación, así como 5 estudios que evaluaban la tasa de oportunidades perdidas de vacunación en América Latina. Cuando no cumplían con los criterios para agrupar los resultados, estos se presentan de manera narrativa; para evaluar la certeza de esta evidencia se utilizó el método GRADE.

Resultados.

La evidencia indica que la tasa de oportunidades perdidas de vacunación en América Latina osciló entre 5% y 37% y presentó una estimación consolidada de 17% (IC del 95% [9, 32]) (certeza baja), y que los incentivos monetarios a los equipos de atención médica, la capacitación de los equipos de salud sobre cómo comunicarse con los pacientes y las intervenciones educativas destinadas a los cuidadores probablemente reducen las oportunidades perdidas de vacunación (certeza moderada a muy baja).

Conclusiones.

No hay suficiente evidencia para respaldar la aplicación de alguna intervención como política basándose únicamente en la reducción potencial de las oportunidades perdidas de vacunación si no se tienen en cuenta varios factores, como los costos, la viabilidad, la aceptabilidad y la equidad.

Palabras clave
Vacunación; cobertura de vacunación; inmunización; América Latina

RESUMO

Objetivos.

Estimar a prevalência de oportunidades perdidas de vacinação (OPV) na América Latina e o efeito de intervenções para reduzir as OPV direcionadas aos sistemas de saúde, profissionais de saúde, pacientes e comunidades.

Métodos.

Foi realizada a pesquisa em bancos de dados eletrônicos (MEDLINE, Embase, CINAHL e LILACS) e mediante contato com instituições relevantes, como a Organização Pan-Americana da Saúde (OPAS), com o objetivo de identificar estudos que satisfizessem os critérios de inclusão. Dois revisores identificaram 27 estudos randomizados e não randomizados com avaliação quantitativa da efetividade de intervenções para reduzir as OPV e 5 estudos que avaliavam a taxa de OPV na América Latina. Os resultados foram apresentados de forma descritiva quando não preenchiam os critérios para apresentação conjunta. O sistema GRADE foi usado para avaliar a qualidade das evidências.

Resultados.

As evidências indicam que a taxa de OPV na América Latina variou entre 5% e 37%, com uma estimativa conjunta de 17% (IC 95% [9, 32]) (qualidade da evidência: baixa). Incentivos financeiros e capacitação em comunicação com os pacientes para as equipes de saúde, bem como intervenções educacionais para os cuidadores, provavelmente reduzem as OPV (qualidade da evidência: moderada a muito baixa).

Conclusões.

Não há evidências suficientes para respaldar implementar qualquer intervenção como política com base somente na possível redução das OPV, sem levar em consideração diversos fatores como custos, viabilidade, aceitabilidade e equidade.

Palavras-chave
Vacinação; cobertura vacinal; imunização; América Latina

Immunization is a process by which a person becomes immune or resistant to an infectious disease (¹1. World Health Organization [Internet]. Geneva: WHO; 2021. Vaccines and immunization. Available from: http://www.who.int/topics/immunization/en/.
http://www.who.int/topics/immunization/e... ). Vaccinations are a well-studied tool for achieving immunization and avert approximately 3 million deaths per year globally. For example, vaccination programs, like the Expanded Program on Immunization (EPI), prevented approximately 174 000 deaths of children between 2006 and 2011 (²2. World Health Organization [Internet]. Geneva: WHO; 2021. Immunization coverage 2014. Available from: http://www.who.int/mediacentre/factsheets/fs378/en/.
http://www.who.int/mediacentre/factsheet... , ³3. Llau A. National vaccine coverage trends and funding in Latin America and the Caribbean. Vaccine 2021;39:317–23.). Strong evidence suggests vaccination prevents morbidity and mortality from diseases such as diphtheria, measles, pertussis, pneumonia, polio, rotavirus diarrhea, rubella, and tetanus, as well as cancers such as liver and cervical cancer (⁴4. Andre FE, Booy R, Bock HL, Clemens J, Datta SK, John TJ, et al. Vaccination greatly reduces disease, disability, death and inequity worldwide. Bull World Health Organ. 2008;86(2):140–6.). Additionally, vaccines are often safer than therapeutic measures and can eradicate diseases in specific communities if up to 95% of the population achieves complete immunization (⁴4. Andre FE, Booy R, Bock HL, Clemens J, Datta SK, John TJ, et al. Vaccination greatly reduces disease, disability, death and inequity worldwide. Bull World Health Organ. 2008;86(2):140–6.). For example, the Region of the Americas (the Region) has eliminated smallpox (in 1971), poliomyelitis (1994), rubella and congenital rubella syndrome (2015), and measles (2016) through immunization (⁵5. Peltola H, Davidkin I, Paunio M, Valle M, Leinikki P, Heinonen OP. Mumps and rubella eliminated from Finland. JAMA. 2000; 284(20):2643–7., ⁶6. Pan American Health Organization [Internet]. Washington, DC: PAHO; 2016. Region of the Americas is declared free of measles. Available from: https://www3.paho.org/hq/index.php?option=com_content&view=article&id=12528:region-americas-declared- free-measles&Itemid=1926&lang=en
https://www3.paho.org/hq/index.php?optio... ). Immunization can also contribute to considerable cost-savings for health systems (⁴4. Andre FE, Booy R, Bock HL, Clemens J, Datta SK, John TJ, et al. Vaccination greatly reduces disease, disability, death and inequity worldwide. Bull World Health Organ. 2008;86(2):140–6.). Despite this, many children and adults are still affected by vaccine-preventable diseases. Estimates from the World Health Organization (WHO) suggest that approximately 6.9 million children died from vaccine-preventable diseases in 2011, while approximately 22.6 million infants were not provided essential vaccines (²2. World Health Organization [Internet]. Geneva: WHO; 2021. Immunization coverage 2014. Available from: http://www.who.int/mediacentre/factsheets/fs378/en/.
http://www.who.int/mediacentre/factsheet... ).

Immunization schedules and vaccine coverage vary nationally and internationally. For example, coverage in Latin America and the Caribbean has considerably decreased since 2013, less than half of countries in the Region reached the WHO 90% coverage target in 2017, and coverage continues to vary dramatically between countries in the Region (³3. Llau A. National vaccine coverage trends and funding in Latin America and the Caribbean. Vaccine 2021;39:317–23.). Due to the COVID-19 pandemic, recent declines in vaccine coverage have the potential to erode historic and hard-fought gains in the Region. A comparison of the number of children receiving diphtheria–tetanus–pertussis (DTP3) and measles, mumps, and rubella (MMR) vaccinations from January to June of 2019 and 2020, respectively, indicates that vaccination coverage decreased by 33% for DPT3 and by 24% for MMR in the Region (⁷7. Pan American Health Organization. Annual Report of the Director of the Pan American Sanitary Bureau 2020. Saving Lives and Improving Health and Well-Being. Washington, DC: PAHO; 2020. Available from: https://iris.paho.org/handle/10665.2/52852
https://iris.paho.org/handle/10665.2/528... ). A number of challenges impede vaccination coverage in the Region, including hard-to-reach communities, political instability, unfavorable socioeconomic factors, religious/cultural beliefs, vaccine hesitancy, limited resources, poor management of health systems, and inadequate monitoring and supervision (²2. World Health Organization [Internet]. Geneva: WHO; 2021. Immunization coverage 2014. Available from: http://www.who.int/mediacentre/factsheets/fs378/en/.
http://www.who.int/mediacentre/factsheet... , ⁸8. MacIntyre CR, Leask J. Immunization myths and realities: responding to arguments against immunization. J Paediatr Child Health. 2003;39(7):487–91.–¹⁰10. Pan American Health Organization [Internet]. Washington, DC: PAHO; 2021. Immunization. Available from: https://www.paho.org/en/topics/immunization
https://www.paho.org/en/topics/immunizat... ).

WHO recommends that immunization should be offered to all eligible individuals at every contact point with health services (³3. Llau A. National vaccine coverage trends and funding in Latin America and the Caribbean. Vaccine 2021;39:317–23., ¹¹11. Hutchins SS, Jansen HA, Robertson SE, Evans P, Kim-Farley RJ. Studies of missed opportunities for immunization in developing and industrialized countries. Bull World Health Organ. 1993;71(5): 549–60.). When it is not offered, this is referred to as a “missed opportunity for vaccination” (MOV). An earlier review identified four main reasons for missed opportunities: 1) failure to simultaneously administer all vaccines for an eligible child; 2) false contraindications to immunization; 3) health worker practices; and 4) logistical problems (¹¹11. Hutchins SS, Jansen HA, Robertson SE, Evans P, Kim-Farley RJ. Studies of missed opportunities for immunization in developing and industrialized countries. Bull World Health Organ. 1993;71(5): 549–60.). Though patients may seek immunization services, healthcare systems may not accommodate these requests due to lack of an appointment, immunization not being the primary reason for the visit, and despite there being no health-related contraindications. Additionally, patients may seek care in health facilities where immunization is simply not offered (¹²12. Orenstein WA, Cutts F, Gindler JS, Hutchings S, Bernier R, Dini E, editors. Opportunities to vaccinate. Annual Immunization Conference; 1991; Washington, D.C.).

Research has identified several potential approaches (e.g., one-on-one coaching with caregivers) to reduce MOVs (¹³13. Jaca A, Mathebula L, Iweze A, Pienaar E, Wiysonge CS. A systematic review of strategies for reducing missed opportunities for vaccination. Vaccine. 2018;36(21):2921–7.). Interventions aimed at reducing MOVs are likely to increase vaccination coverage and therefore magnify all beneficial downstream consequences associated with immunization.

Preexisting reviews have identified the main reasons for low vaccine coverage and MOVs (¹¹11. Hutchins SS, Jansen HA, Robertson SE, Evans P, Kim-Farley RJ. Studies of missed opportunities for immunization in developing and industrialized countries. Bull World Health Organ. 1993;71(5): 549–60., ¹⁴14. Rainey JJ, Watkins M, Ryman TK, Sandhu P, Bo A, Banerjee K. Reasons related to non-vaccination and under-vaccination of children in low and middle income countries: findings from a systematic review of the published literature, 1999-2009. Vaccine. 2011;29(46):8215–21., ¹⁵15. Favin M, Steinglass R, Fields R, Banerjee K, Sawhney M. Why children are not vaccinated: a review of the grey literature. Int Health. 2012;4(4):229–38.). Additionally, an earlier systematic review included evidence from randomized controlled trials (RCTs) to measure the effect of interventions on MOVs (¹³13. Jaca A, Mathebula L, Iweze A, Pienaar E, Wiysonge CS. A systematic review of strategies for reducing missed opportunities for vaccination. Vaccine. 2018;36(21):2921–7.). Although RCTs are the ideal study design for comparing effects of interventions, utilizing evidence from non-randomized studies can provide a comprehensive summary of all evidence on benefits and harms (¹⁶16. Schunemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2019;111:105–14.). Additionally, previously published reviews have not adequately assessed the certainty of, or confidence in, included evidence.

Regional decisionmakers have called for a comparison of interventions for reducing MOVs (Objective 1) and a summary of evidence (published by national and international organizations) on the rate, or prevalence, of MOVs in Latin America (Objective 2) to comprehensively inform regional decisionmakers on the magnitude of MOVs and potential strategies for reducing these.

MATERIALS AND METHODS

This report is primarily an update of another review—an internal, unpublished systematic review for the Pan American Health Organization (PAHO)—developed in response to a call for evidence on interventions to reduce MOVs. In this update, we included randomized and non-randomized studies on interventions for reducing MOVs (Objective 1). For this current review, we also summarized results from studies measuring the rate of MOVs in Latin America (Objective 2). Reporting of this systematic review was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement and conducted according to a protocol approved by PAHO (available upon request) (¹⁷17. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006–12.).

Study eligibility

Interventions for reducing MOVs (Objective 1).

We included studies on interventions for reducing MOVs for immunizations recommended by WHO with a particular focus on vaccination of the following populations: children under 5 years of age, women of childbearing age, and older adults. We did not include immunizations for human papillomavirus (HPV) due to inconsistent inclusion of HPV vaccination in immunization schedules within the Region (¹⁸18. Nogueira-Rodrigues A. HPV Vaccination in Latin America: Global Challenges and Feasible Solutions. Am Soc Clin Oncol Educ Book. 2019;39:e45–e52.).

We included RCTs, quasi-randomized controlled trials (CCTs), controlled before-and-after studies (CBAs), and interrupted time series (ITS) quantifying the effect of any intervention (of any duration and in any setting) used for reducing MOVs. We included studies in any language published after 1990. We categorized the interventions as follows:

• Interventions targeting the health system including changes in organization of care within a particular healthcare setting;

• Interventions targeting health workers including hospital administrators, clinicians, nurses, and any other individuals involved in the immunization process;

• Interventions targeting patients or community members including children, adults, caregivers or parents, women of childbearing age, and older adults;

• Mixed interventions targeting one or more of the above categories.

We included studies that compared interventions to another intervention or no intervention, or where the same intervention was monitored over time.

Outcomes of interest.

We summarized the following outcomes from included studies:

MOVs defined as any healthcare visit that did not result in participants of any age receiving the needed vaccine (¹¹11. Hutchins SS, Jansen HA, Robertson SE, Evans P, Kim-Farley RJ. Studies of missed opportunities for immunization in developing and industrialized countries. Bull World Health Organ. 1993;71(5): 549–60.). MOVs were further classified as:

1. Attributable to the healthcare system;

2. Attributable to the knowledge, attitudes, and/or decisions of individual patients, caregivers, etc.;

3. Attributable to the knowledge and/or attitudes of healthcare personnel.

Rate of MOVs in Latin America (Objective 2).

We included any report or study on the rate, or prevalence, of MOVs in Latin America that followed PAHO methodology (¹⁹19. Organización Panamericana de la Salud. Metodología para la evaluación de oportunidades perdidas de vacunación. Washington, DC: OPS; 2014.).

Search strategy

Interventions for reducing MOVs (Objective 1).

We searched the MEDLINE, EMBASE, CINAHL, and LILACS electronic databases. The original systematic review conducted a search between 1990 and April 2014; we updated the original search strategy, with minor modifications, and ran the search from April 2014 to November 2019. A combination of MeSH and keyword terms on the topics of immunization and missed opportunities for vaccination were used (full search strategy available upon request).

Rate of MOVs in Latin America (Objective 2).

We included all studies developed by the Comprehensive Family Immunization Unit of the Family, Health Promotion and Life Course Department of PAHO, which were retrieved by one author (MV), an advisor in the department.

Gray literature.

Relevant researchers and organizations (WHO, PAHO, UNICEF) were contacted to retrieve any additional material for both Objective 1 and 2.

Reference lists.

We reviewed reference lists of relevant citations to retrieve any additional study meeting our eligibility criteria for Objectives 1 and 2.

Data collection and analysis (Objective 1)

Selection of studies.

We used a pretested form to select studies meeting eligibility criteria. After removing duplicate results, we screened titles/abstracts and full-text articles (authors MT and RB) in succession. We consulted a third author (ACL) to resolve disagreements.

Data abstraction and management.

Data abstraction was conducted independently and in duplicate by authors (MT, RB, ACL) using a pretested form. We abstracted study characteristics (author, country, journal of publication, year of publication) in addition to data on included populations, interventions, comparisons, and outcomes of interest. We contacted corresponding authors for clarification when needed.

Risk of bias assessment.

We (MT, RB) independently and in duplicate assessed risk of bias for which we used the Cochrane Risk of Bias tool for randomized clinical trials, and suggested risk of bias criteria for non-randomized studies by the Cochrane Effective Practice and Organization of Care group (²⁰20. Cochrane Effective Practice and Organisation of Care. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2013. Available from: http://epocoslo.cochrane.org/epoc-specific-resources-review-authors
http://epocoslo.cochrane.org/epoc-specif... ).

Data synthesis.

When criteria for pooling results were met (the population, intervention, and comparator were judged to be similar enough across studies), we conducted meta-analysis in Review Manager 5.3 to calculate summary estimates. We used a frequentist approach and random-effects models to account for potential, unexplained heterogeneity between studies. We used the participants or clinic visits as the unit of analysis. When it was not possible to conduct meta-analysis (e.g., important clinical heterogeneity precluded us from conducting a meta-analysis), we reported results narratively. We also accounted for differences in study design and synthesized data separately for RCTs and non-randomized studies. For each comparison, we reported any pooled estimates at an outcome level as risk ratios (RR) or odds ratios (OR) in Grading of Recommendations Assessment, Development and Evaluation (GRADE) Summary of Findings Tables (²¹21. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.). We used the event rate of the control groups to estimate the baseline risk of associated outcomes and obtain absolute estimates of effect.

Assessment of certainty of the evidence.

The certainty of the evidence can be rated as high, moderate, low, or very low and is influenced by study design and other domains that can decrease (issues of risk of bias, inconsistency, imprecision, indirectness, and publication bias) or increase the certainty (large magnitude of effect, dose-response gradient, and residual confounding acting opposite to what was observed). We assessed the certainty of the evidence at an outcome level using the using the GRADE approach (²²22. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6.).

Handling of missing data and other analyses.

We planned to contact study authors to obtain missing outcome data if such missing data were judged to affect the certainty of the evidence. We did not conduct any subgroup or sensitivity analyses.

Data collection and analysis (Objective 2)

Data abstraction and management.

We abstracted data from selected studies using a pretested data abstraction form. We collected information on the country, target population, data collection period, number of included participants and healthcare centers, number of participants eligible for vaccination, number of MOVs, and reasons for MOVs. For studies including child populations, we collected information about caregivers and any other person(s) responsible for making vaccination-related decisions.

Data analysis.

When criteria for pooling were not met, we summarized data narratively. When studies included/used similar populations and methodology, we calculated an overall rate of MOVs by pooling the proportion of MOVs across studies using a generalized linear mixed model, implemented using the metaprop package in the software R (²³23. R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2016., ²⁴24. Viechtbauer W. Conducting Meta-Analyses in R with the metafor Package. J Stat Softw. 2010;36(3):1–48.). We assessed the certainty of the evidence using the same methodology detailed above for Objective 1.

RESULTS

How to interpret the results

The GRADE certainty of the evidence reflects how certain we are that the true effect lies within a particular range of results. For example, low certainty evidence implies that the results may be importantly different to what was observed (²¹21. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.). In addition to the certainty of the evidence, interpreting results requires, for example, making a judgment of the importance of the magnitude of reduction in MOVs observed. For example, an absolute risk reduction of 3% of MOVs when providing training to the healthcare team on how to communicate with patients (compared with standard of care) may be too small to be considered important and worth the efforts necessary to provide such training. Finally, it is important to consider that, although important, MOVs are a surrogate for outcomes that are important to patients, such as prevention of diseases. No included studies measured patient-important outcomes.

Results (Objective 1)

The original electronic search retrieved 469 citations, and the contacted organizations suggested 30 papers. We screened all associated titles and abstracts, and 49 articles in full text. The updated search identified 350 new citations, of which 42 were ultimately full-text screened. A total of 27 studies from both searches met eligibility criteria, 6 of which are from the updated search (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91.–⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.). The results of the search and screening process are detailed in a PRISMA flow diagram (Figure 1).

Description of included studies

We included 15 randomized controlled trials (²⁶26. Daley MF, Steiner JF, Kempe A, Beaty BL, Pearson KA, Jones JS, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr. 2004;4(3):217–23., ²⁸28. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health. 1999;89(2):171–5., ³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7., ³⁵35. Irigoyen MM, Findlay S, Wang D, Chen S, Chimkin F, Pena O et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr. 2006;6(2):100–4.–³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7., ⁴³43. Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999;103(1):31–8., ⁴⁵45. Shaw JS, Samuels RC, Larusso EM, Bernstein HH, Rodewald LE, Szilagyi PG, et al. Impact of an encounter-based prompting system on resident vaccine administration performance and immunization knowledge. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 2000;105(4 Pt 2):978–83., ⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200., ⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061., ⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.), 2 non-randomized controlled trials (⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73., ⁵⁰50. Wise KA, Sebastian SJ, Haas-Gehres AC, Moore-Clingenpeel MD, Lamberjack KE. Pharmacist impact on pediatric vaccination errors and missed opportunities in the setting of clinical decision support. J Am Pharm Assoc. 2017;57(3):356–61.), 8 before-and-after studies (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91., ²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35., ³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62., ³⁴34. Hicks P, Tarr GAM, Hicks XP. Reminder cards and immunization rates among Latinos and the rural poor in Northeast Colorado. J Am Board Fam Med. 2007;20(6):581–6., ⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92., ⁴²42. Minkovitz CS, Belote AD, Higman SM, Serwint JR, Weiner JP. Effectiveness of a practice-based intervention to increase vaccination rates and reduce missed opportunities. Arch Pediatr Adolesc Med. 2001;155(3):382–6., ⁴⁴44. Sabnis SS, Pomeranz AJ, Amateau MM. The effect of education, feedback, and provider prompts on the rate of missed vaccine opportunities in a community health center. Clin Pediatr (Phila). 2003;42(2):147–51., ⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5.), and 1 interrupted time series (³⁰30. Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics. 2007;120(4):707–14.), all published between 1992 and 2019 (study characteristics table available upon request).

Of these studies, 20 were conducted in the United States of America (²⁶26. Daley MF, Steiner JF, Kempe A, Beaty BL, Pearson KA, Jones JS, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr. 2004;4(3):217–23., ²⁸28. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health. 1999;89(2):171–5.–³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62.–³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7.–⁴⁵45. Shaw JS, Samuels RC, Larusso EM, Bernstein HH, Rodewald LE, Szilagyi PG, et al. Impact of an encounter-based prompting system on resident vaccine administration performance and immunization knowledge. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 2000;105(4 Pt 2):978–83., ⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73.–⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.), 3 in Australia (³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40., ⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92., ⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5.), 2 in India (²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7.), 1 in Nigeria (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91.), and 1 in Sudan (³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9.). No studies were conducted in multiple countries. Fourteen studies were based in a primary care setting (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91., ²⁶26. Daley MF, Steiner JF, Kempe A, Beaty BL, Pearson KA, Jones JS, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr. 2004;4(3):217–23., ²⁸28. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health. 1999;89(2):171–5., ²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35., ³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62., ³⁴34. Hicks P, Tarr GAM, Hicks XP. Reminder cards and immunization rates among Latinos and the rural poor in Northeast Colorado. J Am Board Fam Med. 2007;20(6):581–6., ³⁶36. Kempe A, Lowery NE, Pearson KA, Renfrew BL, Jones JS, Steiner JF, et al. Immunization recall: effectiveness and barriers to success in an urban teaching clinic. J Pediatr. 2001;139(5):630–5., ³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623., ⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92., ⁴³43. Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999;103(1):31–8., ⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73., ⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061., ⁵⁰50. Wise KA, Sebastian SJ, Haas-Gehres AC, Moore-Clingenpeel MD, Lamberjack KE. Pharmacist impact on pediatric vaccination errors and missed opportunities in the setting of clinical decision support. J Am Pharm Assoc. 2017;57(3):356–61.), 8 in a hospital setting (²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ³⁰30. Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics. 2007;120(4):707–14., ³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9., ³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40., ⁴²42. Minkovitz CS, Belote AD, Higman SM, Serwint JR, Weiner JP. Effectiveness of a practice-based intervention to increase vaccination rates and reduce missed opportunities. Arch Pediatr Adolesc Med. 2001;155(3):382–6., 44–46), 1 in a private clinic (³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7.), and the 3 in a community setting (³⁵35. Irigoyen MM, Findlay S, Wang D, Chen S, Chimkin F, Pena O et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr. 2006;6(2):100–4., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7., ⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.). One study was based in both a primary care setting and hospital setting (⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200.).

One study included older adults (³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40.), 1 study children and adolescents with asthma (³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69.), and 1 adolescents (³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62.). Of the remaining studies, 21 included infants or children (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91.–²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35., ³⁰30. Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics. 2007;120(4):707–14., ³⁴34. Hicks P, Tarr GAM, Hicks XP. Reminder cards and immunization rates among Latinos and the rural poor in Northeast Colorado. J Am Board Fam Med. 2007;20(6):581–6.–³⁶36. Kempe A, Lowery NE, Pearson KA, Renfrew BL, Jones JS, Steiner JF, et al. Immunization recall: effectiveness and barriers to success in an urban teaching clinic. J Pediatr. 2001;139(5):630–5., ³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7.–⁴³43. Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999;103(1):31–8., ⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5., ⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73., ⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061.–⁵⁵55. Quiroz N, Jimeno J, Sales V, Alvarez S, Fossatti L. Informe final: Evaluación de oportunidades perdidas de vacunación en la República de Panamá. [Report acquired through the Pan American Health Organization Comprehensive Family Immunization Unit.] Panama City: Vax Trials; 2014.), while 3 studies included adults (³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7., ³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623., ⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92.), with 1 specifically including pregnant women (³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7.).

Due to the complexity of interventions intended to reduce MOVs, many of the studies reported that the unit of analysis differed from the unit of randomization. Most often researchers randomized practices or physicians and collected and analyzed data at the level of participants or visits. In 10 studies, the unit of intervention was the caregiver or parent of the child (²⁶26. Daley MF, Steiner JF, Kempe A, Beaty BL, Pearson KA, Jones JS, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr. 2004;4(3):217–23., ³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³⁵35. Irigoyen MM, Findlay S, Wang D, Chen S, Chimkin F, Pena O et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr. 2006;6(2):100–4., ³⁶36. Kempe A, Lowery NE, Pearson KA, Renfrew BL, Jones JS, Steiner JF, et al. Immunization recall: effectiveness and barriers to success in an urban teaching clinic. J Pediatr. 2001;139(5):630–5., ³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7., ⁴³43. Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999;103(1):31–8., ⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5., ⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73., ⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.). Fourteen targeted health workers (²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7.–³⁰30. Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics. 2007;120(4):707–14., ³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7., ³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623., ³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7., ⁴⁴44. Sabnis SS, Pomeranz AJ, Amateau MM. The effect of education, feedback, and provider prompts on the rate of missed vaccine opportunities in a community health center. Clin Pediatr (Phila). 2003;42(2):147–51.–⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5., ⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061., ⁵⁰50. Wise KA, Sebastian SJ, Haas-Gehres AC, Moore-Clingenpeel MD, Lamberjack KE. Pharmacist impact on pediatric vaccination errors and missed opportunities in the setting of clinical decision support. J Am Pharm Assoc. 2017;57(3):356–61., ⁵⁴54. Perú, Ministerio de Salud; Organización Panamericana de la Salud. Investigación de oportunidades perdidas para vacunación (OPPV) en Perú. [Report acquired through the Pan American Health Organization Comprehensive Family Immunization Unit.] Perú, Ministerio de Salud; 2014.), 2 both health workers and caregivers (³⁴34. Hicks P, Tarr GAM, Hicks XP. Reminder cards and immunization rates among Latinos and the rural poor in Northeast Colorado. J Am Board Fam Med. 2007;20(6):581–6., ⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200.), 1 targeted adult patients (⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92.), and 3 involved health system and organizational changes (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91., ³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62., ³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9.).

Most included reported estimates of effect associated with the following interventions: improvements in accessibility to vaccination site (³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9.); reminders to healthcare teams (³⁰30. Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics. 2007;120(4):707–14., ³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62., ³⁴34. Hicks P, Tarr GAM, Hicks XP. Reminder cards and immunization rates among Latinos and the rural poor in Northeast Colorado. J Am Board Fam Med. 2007;20(6):581–6., ⁴²42. Minkovitz CS, Belote AD, Higman SM, Serwint JR, Weiner JP. Effectiveness of a practice-based intervention to increase vaccination rates and reduce missed opportunities. Arch Pediatr Adolesc Med. 2001;155(3):382–6., ⁴⁵45. Shaw JS, Samuels RC, Larusso EM, Bernstein HH, Rodewald LE, Szilagyi PG, et al. Impact of an encounter-based prompting system on resident vaccine administration performance and immunization knowledge. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 2000;105(4 Pt 2):978–83., ⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200., ⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061.), patients and caregivers (²⁶26. Daley MF, Steiner JF, Kempe A, Beaty BL, Pearson KA, Jones JS, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr. 2004;4(3):217–23., ³⁵35. Irigoyen MM, Findlay S, Wang D, Chen S, Chimkin F, Pena O et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr. 2006;6(2):100–4., ³⁶36. Kempe A, Lowery NE, Pearson KA, Renfrew BL, Jones JS, Steiner JF, et al. Immunization recall: effectiveness and barriers to success in an urban teaching clinic. J Pediatr. 2001;139(5):630–5.); educational interventions for healthcare teams (²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ⁴⁴44. Sabnis SS, Pomeranz AJ, Amateau MM. The effect of education, feedback, and provider prompts on the rate of missed vaccine opportunities in a community health center. Clin Pediatr (Phila). 2003;42(2):147–51., ⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5., ⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061.), patients and caregivers (⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92., ⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73., ⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.); monetary incentives (²⁸28. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health. 1999;89(2):171–5.) and fines (⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7.), free vaccination programs (²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35.), exemption from caregiver’s authorization (⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200.), patient tracking (⁴³43. Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999;103(1):31–8.), provider training on communication with patients (³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623.), recommendations provided by clinicians (³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7.), quality improvement programs (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91.), and pharmacist presence (⁵⁰50. Wise KA, Sebastian SJ, Haas-Gehres AC, Moore-Clingenpeel MD, Lamberjack KE. Pharmacist impact on pediatric vaccination errors and missed opportunities in the setting of clinical decision support. J Am Pharm Assoc. 2017;57(3):356–61.).

There were significant variations in how the primary outcome of interest, reduction in MOVs, was measured. Some studies reported MOVs per patient per year (⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200.), while others reported MOVs for specific vaccines (⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7.), the number of visits at which MOVs occurred (⁴⁵45. Shaw JS, Samuels RC, Larusso EM, Bernstein HH, Rodewald LE, Szilagyi PG, et al. Impact of an encounter-based prompting system on resident vaccine administration performance and immunization knowledge. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 2000;105(4 Pt 2):978–83.), mean number of MOVs per infant (⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73.), captured opportunities (i.e., opposite of MOV) (³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69.), and/or opportunistic immunizations (i.e., opportunities that would have been missed) (⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5.).

Risk of bias assessments

Major issues of risk of bias were observed in 15 RCTs (risk of bias assessments available upon request); these included issues related to random sequence generation and allocation concealment. Eight RCTs were judged as having unclear or high risk of bias due to issues with random sequence generation (²⁸28. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health. 1999;89(2):171–5., ³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7., ³⁵35. Irigoyen MM, Findlay S, Wang D, Chen S, Chimkin F, Pena O et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr. 2006;6(2):100–4.–³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623., ³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7., ⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200., ⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061., ⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.), and 8 RCTs were judged to have unclear or high risk of bias due to issues with allocation concealment (²⁶26. Daley MF, Steiner JF, Kempe A, Beaty BL, Pearson KA, Jones JS, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr. 2004;4(3):217–23., ³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9., ³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40., ⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7., ⁴⁵45. Shaw JS, Samuels RC, Larusso EM, Bernstein HH, Rodewald LE, Szilagyi PG, et al. Impact of an encounter-based prompting system on resident vaccine administration performance and immunization knowledge. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 2000;105(4 Pt 2):978–83., ⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200., ⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.). Two non-randomized controlled trials (⁴⁷47. Stille CJ, Christison-Lagay J, Bernstein BA, Dworkin PH. A simple provider-based educational intervention to boost infant immunization rates: a controlled trial. Clin Pediatr (Phila). 2001;40(7): 365–73., ⁵⁰50. Wise KA, Sebastian SJ, Haas-Gehres AC, Moore-Clingenpeel MD, Lamberjack KE. Pharmacist impact on pediatric vaccination errors and missed opportunities in the setting of clinical decision support. J Am Pharm Assoc. 2017;57(3):356–61.) and 9 before-and-after studies (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91., ²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35., ³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62., ³⁴34. Hicks P, Tarr GAM, Hicks XP. Reminder cards and immunization rates among Latinos and the rural poor in Northeast Colorado. J Am Board Fam Med. 2007;20(6):581–6., ⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92., ⁴²42. Minkovitz CS, Belote AD, Higman SM, Serwint JR, Weiner JP. Effectiveness of a practice-based intervention to increase vaccination rates and reduce missed opportunities. Arch Pediatr Adolesc Med. 2001;155(3):382–6., ⁴⁴44. Sabnis SS, Pomeranz AJ, Amateau MM. The effect of education, feedback, and provider prompts on the rate of missed vaccine opportunities in a community health center. Clin Pediatr (Phila). 2003;42(2):147–51., ⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5.) were at high risk of bias due to issues with random sequence generation. Finally, 22 out of the 27 included studies were judged to have unclear or high risk of bias due to incomplete or missing outcome data (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91.–²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35., ³¹31. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–69., ³³33. Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med. 1994;3(3):257–62.–³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623., ³⁹39. MacIntyre CR, Kainer MA, Brown GV. A randomised, clinical trial comparing the effectiveness of hospital and community-based reminder systems for increasing uptake of influenza and pneumococcal vaccine in hospitalised patients aged 65 years and over. Gerontology. 2003;49(1):33–40., ⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92., ⁴²42. Minkovitz CS, Belote AD, Higman SM, Serwint JR, Weiner JP. Effectiveness of a practice-based intervention to increase vaccination rates and reduce missed opportunities. Arch Pediatr Adolesc Med. 2001;155(3):382–6.–⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.).

We summarized the effect of interventions for reducing MOVs and the certainty of this evidence in Table 1. Most studies compared interventions to standard of care (or to no intervention). The study characteristics table (available upon request) details all comparisons.

Interventions targeting healthcare team

Eight studies identified interventions targeting the healthcare team. There was moderate certainty evidence that monetary incentives (7.5% reduction in MOVs) (²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35.) and training on how to communicate with patients (RR 0.97; 95% CI [0.95, 0.98]) (³⁷37. Lin CJ, Nowalk MP, Pavlik VN, Brown AE, Zhang S, Raviotta JM, et al. Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial. BMC Infect Dis. 2016;16(1):623.) probably reduce MOVs. There was low certainty evidence suggesting that reminders attached to medical records (RR 0.50; 95% CI [0.19, 1.33] from RCTs) (³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9.) may reduce MOVs, that educational interventions and performance feedback may reduce MOVs (RR 0.27; 95% CI [0.20, 0.36]), and that educational interventions may reduce MOVs (RR 0.56; 95% CI [0.35, 0.88]) (²⁷27. Deivanayagam N, Nedunchelian K, Mala N, Ashok TP, Rathnam SR, Ahmed SS. Missed opportunities for immunization in children under 2 years attending an urban teaching hospital. Indian Pediat. 1995;32(1):51–7., ⁴⁴44. Sabnis SS, Pomeranz AJ, Amateau MM. The effect of education, feedback, and provider prompts on the rate of missed vaccine opportunities in a community health center. Clin Pediatr (Phila). 2003;42(2):147–51., ⁴⁶46. Skull S, Krause V, Roberts L, Dalton C. Evaluating the potential for opportunistic vaccination in a Northern Territory hospital. J Paediatr Child Health. 1999;35(5):472–5.). One study on web-based training (⁴⁹49. Werk LN, Diaz MC, Cadilla A, Franciosi JP, Hossain MJ. Promoting Adherence to Influenza Vaccination Recommendations in Pediatric Practice. J Prim Care Community Health. 2019;10:2150132719853061.) and another using provision of expert recommendations (³²32. Giduthuri JG, Purohit V, Maire N, Kudale A, Utzinger J, Schindler C, et al. Influenza vaccination of pregnant women: Engaging clinicians to reduce missed opportunities for vaccination. Vaccine. 2019;37(14):1910–7.) did not report reductions in MOVs.

Interventions targeting patients or caregivers

Seven studies identified interventions targeting patients or caregivers. There was moderate certainty evidence that educational interventions likely reduce MOVs (RR 0.89; 95% CI [0.73, 1.09]) (⁵¹51. Wood D, Schuster M, Donald-Sherbourne C, Duan N, Mazel R, Halfon N. Reducing missed opportunities to vaccinate during child health visits. How effective are parent education and case management? Arch Pediatr Adolesc Med. 1998;152(3):238–43.). There was low certainty evidence suggesting that monetary fines may reduce MOVs (RR 0.87; 95% CI [0.80, 0.94]) (⁴¹41. Minkovitz C, Holt E, Hughart N, Hou W, Thomas L, Dini E, et al. The effect of parental monetary sanctions on the vaccination status of young children: an evaluation of welfare reform in Maryland. Arch Pediatr Adolesc Med. 1999;153(12):1242–7.). The certainty of evidence was very low for reminders and alerts, either mailed or by telephone (²⁶26. Daley MF, Steiner JF, Kempe A, Beaty BL, Pearson KA, Jones JS, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambul Pediatr. 2004;4(3):217–23., ³⁵35. Irigoyen MM, Findlay S, Wang D, Chen S, Chimkin F, Pena O et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr. 2006;6(2):100–4., ³⁶36. Kempe A, Lowery NE, Pearson KA, Renfrew BL, Jones JS, Steiner JF, et al. Immunization recall: effectiveness and barriers to success in an urban teaching clinic. J Pediatr. 2001;139(5):630–5.), and SMS (short message service) (⁴⁰40. McIver R, Dyda A, McNulty AM, Knight V, Wand HC, Guy RJ. Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. J Am Med Inform Assoc. 2016;23(e1):e88–e92.). One study assessed patient tracking and outreach for reducing MOVs, but no estimates could be abstracted (⁴³43. Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics. 1999;103(1):31–8.).

Interventions targeting the healthcare system

Five studies identified interventions targeting healthcare systems. There was low certainty evidence suggesting that presence of a pharmacist (OR 0.56; 95% CI [0.35, 0.90]) (⁵⁰50. Wise KA, Sebastian SJ, Haas-Gehres AC, Moore-Clingenpeel MD, Lamberjack KE. Pharmacist impact on pediatric vaccination errors and missed opportunities in the setting of clinical decision support. J Am Pharm Assoc. 2017;57(3):356–61.), offering free vaccination programs (RR 1.00; 95% CI [0.95, 1.05]) (²⁹29. Fairbrother G, Friedman S, Hanson KL, Butts GC. Effect of the vaccines for children program on inner-city neighborhood physicians. Arch Pediatr Adolesc Med. 1997;151(12):1229–35.), and exempting patients from caregiver authorization (researchers reported no statistical differences) may reduce MOVs (⁴⁸48. Szilagyi PG, Rodewald LE, Humiston SG, Pollard L, Klossner K, Jones AM, et al. Reducing missed opportunities for immunizations. Easier said than done. Arch Pediatr Adolesc Med. 1996;150(11): 1193–200.). There was very low certainty evidence suggesting that quality improvement programming may result in little to no reduction of MOVs (no reduction reported) (²⁵25. Adamu AA, Uthman OA, Gadanya MA, Wiysonge CS. Implementation and evaluation of a collaborative quality improvement program to improve immunization rate and reduce missed opportunities for vaccination in primary health-care facilities: a time series study in Kano, Nigeria. Expert Rev Vaccines. 2019;18(9):969–91.). One study measuring outcomes of improving accessibility to vaccination sites did not report reduction of MOVs (³⁸38. Loevinsohn BP, Gareaballah E. Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan. Bull World Health Organ. 1992;70(3):335–9.).

Results of studies developed by PAHO (Objective 2)

We included five studies conducted between 2012 and 2018 in Honduras, Ecuador, Peru, Panama, and Bogotá (Colombia) (⁵²52. Martínez Reyes IE, Varela Murillo Ml, Sevilla Maradiaga JJ, Estrada OM, McField Montes GJ, Pardo Cruz F, et al. Oportunidades Perdidas de Vacunación en los niños menores de cinco años en los Establecimientos de Salud del Primer Nivel de Atención, Honduras, mayo del 2018. Tegucigalpa: Gobierno de la República de Honduras, Secretaría de Salud; 2018.–⁵⁶56. Sigma Dos Colombia; Organización Panamericana de la Salud. Informe Final: Oportunidades Perdidas de Vacunación en Colombia. [Report acquired through the Pan American Health Organization Comprehensive Family Immunization Unit.] Bogotá: Sigma Dos Colombia, Organización Panamericana de la Salud; 2014.). Studies included 368 to 2 495 participants. Three of the studies focused on reducing MOVs in children under 5 years of age (⁵²52. Martínez Reyes IE, Varela Murillo Ml, Sevilla Maradiaga JJ, Estrada OM, McField Montes GJ, Pardo Cruz F, et al. Oportunidades Perdidas de Vacunación en los niños menores de cinco años en los Establecimientos de Salud del Primer Nivel de Atención, Honduras, mayo del 2018. Tegucigalpa: Gobierno de la República de Honduras, Secretaría de Salud; 2018., ⁵³53. Jimbo Sotomayor R, Armijos Acurio L, Sánchez Choez X, Vilema Ortiz M, Ghisays G, Moyota Quinzo D, et al. Missed opportunities of vaccination in primary health care establishments in Ecuador. Vacunas. 2019;20(2):46–52., ⁵⁵55. Quiroz N, Jimeno J, Sales V, Alvarez S, Fossatti L. Informe final: Evaluación de oportunidades perdidas de vacunación en la República de Panamá. [Report acquired through the Pan American Health Organization Comprehensive Family Immunization Unit.] Panama City: Vax Trials; 2014.), one on children under 3 (⁵⁴54. Perú, Ministerio de Salud; Organización Panamericana de la Salud. Investigación de oportunidades perdidas para vacunación (OPPV) en Perú. [Report acquired through the Pan American Health Organization Comprehensive Family Immunization Unit.] Perú, Ministerio de Salud; 2014.), and another on children under 2 years (⁵⁶56. Sigma Dos Colombia; Organización Panamericana de la Salud. Informe Final: Oportunidades Perdidas de Vacunación en Colombia. [Report acquired through the Pan American Health Organization Comprehensive Family Immunization Unit.] Bogotá: Sigma Dos Colombia, Organización Panamericana de la Salud; 2014.). All studies used a cross-sectional design, and one added a qualitative design component (⁵²52. Martínez Reyes IE, Varela Murillo Ml, Sevilla Maradiaga JJ, Estrada OM, McField Montes GJ, Pardo Cruz F, et al. Oportunidades Perdidas de Vacunación en los niños menores de cinco años en los Establecimientos de Salud del Primer Nivel de Atención, Honduras, mayo del 2018. Tegucigalpa: Gobierno de la República de Honduras, Secretaría de Salud; 2018.). Table 2 summarizes the main characteristics of these studies.

Studies reported a total of 579 MOVs for 2 030 children eligible for vaccination (a rate of 5%–37% MOVs per country) (Table 2). The pooled rate of MOVs, estimated from data from five countries in the Region, was 17% (95% CI [9, 32]). Statistical tests suggest important heterogeneity among included studies (I² 97%). All studies reported that parents/caregivers, primarily women, made vaccination-related decisions for their children, and MOVs did not seem to be associated with the caregiver’s level of education (32%–72% of caregivers completed high school) (Table 2). The reasons for MOVs varied across countries, and the largest proportion of MOVs were attributable to healthcare personnel knowledge, attitudes, and practices (Figure 2).

Certainty of the evidence for Objectives 1 and 2

The certainty of the evidence on interventions for reducing MOVs ranged from very low to moderate certainty, due to serious issues of risk of bias and imprecision, and studies measuring the prevalence of MOVs in Latin America were of low certainty. The main limitation of this second body of evidence was indirectness (applicability concerns) and imprecision, with the extremes of the confidence interval of the pooled estimate suggesting a small prevalence of MOVs at one extreme and a large prevalence of MOVs at the other.

DISCUSSION

This review summarizes the proportion of MOVs in the Region and the best available evidence on interventions (targeting healthcare teams, patients or caregivers, and healthcare systems) for reducing MOVs. Results from studies from the Region suggest that 17% of recommended vaccines are missed (low certainty evidence). Results also suggest that monetary incentives to healthcare teams, training healthcare teams on patient communication, and educational interventions for caregivers probably reduce MOVs (moderate to very low certainty across interventions and outcomes).

Overall completeness and applicability of evidence

For our first research objective, reports meeting eligibility criteria studied interventions for reducing MOVs for children and were conducted in Australia and the United States of America—countries with high vaccine coverage. This may limit the applicability of the findings to adults or settings in low- and middle-income countries, including several countries within the Region. Although there was a variety of included interventions, which readers might interpret as increased applicability, each of the interventions seemed to have its own unique effect. Because of this, the effect of a specific intervention should not be generalized to other interventions aimed at the same recipient. Cross-sectional studies on the prevalence of MOVs in Latin America were conducted in five countries. The heterogeneity observed among these studies decreases the applicability of the findings to settings outside of the Region.

Agreement and disagreement with other studies or reviews

In a recently published systematic review, authors addressed our first research objective (¹³13. Jaca A, Mathebula L, Iweze A, Pienaar E, Wiysonge CS. A systematic review of strategies for reducing missed opportunities for vaccination. Vaccine. 2018;36(21):2921–7.). In comparison, we did not limit eligibility criteria to RCTs (resulting in 26 additional studies added to this review), performed more comprehensive searches, and evaluated certainty of the evidence using the GRADE approach; this allowed us to more accurately present limitations that may influence results and conclusions. Finally, authors from the previous review suggested that patient education/tracking, outreach sessions, and provider prompts reduced MOVs. Our review suggests that, in addition to the interventions suggested by the previous review, monetary incentives for healthcare teams probably reduce MOVs.

Implications for research

We suggest the development of higher-quality randomized and non-randomized studies on interventions for reducing MOVs, especially for adult and special populations that require vaccination (e.g., women of childbearing age and seniors). We also suggest that this research be conducted in low- or middle-income countries (or those with lower vaccination coverage) and in the context of issues associated with access to care, vaccine hesitancy, and the COVID-19 pandemic. We encourage testing of cross-cutting health system changes, interventions targeting vaccine hesitancy, and community-level interventions. Furthermore, we encourage reporting of outcomes such as costs, vaccine wastage rate, satisfaction with care, occurrence of vaccine-preventable diseases, and healthcare personnel’s attitudes. In addition, uniform definitions of MOVs should be adopted to promote comparability of studies.

Implications for practice and policy

The results of this review suggest an important rate of MOVs in the Region. This is in line with evidence on decreased vaccine coverage, weakening of epidemiological surveillance, emerging outbreaks of vaccine-preventable diseases, inadequate long-term sustainable financing, the challenge of scientific misinformation, and increasing vaccine hesitancy. Every effort must be made to consider interventions for reducing MOVs while maintaining the trust of populations in the Region in vaccination services, establish and follow national guidelines for immunization in the context of COVID-19 transmission, prioritize newborn vaccinations, and maintain periodic and systematic registries of the population pending vaccination (⁷7. Pan American Health Organization. Annual Report of the Director of the Pan American Sanitary Bureau 2020. Saving Lives and Improving Health and Well-Being. Washington, DC: PAHO; 2020. Available from: https://iris.paho.org/handle/10665.2/52852
https://iris.paho.org/handle/10665.2/528... , ⁵⁷57. MacDonald N, Mohsni E, Al-Mazrou Y, Andrus JK, Arora N, Elden S, et al. Global vaccine action plan lessons learned I: Recommendations for the next decade. Vaccine. 2020;38(33):5364–71.). Now more than ever, ensuring that every child who accesses a health center has their vaccination status reviewed and their vaccination schedule completed will be an opportunity to improve vaccination coverage and reduce incidence of vaccine-preventable diseases (¹1. World Health Organization [Internet]. Geneva: WHO; 2021. Vaccines and immunization. Available from: http://www.who.int/topics/immunization/en/.
http://www.who.int/topics/immunization/e... ).

Strengths and limitations of this review

The strengths of this review include not limiting evidence to studies in English, assessment of gray literature, and conducting screening and data abstraction independently and in duplicate. This review is limited by reporting issues in and methodological quality of the included evidence, which largely prevented us from conducting meta-analyses or from presenting the results in a consistent manner across interventions/outcomes. Furthermore, a paucity (and poor reporting) of study data resulted in our analysis not comprehensively accounting for instances where unit of randomization differed from unit of analysis. This may have resulted in confidence intervals that are more precise than they should be. Because the certainty of many of our estimates was rated down for imprecision, we believe that unit of analysis issues did not have an important impact on our conclusions.

Conclusion

MOVs can increase the incidence of vaccine-preventable diseases in the Region. Clinicians and other decisionmakers within the Region should consider this review’s limitations, paucity of evidence on patient-important outcomes, and the influence of several other factors (including costs, feasibility, acceptability, and equity) before implementing interventions for reducing MOVs (monetary incentives to healthcare teams, training healthcare teams on patient communication, and educational interventions for caregivers) at an individual or population level.

Disclaimer.

Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the RPSP/PAJPH and/or those of the Pan American Health Organization.

Acknowledgments.

The authors would like to thank the national officials, subnational authorities, local health workers, and parents in Colombia, Ecuador, Honduras, Panama, and Peru for their participation in these assessments of missed opportunities for vaccination. Without the cooperation and collaboration of all of them, including officers of the Pan American Health Organization and the Albert B. Sabin Institute (Grant 397020 Improving Vaccine Coverage Rates in the Latin American Region), this review would not have been successfully completed.

REFERENCES

Publication Dates

History