Study contributions
Main results
Annual vaccination coverage was below 95% in Brazil. The second dose of the vaccine showed stationary and decreasing trends in the country’s Federative Units. The dropout rate varied greatly throughout the study period.
Implications for services
The results found regarding the trends serve to inform and point to the urgency of planning actions aimed at improving coverage of the triple viral vaccine nationally in Brazil.
Perspectives
Investments in enhanced training of epidemiological surveillance professionals and enhanced computerized systems are necessary, with a view to continuous monitoring, to support actions to promote better and timely vaccine coverage.
Keywords:
Child Vaccination; Vaccination Coverage; Immunization Schedule; Time Series Studies
Abstract
Objective:
to analyze the temporal trend of coverage and dropout rate for triple viral vaccine in Brazil, according to the country’s Federative Units and Macro-Regions, between 2014 and 2021.
Methods:
this was an ecological time series study, using data from the National Immunization Program Information System and the Live Birth Information System; joinpoint regression models were used.
Results:
in Brazil as a whole annual vaccination coverage was below 95% and ranged from 92.3% (2015) to 54.4% (2021); the second dose of the vaccine showed a decreasing temporal trend in the period (average change over the period = -5.8; 95%CI -10.5;-0.8); the temporal trends were stationary and decreasing in the country’s Federative Units; the dropout rate ranged from 22.2% (2014) to 37.4% (2021).
Conclusion:
there was a downward trend in vaccination coverage and an increase in the dropout rate in Brazil as a whole and in the country’s Federative Units.
Keywords:
Child Vaccination; Vaccination Coverage; Immunization Schedule; Time Series Studies
Resumen
Objetivo:
analizar la tendencia temporal de cobertura y tasa deserción de la vacuna triple viral en Brasil, y según Unidades de la Federación y Regiones, entre 2014 y 2021.
Métodos:
estudio de serie temporal ecológica, sobre datos de los sistemas del Inmunizaciones y Nacido Vivo; se utilizaron modelos de regresión de punto de inflexión.
Resultados:
la cobertura anual de vacunación estuvo por debajo del 95% y osciló entre 92,3% (2015) y 54,4% (2021), en Brasil; la segunda dosis mostró una tendencia temporal decreciente en el período (variación promedia en el periodo = -5,8; IC95% -10,5;-0,8); las tendencias temporales fueron estacionarias y decrecientes en las Unidades de la Federación; la tasa deserción de varió del 22,2% (2014) al 37,4% (2021).
Conclusión:
hubo una tendencia a la baja en las coberturas de vacunación y un aumento en la tasa de deserción en Brasil y en las Unidades de la Federación.
Palabras-clave:
Vacunación Infantil; Cobertura de Vacunación; Esquema de Vacunación; Estudios de Series Temporales
INTRODUCTION
Epidemiological surveillance, when integrated with immunization actions, enables control, eradication and elimination of vaccine-preventable diseases, promoting improvement in the population’s health.11. World Health Organization. Eliminating measles and rubella and preventing congenital rubella infection: WHO European Region strategic plan 2005-2010 [Internet]. Copenhagen: World Health Organization; c2005 [cited 2022 Nov 10]. 28 p. Available from: Available from: https://apps.who.int/iris/bitstream/handle/10665/326561/9789289013826-eng.pdf?sequence=3&isAllowed=y
https://apps.who.int/iris/bitstream/hand... ),(22. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Vigilância das Doenças Transmissíveis. Manual de normas e procedimentos para vacinação [Internet]. Brasília: Ministério da Saúde; 2014 [citado 2023 Fev 4]. 176 p. Disponível em: Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf
https://bvsms.saude.gov.br/bvs/publicaco... However, the benefits of immunization are unequally distributed: among poorer, more marginalized and more vulnerable populations, access to these benefits is limited to immunization services.33. World Health Organization. Implementing the immunization agenda 2030: a framework for action through coordinated planning, monitoring & evaluation, ownership & accountability, and communications & advocacy [Internet]. Geneva: World Health Organization; 2021 [cited 2021 Oct 18]. 31 p. Available from: Available from: https://cdn.who.int/media/docs/default-source/immunization/strategy/ia2030/ia2030_frameworkforactionv04.pdf?sfvrsn=e5374082_1&download=true
https://cdn.who.int/media/docs/default-s...
The Immunization Agenda 2030 (IA2030) aims to improve the global population’s access to primary health care and achieve universal coverage of vaccine products. In this sense, childhood vaccination is essential for strengthening public health policies, as well as implementation and progress of immunization programs worldwide.33. World Health Organization. Implementing the immunization agenda 2030: a framework for action through coordinated planning, monitoring & evaluation, ownership & accountability, and communications & advocacy [Internet]. Geneva: World Health Organization; 2021 [cited 2021 Oct 18]. 31 p. Available from: Available from: https://cdn.who.int/media/docs/default-source/immunization/strategy/ia2030/ia2030_frameworkforactionv04.pdf?sfvrsn=e5374082_1&download=true
https://cdn.who.int/media/docs/default-s...
Several countries achieved improvement in child vaccination coverage between 1980 and 2010.44. GBD 2020, Release 1, Vaccine Coverage Collaborators. Measuring routine childhood vaccination coverage in 204 countries and territories, 1980-2019: a systematic analysis for the Global Burden of Disease Study 2020, Release 1. Lancet. 2021;398(10299):503-21. doi: 10.1016/s0140-6736(21)00984-3
https://doi.org/10.1016/s0140-6736(21)00... However, in the 2010s, with the introduction and expansion of new vaccines, particularly in Latin America and the Caribbean, reductions in vaccination coverage were seen, with fewer countries in these regions of the Americas achieving 90% coverage for five of the nine childhood vaccines between 2013 and 2017: only 61% of Latin American and Caribbean countries achieved 90% coverage for the first dose of the triple viral vaccine in 2017.44. GBD 2020, Release 1, Vaccine Coverage Collaborators. Measuring routine childhood vaccination coverage in 204 countries and territories, 1980-2019: a systematic analysis for the Global Burden of Disease Study 2020, Release 1. Lancet. 2021;398(10299):503-21. doi: 10.1016/s0140-6736(21)00984-3
https://doi.org/10.1016/s0140-6736(21)00... ),(55. Llau AF, Williams ML, Tejada CE. National vaccine coverage trends and funding in Latin America and the Caribbean. Vaccine. 2021;39(2):317-23. doi: 10.1016/j.vaccine.2020.11.059
https://doi.org/10.1016/j.vaccine.2020.1...
Difficulties in achieving or maintaining the immunization coverage target are recurrent. In 2020 especially, during the early stages of the novel coronavirus (COVID-19) pandemic, routine childhood immunization services were interrupted due to social distancing measures taken with the aim of preventing SARS-CoV-2 transmission. Consequently, mass vaccination campaigns intended to prevent diseases such as measles, meningitis and polio were not undertaken.66. World Health Organization. At least 80 million children under one at risk of diseases such as diphtheria, measles and polio as COVID-19 disrupts routine vaccination efforts, warn Gavi, WHO and UNICEF [Internet]. Geneva: World Health Organization ; 2020 [cited 2022 Mar 4]. Available from: Available from: https://www.who.int/news-room/detail/22-05-2020-at-least-80-million-children-under-one-at-risk-of-diseases-such-as-diphtheria-measles-and-polio-as-covid-19-disrupts-routine-vaccination-efforts-warn-gavi-who-and-unicef
https://www.who.int/news-room/detail/22-...
The Brazilian National Immunization Program has achieved worldwide recognition, given the geographic dimension and complexity of operations involved in vaccination campaigns, routine vaccination and vaccine blockades in the country.77. Domingues CMAS, Fantinato FFST, Duarte E, Garcia LP. Vacina Brasil e estratégias de formação e desenvolvimento em imunizações. Epidemiol Serv Saude [Internet]. 2019;28(2):e20190223. doi: 10.5123/S1679-49742019000200024
https://doi.org/10.5123/S1679-4974201900... ),(88. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Programa Nacional de Imunizações 30 anos [Internet]. Brasília: Ministério da Saúde; 2003 [citado 2022 Mar 15]. 208 p. Disponível em: Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/livro_30_anos_pni.pdf
https://bvsms.saude.gov.br/bvs/publicaco...
The National Immunization Program offers, free of charge, a variety of immunobiologics for different age groups, from childhood to old age. Through the population’s adherence to vaccination and timely health surveillance, measles transmission in the Americas was interrupted.99. Domingues CMAS, Maranhão AGK, Teixeira AM, Fantinato FFS, Domingues RAS. 46 anos do Programa Nacional de Imunizações: uma história repleta de conquistas e desafios a serem superados. Cad Saude Publica. 2020;36(Supl 2):e00222919. oi: 10.1590/0102-311X00222919
https://doi.org/10.1590/0102-311X0022291... ),(1010. Sato APS. Qual a importância da hesitação vacinal na queda das coberturas vacinais no Brasil? Rev Saude Publica. 2018;52(1):96. doi: 10.11606/S1518-8787.2018052001199
https://doi.org/10.11606/S1518-8787.2018... Measles is an extremely contagious disease, it can cause serious complications and even death, especially in children under 5 years of age and malnourished children.1111. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Coordenação Geral de Desenvolvimento da Epidemiologia em Serviços. Guia de Vigilância em Saúde. 3. ed. Brasília: Ministério da Saúde, 2019. 740 p. However, the circulation of measles in other regions of the world led to the reintroduction of the virus in Brazil in 2018,1212. Organização Pan-Americana da Saúde. Organização Mundial da Saúde. 29ª Conferência Sanitária Pan-Americana. 69ª Sessão do Comitê Regional da OMS para as Américas; 2017 Sep 25-29 [cited 2022 Jun 28]; Washington, DC. Disponível em: Disponível em: https://iris.paho.org/bitstream/handle/10665.2/34446/CSP29-8-p.pdf?sequence=4&isAllowed=y
https://iris.paho.org/bitstream/handle/1... associated with the drop in vaccination coverage in the country.1010. Sato APS. Qual a importância da hesitação vacinal na queda das coberturas vacinais no Brasil? Rev Saude Publica. 2018;52(1):96. doi: 10.11606/S1518-8787.2018052001199
https://doi.org/10.11606/S1518-8787.2018... ),(1313. Parra CM, Ribeiro MAL, Bezerra IMP, Ribeiro MR, Abreu LC. Vaccine coverage and measles incidence in Northern Brazil. J Hum Growth Dev. 2022;32(1):21-9. doi: 10.36311/jhgd.v32.12617
https://doi.org/10.36311/jhgd.v32.12617... ),(1414. Pacheco FC, França GVA, Elidio GA, Domingues CMAS, Oliveira C, Guilhem DB. Trends and spatial distribution of MMR vaccine coverage in Brazil during 2007-2017. Vaccine. 2019;37(20):2651-5. doi: 10.1016/j.vaccine.2019.04.019
https://doi.org/10.1016/j.vaccine.2019.0...
Vaccination coverage is one of the indicators capable of evaluating the performance of vaccination strategies, when measuring the effect of the intervention on an eligible population. Another indicator of vaccination coverage is the dropout rate, which estimates the population’s adherence to the vaccination schedule proposed by the Brazilian National Immunization Program, that is, how many people started but did not complete the vaccination schedule. Vaccination coverage also estimates the effectiveness of interventions, compared to programmed actions.1515. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Articulação Estratégica de Vigilância em Saúde. Guia de Vigilância em Saúde. 5. ed. Brasília: Ministério da Saúde ; 2021 [citado 2021 Ago 2]. 1128 p. Disponível em: Disponível em: https://www.gov.br/saude/pt-br/centraisde-conteudo/publicacoes/publicacoes-svs/vigilancia/guia-de-vigilancia-emsaude_5ed_21nov21_isbn5.pdf/view
https://www.gov.br/saude/pt-br/centraisd... ),(1616. Fundação Maria Cecilia Souto Vidigal. Desigualdades e impactos da covid-19 na atenção à primeira infância. São Paulo: Fundação Maria Cecilia Souto Vidigal; 2022 [citado 2023 Fev 4]. Disponível em: Disponível em: https://www.fmcsv.org.br/pt-BR/biblioteca/impactos-covid-2022/
https://www.fmcsv.org.br/pt-BR/bibliotec...
Surveillance of immunization indicators is essential for achieving and maintaining established coverage targets, aiming to protect the population from vaccine-preventable diseases, especially those that affect children.1515. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Articulação Estratégica de Vigilância em Saúde. Guia de Vigilância em Saúde. 5. ed. Brasília: Ministério da Saúde ; 2021 [citado 2021 Ago 2]. 1128 p. Disponível em: Disponível em: https://www.gov.br/saude/pt-br/centraisde-conteudo/publicacoes/publicacoes-svs/vigilancia/guia-de-vigilancia-emsaude_5ed_21nov21_isbn5.pdf/view
https://www.gov.br/saude/pt-br/centraisd... ),(1717. Ministério da Saúde (BR). Departamento de Informática do SUS (DATASUS). Imunizações - Cobertura - Brasil [Internet]. Brasília: Ministério da Saúde; 2022 [citado 2022 Jul 12]. Disponível em: Disponível em: http://tabnet.datasus.gov.br/cgi/dhdat.exe?bd_pni/cpnibr.def
http://tabnet.datasus.gov.br/cgi/dhdat.e...
Brazil offers triple viral vaccination - against measles, mumps and rubella (MMR) - on the childhood vaccination schedule, with a first dose at 12 months old; and the second dose of MMR vaccine or, alternatively, a dose of tetraviral vaccine - against measles, mumps, rubella and varicella (MMRV) at 15 months old. This has been the National Immunization Program guideline since 2014.
The objective of this study was to analyze the temporal trend of MMR vaccination coverage and dropout rate in Brazil, according to the country’s Federative Units (FUs) and Macro-Regions, from 2014 to 2021.
METHODS
This was an ecological time series study, using data from the National Immunization Program Information System (Sistema de Informações do Programa Nacional de Imunizações - SI-PNI)1818. Ministério da Saúde (BR). Departamento de Informática do SUS (DATASUS). Nascidos Vivos - Brasil [Internet]. Brasília: Ministério da Saúde ; 2022 [citado 2022 Jun 5]. Disponível em: Disponível em: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def
http://tabnet.datasus.gov.br/cgi/tabcgi.... and the Live Birth Information System (Sistema de Informações sobre Nascidos Vivos - SINASC),1919. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Coordenação-Geral do Programa Nacional de Imunizações. Informe técnico de introdução da vacina tetra viral: Vacina sarampo, caxumba, rubéola e varicela (atenuada) [Internet]. Brasília: Ministério da Saúde ; 2013 [citado 2021 Nov 30]. Disponível em: Disponível em: http://www.sopape.com.br/data/conteudo/arquivos/informe_tecnico_introducao_vacina_tetraviral.pdf
http://www.sopape.com.br/data/conteudo/a... for the period 2014-2021, taking the Brazilian territory as a whole, its FUs and its macro-regions as units of analysis.
The SI-PNI system aggregates information related to the records of administered vaccine doses, by period of time and geographic area of vaccine administration.1818. Ministério da Saúde (BR). Departamento de Informática do SUS (DATASUS). Nascidos Vivos - Brasil [Internet]. Brasília: Ministério da Saúde ; 2022 [citado 2022 Jun 5]. Disponível em: Disponível em: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def
http://tabnet.datasus.gov.br/cgi/tabcgi.... The SINASC system holds information regarding births registered in Brazil.1919. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Coordenação-Geral do Programa Nacional de Imunizações. Informe técnico de introdução da vacina tetra viral: Vacina sarampo, caxumba, rubéola e varicela (atenuada) [Internet]. Brasília: Ministério da Saúde ; 2013 [citado 2021 Nov 30]. Disponível em: Disponível em: http://www.sopape.com.br/data/conteudo/arquivos/informe_tecnico_introducao_vacina_tetraviral.pdf
http://www.sopape.com.br/data/conteudo/a... The two databases are freely accessible, being made available by the Brazilian National Health System Department of Information Technology (Departamento de Informática do Sistema Único de Saúde - DATASUS).1818. Ministério da Saúde (BR). Departamento de Informática do SUS (DATASUS). Nascidos Vivos - Brasil [Internet]. Brasília: Ministério da Saúde ; 2022 [citado 2022 Jun 5]. Disponível em: Disponível em: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def
http://tabnet.datasus.gov.br/cgi/tabcgi.... ),(1919. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Coordenação-Geral do Programa Nacional de Imunizações. Informe técnico de introdução da vacina tetra viral: Vacina sarampo, caxumba, rubéola e varicela (atenuada) [Internet]. Brasília: Ministério da Saúde ; 2013 [citado 2021 Nov 30]. Disponível em: Disponível em: http://www.sopape.com.br/data/conteudo/arquivos/informe_tecnico_introducao_vacina_tetraviral.pdf
http://www.sopape.com.br/data/conteudo/a...
We consulted the SI-PNI and SINASC records and processed the resulting data using the Health Information Tabulator (Tabulador de Informações em Saúde - TabNet), an application made available by DATASUS.1818. Ministério da Saúde (BR). Departamento de Informática do SUS (DATASUS). Nascidos Vivos - Brasil [Internet]. Brasília: Ministério da Saúde ; 2022 [citado 2022 Jun 5]. Disponível em: Disponível em: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def
http://tabnet.datasus.gov.br/cgi/tabcgi.... ),(1919. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Coordenação-Geral do Programa Nacional de Imunizações. Informe técnico de introdução da vacina tetra viral: Vacina sarampo, caxumba, rubéola e varicela (atenuada) [Internet]. Brasília: Ministério da Saúde ; 2013 [citado 2021 Nov 30]. Disponível em: Disponível em: http://www.sopape.com.br/data/conteudo/arquivos/informe_tecnico_introducao_vacina_tetraviral.pdf
http://www.sopape.com.br/data/conteudo/a... Data from both systems were accessed on October 25, 2022 and filtered using TabNet, as follows:
a) Administered doses - SI-PNI
- period (2014 - 2021);
- FU;
- imunobiolologic (MMR and MMRV vaccines); and
- dose (1st dose and 2nd dose);
b) Live birth population - SINASC
- period (2014 - 2021);
- FU;
- year of birth; and
- birth according to mother’s place of residence.
Vaccination coverage was calculated using the following formula:
When selecting the “second dose” variable, we compared the amounts of MMR vaccine and MMRV vaccine administered in each FU and opted for the vaccine with the highest number of doses administered there. This procedure was necessary as there was variation in the distribution logistics of these vaccines in the Brazil throughout the analyzed period.2020. Lima MM, Favacho ARM, Souza-Santos R, Gama SGN. Características e tendência temporal das taxas de mortalidade de crianças e adolescentes em Mato Grosso e no Brasil, 2009 a 2020. Epidemiol Serv Saude. 2022;31(3):e2022491. doi: 10.1590/S2237-96222022000300017
https://doi.org/10.1590/S2237-9622202200... The median value was used when calculating vaccine coverage by macro-region.
The dropout rate was calculated based on the first administered doses of the MMR vaccine and the second administered doses of the MMR or MMRV vaccines, using the same choice criterion defined for calculating vaccination coverage. The dropout rate was calculated using the following formula:
When FUs had inconsistent dropout rates, such as values < 1% or negative values, these were replaced by the dropout rate value for the previous year.
Joinpoint regression analysis models based on the Monte Carlo permutation method were used for temporal analysis of vaccination coverage and dropout rate. This regression model verifies whether a line with multiple points is statistically better for describing the temporal evolution of vaccination coverage and dropout rate, compared to a straight line. Classifying temporal trend as not significant (p-value > 0.05), positive (p-value < 0.05 and positive regression coefficient) or negative (p-value < 0.05 and negative regression coefficient) allowed us to calculate annual percentage change (APC) and the average change over the period (ACP). In the regression model, years with a dropout rate < 1 were excluded, by FU.2121. Aragão CMC, Mascarenhas MDM. Tendência temporal das notificações de lesão autoprovocada em adolescentes no ambiente escolar, Brasil, 2011-2018. Epidemiol Serv Saude. 2022;31(1):e202820. doi:10.1590/S1679-49742022000100028
https://doi.org/10.1590/S1679-4974202200... ),(2222. QGIS: Geographic Information System. Open Source Geospatial Foundation Project [Software]. [s.l.]: QGIS; 2021 [cited 2023 Feb 4]. Available from: Available from: http://qgis.osgeo.org
http://qgis.osgeo.org... ) A 95% confidence interval (95%CI) was used for all temporal trends.
We generated thematic maps of vaccination coverage ACP and dropout rate ACP per FU. The ACP value strata used in the thematic maps were obtained by adopting the QGIS program natural breaks procedure.
The digital grid for Brazil and its Federative Units was obtained from the webpage of the Brazilian Institute of Geography and Statistics (Instituto Brasileiro de Geografia e Estatística - IBGE) (https://www.ibge.gov.br/geociencias/downloads-geociencias.html), which we accessed on December 29, 2022.
In order to perform the analyses, we used the Join Point Regression Program, version 4.9.1.0, dated April 2022 (Statistical Research and Applications Branch, National Cancer Institute), and the QGIS Geographic Information System.2323. Muhoza P, Danovaro-Holliday MC, Diallo MS, Murphy P, Sodha SV, Requejo JH, et al. Routine Vaccination Coverage - Worldwide, 2020. MMWR Morb Mortal Wkly Rep. 2021;70(43):1495-500. doi: 10.15585/mmwr.mm7043a1
https://doi.org/10.15585/mmwr.mm7043a1...
As only secondary public domain and freely accessible data sources were used, the study project did not need to be submitted to a Research Ethics Committee.
RESULTS
In Brazil, MMR vaccine coverage ranged from 92.3% to 54.4% between 2015, 2017, 2018, 2020 and 2021. As for the second dose, MMR or MMRV vaccine coverage was below 95% in the period studied period. The dropout rate remained high throughout the period, ranging from 22.2% (2014) to 37.4% (2021) (Figure 1).
For Brazil as a whole, the temporal trend as per the MMR vaccine first dose coverage regression model was not significant, both in the first period, from 2014 to 2019 (APC = -2.4; 95%CI -8.6;4.2), and in the second period, from 2019 to 2021 (APC = -6.7; 95%CI -30.3;25.1) (Table 1).
As for coverage of the second MMR dose or its replacement by a dose of MMRV vaccine, the regression model showed the same periods of non-significant temporal vaccine coverage trends as the first dose. However, for the period as a whole, from 2014 to 2021, a negative trend was found (ACP = -5.8; 95%CI = -10.5;-0.8), from 91.0% (2014) to 54.4% (2021) (Figure 1).
The temporal trend in the dropout rate regression model was not considered to be significant, both in the period 2014-2019 and also in the period 2019-2021.
Brazilian macro-regions
The results of the temporal trend analysis of MMR vaccine first dose coverage for the Brazilian macro-regions were not significant for the North, Northeast and Southeast regions, from 2014 to 2019 (Table 1).
The North and Northeast regions showed the same temporal behaviors as Brazil as a whole for first dose MMR vaccine coverage, between 2014 and 2021. The trend was negative (ACP = -5.4; 95%CI -9.2;-1.4), decreasing from 105.0% (2014) to 71.0% (2021) in the Northern region (Table 2 and Figure 2A). In the case of MMR (or MMRV) vaccine second dose coverage, a non-significant temporal trend prevailed in the Brazilian regions. A negative trend was found for the Southeast region (ACP = -3.7; 95%CI -6.3;-1.1), falling from 92% (2014) to 66.6% (2021) (Table 2 and Figure 2B). The dropout rate was not significant in any of the Brazilian regions throughout the study period (Table 3).
MMR (1st dose), and MMR or MMRV (2nd dose), in the national macro-regions and Federative Units, Brazil, 2014-2021
Federative Units
The results of the temporal trend analysis of MMR vaccine first dose coverage by FU showed a negative trend in Roraima (APC = -9.4; 95%CI -13.4;-5.1), from 105.0% (2014) to 65.1% (2019) (Tables 1 and 2). With regard to the temporal trend in the period as a whole, from 2014 to 2021, Acre and Rio de Janeiro reported negative trends (Figure 2A).
The FUs that make up the Southeast region showed the same temporal behaviors for MMR vaccine first dose coverage as the region as a whole (Table 1).
Regarding MMR vaccine (or MMRV vaccine) second dose coverage, Roraima showed a negative trend in the period 2014-2019 (APC = -4.0; 95%CI -7.6;-0.3) and in the period 2019-2021 (APC = -27.5; 95%CI -38.7;-14.2), ranging from 85.6% to 37.0% in the period as a whole, from 2014 to 2021. In this longer period, negative trends were also found for Rondônia, Amapá, Pernambuco, Alagoas, Sergipe and Rio de Janeiro (Tables 1 and 2; Figure 2B).
Dropout rate trends over time were not assessed for Roraima and the Federal District because they were less than 1% between 2017 and 2019 (Table 3).
Vaccination coverage and dropout rate indicators, according to the period observed, showed different trends within the same region of the country, showing temporal heterogeneity between the FUs (Tables 1, 2 and 3; Figure 2C).
Spatial distribution of annual average percentage changes in immunization indicators and classification of the dropout rate trend in the Federative Units, Brazil, 2014-2021
DISCUSSION
In this study, coverage of both the first MMR vaccine dose and the second MMR dose - or its replacement with a dose of MMRV vaccine - decreased in Brazil as a whole, in the period selected for the study. The FUs, in particular, showed stationary or decreasing trends in vaccine coverage, either for the first or the second vaccine dose, over the period studied.
It should be noted that the temporal trend periods for second dose vaccination coverage were from 2014-2019 and 2019-2021 for all the Brazilian macro-regions. However, some FUs, such as Maranhão, Rio Grande do Norte, Bahia and the Federal District, differ from each other because their trend periods were 2014-2016 and 2016-2020, diverging from the periods applying to their respective macro-regions. These divergences point to the possibility of different factors interfering, at different times, in the vaccination coverage found.1010. Sato APS. Qual a importância da hesitação vacinal na queda das coberturas vacinais no Brasil? Rev Saude Publica. 2018;52(1):96. doi: 10.11606/S1518-8787.2018052001199
https://doi.org/10.11606/S1518-8787.2018...
The dropout rate, indicative of the portion of the population that did not complete the vaccination schedule,1515. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Articulação Estratégica de Vigilância em Saúde. Guia de Vigilância em Saúde. 5. ed. Brasília: Ministério da Saúde ; 2021 [citado 2021 Ago 2]. 1128 p. Disponível em: Disponível em: https://www.gov.br/saude/pt-br/centraisde-conteudo/publicacoes/publicacoes-svs/vigilancia/guia-de-vigilancia-emsaude_5ed_21nov21_isbn5.pdf/view
https://www.gov.br/saude/pt-br/centraisd... had a stationary trend, both in Brazil as a whole and in all the country’s regions. The Northeast and Southeast regions had the same trend periods for the dropout rate (2014-2019 and 2019-2021), in relation to Brazil as a whole. Among the FUs, only Rondônia, Maranhão, Pernambuco, Sergipe, Bahia, São Paulo and Santa Catarina reported the same periods as Brazil and the Northeast and Southeast regions, in the temporal context of the study.
In the FUs with periods and trend behaviors different from their respective regions, heterogeneity can be seen within their regions, with regard to vaccination coverage and dropout rates.
Childhood vaccine coverage has made progress. However, in the period from 2010 to 2019, coverage of the third dose of DTP vaccine (diphtheria, tetanus and pertussis), the first dose of MMR vaccine and third dose of vaccine against poliomyelitis stagnated or decreased. Worldwide, 94 countries and territories (46%) recorded reductions in these coverage levels.
Global coverage of the first MMR vaccine dose stagnated at a level between 84% and 86% in the period 2010-2019, while coverage of the second MMR vaccine dose has increased from 42% to 71%, reflecting the introduction of the second dose in many countries.2424. Braz RM, Domingues CMAS, Teixeira AMS, Luna EJA. Classificação de risco de transmissão de doenças imunopreveníveis a partir de indicadores de coberturas vacinais nos municípios brasileiros. Epidemiol Serv Saude. 2016;25(4):745-54. doi: 10.5123/S1679-49742016000400008
https://doi.org/10.5123/S1679-4974201600...
The second dose of the MMR and/or MMRV vaccine is not included in all vaccination schedules worldwide.2424. Braz RM, Domingues CMAS, Teixeira AMS, Luna EJA. Classificação de risco de transmissão de doenças imunopreveníveis a partir de indicadores de coberturas vacinais nos municípios brasileiros. Epidemiol Serv Saude. 2016;25(4):745-54. doi: 10.5123/S1679-49742016000400008
https://doi.org/10.5123/S1679-4974201600... In the case of Brazil, the inclusion of the second dose occurred in 2013 and its coverage remained below the target recommended by the National Immunization Program (< 95%) between 2014 and 2021.2525. Oliveira GS, Bitencourt EL, Amaral PFF, Vaz GP, Reis Júnior PM, Costa SB. Cobertura vacinal: uma análise comparativa entre os estados da Região Norte do Brasil. Rev Pat Tocantins. 2020;7(1):14-7. doi: 10.20873/uft.2446-6492.2020v7n1p14
https://doi.org/10.20873/uft.2446-6492.2...
Heterogeneity of vaccination coverage of nine vaccines on the childhood schedule, among the Brazilian regions, is more prominent in the Midwest, where it was higher (90.6%), compared to the other regions of the country from 2015 to 2019. The FUs that make up the Northern region also showed temporal heterogeneity in the vaccination coverage of nine vaccines on the childhood schedule, with Rondônia standing out with the best coverage (100%) and Pará with the worst coverage (69.4%), also between 2015 and 2019.2626. Veljkovic M, Loncarevic G, Kanazir M, Kisic-Tepavcevic D, Gazibara T. Trend in mandatory immunisation coverage: linear and joinpoint regression approach, Serbia, 2000 to 2017. Euro Surveill. 2021;26(26):2000417. doi: 10.2807/1560-7917.ES.2021.26.26.2000417
https://doi.org/10.2807/1560-7917.ES.202...
A study was conducted in Serbia on the temporal trends of mandatory childhood vaccination coverage between 2000 and 2017, using linear regression and joinpoint statistical methods. The linear regression revealed a significant drop in coverage of the first doses of poliomyelitis, DTP and MMR vaccines.
In the same period, coverage of all subsequent revaccinations decreased significantly.2727. Nações Unidas Brasil. Análise da ONU mostra ligação entre falta de equidade de vacinas e aumento da desigualdade [Internet]. Brasília: Nações Unidas Brasil; 2022 [atualizado 2022 Mar 28; citado 2023 Fev 4]. Disponível em: Disponível em: https://brasil.un.org/pt-br/176045-analise-da-onu-mostra-ligacao-entre-falta-de-equidade-de-vacinas-e-aumento-da-desigualdade
https://brasil.un.org/pt-br/176045-anali...
The impact of the COVID-19 pandemic contributed to an 84% reduction in global coverage of the first MMR vaccine dose, while coverage of the second MMR or MMRV vaccine dose remained stable, with average percentage values of 71% in 2019 and 70% in 2020, estimated based on recurring heterogeneity between the different regions of the world.2828. VanderEnde K, Gacic-Dobo M, Diallo MS, Conklin LM, Wallace AS. Routine Vaccination Coverage - Worldwide, 2027. MMWR Morb Mortal Wkly Rep. 2018;67(1):1261-4. doi: 10.15585/mmwr.mm6745a2
https://doi.org/10.15585/mmwr.mm6745a2...
High dropout rates are repeatedly found globally: in 2017, 6.2 million (31%) children started but did not complete the DTP vaccine schedule.2929. Danovaro-Holliday MC, Kretsinger K, Gacic-Dobo M. Measuring and ensuring routine childhood vaccination coverage. Lancet. 2021;398(10299):468-9. doi: 10.1016/S0140-6736(21)01228-9
https://doi.org/10.1016/S0140-6736(21)01... It is noteworthy that high dropout rates can mean reduced herd immunity and increased of cases of vaccine-preventable diseases.1515. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Articulação Estratégica de Vigilância em Saúde. Guia de Vigilância em Saúde. 5. ed. Brasília: Ministério da Saúde ; 2021 [citado 2021 Ago 2]. 1128 p. Disponível em: Disponível em: https://www.gov.br/saude/pt-br/centraisde-conteudo/publicacoes/publicacoes-svs/vigilancia/guia-de-vigilancia-emsaude_5ed_21nov21_isbn5.pdf/view
https://www.gov.br/saude/pt-br/centraisd...
The United Nations Development Programme reported that only 1% of the 10.7 billion doses of vaccines distributed worldwide were administered in low-income countries as at mid-2022. As such, the Immunization Agenda 2030 can not only help improve the quality of coverage estimates, but also help to identify and reach people needing to be vaccinated, including those from displaced and marginalized populations who are not being fully immunized in a timely manner.2828. VanderEnde K, Gacic-Dobo M, Diallo MS, Conklin LM, Wallace AS. Routine Vaccination Coverage - Worldwide, 2027. MMWR Morb Mortal Wkly Rep. 2018;67(1):1261-4. doi: 10.15585/mmwr.mm6745a2
https://doi.org/10.15585/mmwr.mm6745a2... ),(2929. Danovaro-Holliday MC, Kretsinger K, Gacic-Dobo M. Measuring and ensuring routine childhood vaccination coverage. Lancet. 2021;398(10299):468-9. doi: 10.1016/S0140-6736(21)01228-9
https://doi.org/10.1016/S0140-6736(21)01...
Barriers to vaccine equity may be related to lack of credibility of the information and guidance provided by health authorities and health professionals regarding vaccination. “Fake news” decreases the population’s confidence in the health system and, in particular, with regard to vaccination actions and campaigns. However, as government policies expand the availability of vaccines and health professionals engage in the vaccination process, this process is strengthened, as is the health system as a whole.1010. Sato APS. Qual a importância da hesitação vacinal na queda das coberturas vacinais no Brasil? Rev Saude Publica. 2018;52(1):96. doi: 10.11606/S1518-8787.2018052001199
https://doi.org/10.11606/S1518-8787.2018...
As for the limitations of this study, it is worth mentioning possible uncertainties/imprecision in the calculation of vaccination coverage, when the denominator used to calculate the rates includes population estimates that underestimate or overestimate the population under 1 year old, in addition to the insufficient number of observations analyzed. Moreover, constant changes in immunization information systems can lead to typing errors and information that is not migrated from one system to another and, consequently, underestimated vaccine coverage and an overestimated dropout rate.
This study makes progress by identifying temporal heterogeneity and periods of trends, in addition to differences in the geographical distribution of indicators, this being a form of analysis that should be incorporated into the routine of health services, in addition to addressing the dropout rate, which is an immunization indicator little discussed in the scientific literature.
We conclude that further studies are needed to characterize the spatial heterogeneity of MMR vaccine coverage and its dropout rate, as well as possibly associated factors. Furthermore, immunization services need to monitor temporal trends in vaccine coverage, with the aim of intensifying educational actions aimed at greater timely adherence by the population to vaccination.
REFERENCES
- 1World Health Organization. Eliminating measles and rubella and preventing congenital rubella infection: WHO European Region strategic plan 2005-2010 [Internet]. Copenhagen: World Health Organization; c2005 [cited 2022 Nov 10]. 28 p. Available from: Available from: https://apps.who.int/iris/bitstream/handle/10665/326561/9789289013826-eng.pdf?sequence=3&isAllowed=y
» https://apps.who.int/iris/bitstream/handle/10665/326561/9789289013826-eng.pdf?sequence=3&isAllowed=y - 2Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Vigilância das Doenças Transmissíveis. Manual de normas e procedimentos para vacinação [Internet]. Brasília: Ministério da Saúde; 2014 [citado 2023 Fev 4]. 176 p. Disponível em: Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf - 3World Health Organization. Implementing the immunization agenda 2030: a framework for action through coordinated planning, monitoring & evaluation, ownership & accountability, and communications & advocacy [Internet]. Geneva: World Health Organization; 2021 [cited 2021 Oct 18]. 31 p. Available from: Available from: https://cdn.who.int/media/docs/default-source/immunization/strategy/ia2030/ia2030_frameworkforactionv04.pdf?sfvrsn=e5374082_1&download=true
» https://cdn.who.int/media/docs/default-source/immunization/strategy/ia2030/ia2030_frameworkforactionv04.pdf?sfvrsn=e5374082_1&download=true - 4GBD 2020, Release 1, Vaccine Coverage Collaborators. Measuring routine childhood vaccination coverage in 204 countries and territories, 1980-2019: a systematic analysis for the Global Burden of Disease Study 2020, Release 1. Lancet. 2021;398(10299):503-21. doi: 10.1016/s0140-6736(21)00984-3
» https://doi.org/10.1016/s0140-6736(21)00984-3 - 5Llau AF, Williams ML, Tejada CE. National vaccine coverage trends and funding in Latin America and the Caribbean. Vaccine. 2021;39(2):317-23. doi: 10.1016/j.vaccine.2020.11.059
» https://doi.org/10.1016/j.vaccine.2020.11.059 - 6World Health Organization. At least 80 million children under one at risk of diseases such as diphtheria, measles and polio as COVID-19 disrupts routine vaccination efforts, warn Gavi, WHO and UNICEF [Internet]. Geneva: World Health Organization ; 2020 [cited 2022 Mar 4]. Available from: Available from: https://www.who.int/news-room/detail/22-05-2020-at-least-80-million-children-under-one-at-risk-of-diseases-such-as-diphtheria-measles-and-polio-as-covid-19-disrupts-routine-vaccination-efforts-warn-gavi-who-and-unicef
» https://www.who.int/news-room/detail/22-05-2020-at-least-80-million-children-under-one-at-risk-of-diseases-such-as-diphtheria-measles-and-polio-as-covid-19-disrupts-routine-vaccination-efforts-warn-gavi-who-and-unicef - 7Domingues CMAS, Fantinato FFST, Duarte E, Garcia LP. Vacina Brasil e estratégias de formação e desenvolvimento em imunizações. Epidemiol Serv Saude [Internet]. 2019;28(2):e20190223. doi: 10.5123/S1679-49742019000200024
» https://doi.org/10.5123/S1679-49742019000200024 - 8Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Programa Nacional de Imunizações 30 anos [Internet]. Brasília: Ministério da Saúde; 2003 [citado 2022 Mar 15]. 208 p. Disponível em: Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/livro_30_anos_pni.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/livro_30_anos_pni.pdf - 9Domingues CMAS, Maranhão AGK, Teixeira AM, Fantinato FFS, Domingues RAS. 46 anos do Programa Nacional de Imunizações: uma história repleta de conquistas e desafios a serem superados. Cad Saude Publica. 2020;36(Supl 2):e00222919. oi: 10.1590/0102-311X00222919
» https://doi.org/10.1590/0102-311X00222919 - 10Sato APS. Qual a importância da hesitação vacinal na queda das coberturas vacinais no Brasil? Rev Saude Publica. 2018;52(1):96. doi: 10.11606/S1518-8787.2018052001199
» https://doi.org/10.11606/S1518-8787.2018052001199 - 11Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Coordenação Geral de Desenvolvimento da Epidemiologia em Serviços. Guia de Vigilância em Saúde. 3. ed. Brasília: Ministério da Saúde, 2019. 740 p.
- 12Organização Pan-Americana da Saúde. Organização Mundial da Saúde. 29ª Conferência Sanitária Pan-Americana. 69ª Sessão do Comitê Regional da OMS para as Américas; 2017 Sep 25-29 [cited 2022 Jun 28]; Washington, DC. Disponível em: Disponível em: https://iris.paho.org/bitstream/handle/10665.2/34446/CSP29-8-p.pdf?sequence=4&isAllowed=y
» https://iris.paho.org/bitstream/handle/10665.2/34446/CSP29-8-p.pdf?sequence=4&isAllowed=y - 13Parra CM, Ribeiro MAL, Bezerra IMP, Ribeiro MR, Abreu LC. Vaccine coverage and measles incidence in Northern Brazil. J Hum Growth Dev. 2022;32(1):21-9. doi: 10.36311/jhgd.v32.12617
» https://doi.org/10.36311/jhgd.v32.12617 - 14Pacheco FC, França GVA, Elidio GA, Domingues CMAS, Oliveira C, Guilhem DB. Trends and spatial distribution of MMR vaccine coverage in Brazil during 2007-2017. Vaccine. 2019;37(20):2651-5. doi: 10.1016/j.vaccine.2019.04.019
» https://doi.org/10.1016/j.vaccine.2019.04.019 - 15Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Articulação Estratégica de Vigilância em Saúde. Guia de Vigilância em Saúde. 5. ed. Brasília: Ministério da Saúde ; 2021 [citado 2021 Ago 2]. 1128 p. Disponível em: Disponível em: https://www.gov.br/saude/pt-br/centraisde-conteudo/publicacoes/publicacoes-svs/vigilancia/guia-de-vigilancia-emsaude_5ed_21nov21_isbn5.pdf/view
» https://www.gov.br/saude/pt-br/centraisde-conteudo/publicacoes/publicacoes-svs/vigilancia/guia-de-vigilancia-emsaude_5ed_21nov21_isbn5.pdf/view - 16Fundação Maria Cecilia Souto Vidigal. Desigualdades e impactos da covid-19 na atenção à primeira infância. São Paulo: Fundação Maria Cecilia Souto Vidigal; 2022 [citado 2023 Fev 4]. Disponível em: Disponível em: https://www.fmcsv.org.br/pt-BR/biblioteca/impactos-covid-2022/
» https://www.fmcsv.org.br/pt-BR/biblioteca/impactos-covid-2022/ - 17Ministério da Saúde (BR). Departamento de Informática do SUS (DATASUS). Imunizações - Cobertura - Brasil [Internet]. Brasília: Ministério da Saúde; 2022 [citado 2022 Jul 12]. Disponível em: Disponível em: http://tabnet.datasus.gov.br/cgi/dhdat.exe?bd_pni/cpnibr.def
» http://tabnet.datasus.gov.br/cgi/dhdat.exe?bd_pni/cpnibr.def - 18Ministério da Saúde (BR). Departamento de Informática do SUS (DATASUS). Nascidos Vivos - Brasil [Internet]. Brasília: Ministério da Saúde ; 2022 [citado 2022 Jun 5]. Disponível em: Disponível em: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def
» http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def - 19Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Coordenação-Geral do Programa Nacional de Imunizações. Informe técnico de introdução da vacina tetra viral: Vacina sarampo, caxumba, rubéola e varicela (atenuada) [Internet]. Brasília: Ministério da Saúde ; 2013 [citado 2021 Nov 30]. Disponível em: Disponível em: http://www.sopape.com.br/data/conteudo/arquivos/informe_tecnico_introducao_vacina_tetraviral.pdf
» http://www.sopape.com.br/data/conteudo/arquivos/informe_tecnico_introducao_vacina_tetraviral.pdf - 20Lima MM, Favacho ARM, Souza-Santos R, Gama SGN. Características e tendência temporal das taxas de mortalidade de crianças e adolescentes em Mato Grosso e no Brasil, 2009 a 2020. Epidemiol Serv Saude. 2022;31(3):e2022491. doi: 10.1590/S2237-96222022000300017
» https://doi.org/10.1590/S2237-96222022000300017 - 21Aragão CMC, Mascarenhas MDM. Tendência temporal das notificações de lesão autoprovocada em adolescentes no ambiente escolar, Brasil, 2011-2018. Epidemiol Serv Saude. 2022;31(1):e202820. doi:10.1590/S1679-49742022000100028
» https://doi.org/10.1590/S1679-49742022000100028 - 22QGIS: Geographic Information System. Open Source Geospatial Foundation Project [Software]. [s.l.]: QGIS; 2021 [cited 2023 Feb 4]. Available from: Available from: http://qgis.osgeo.org
» http://qgis.osgeo.org - 23Muhoza P, Danovaro-Holliday MC, Diallo MS, Murphy P, Sodha SV, Requejo JH, et al. Routine Vaccination Coverage - Worldwide, 2020. MMWR Morb Mortal Wkly Rep. 2021;70(43):1495-500. doi: 10.15585/mmwr.mm7043a1
» https://doi.org/10.15585/mmwr.mm7043a1 - 24Braz RM, Domingues CMAS, Teixeira AMS, Luna EJA. Classificação de risco de transmissão de doenças imunopreveníveis a partir de indicadores de coberturas vacinais nos municípios brasileiros. Epidemiol Serv Saude. 2016;25(4):745-54. doi: 10.5123/S1679-49742016000400008
» https://doi.org/10.5123/S1679-49742016000400008 - 25Oliveira GS, Bitencourt EL, Amaral PFF, Vaz GP, Reis Júnior PM, Costa SB. Cobertura vacinal: uma análise comparativa entre os estados da Região Norte do Brasil. Rev Pat Tocantins. 2020;7(1):14-7. doi: 10.20873/uft.2446-6492.2020v7n1p14
» https://doi.org/10.20873/uft.2446-6492.2020v7n1p14 - 26Veljkovic M, Loncarevic G, Kanazir M, Kisic-Tepavcevic D, Gazibara T. Trend in mandatory immunisation coverage: linear and joinpoint regression approach, Serbia, 2000 to 2017. Euro Surveill. 2021;26(26):2000417. doi: 10.2807/1560-7917.ES.2021.26.26.2000417
» https://doi.org/10.2807/1560-7917.ES.2021.26.26.2000417 - 27Nações Unidas Brasil. Análise da ONU mostra ligação entre falta de equidade de vacinas e aumento da desigualdade [Internet]. Brasília: Nações Unidas Brasil; 2022 [atualizado 2022 Mar 28; citado 2023 Fev 4]. Disponível em: Disponível em: https://brasil.un.org/pt-br/176045-analise-da-onu-mostra-ligacao-entre-falta-de-equidade-de-vacinas-e-aumento-da-desigualdade
» https://brasil.un.org/pt-br/176045-analise-da-onu-mostra-ligacao-entre-falta-de-equidade-de-vacinas-e-aumento-da-desigualdade - 28VanderEnde K, Gacic-Dobo M, Diallo MS, Conklin LM, Wallace AS. Routine Vaccination Coverage - Worldwide, 2027. MMWR Morb Mortal Wkly Rep. 2018;67(1):1261-4. doi: 10.15585/mmwr.mm6745a2
» https://doi.org/10.15585/mmwr.mm6745a2 - 29Danovaro-Holliday MC, Kretsinger K, Gacic-Dobo M. Measuring and ensuring routine childhood vaccination coverage. Lancet. 2021;398(10299):468-9. doi: 10.1016/S0140-6736(21)01228-9
» https://doi.org/10.1016/S0140-6736(21)01228-9
ASSOCIATED ACADEMIC WORK
This article is derived from the Ph.D. thesis entitled Triple viral vaccine dropout and vaccination coverage and associated factors: A spatial and temporal approach, written by Lívia de Lima Moura, within the Postgraduate Program in Public Health Epidemiology, at the Escola Nacional de Saúde Pública Sergio Arouca/Fundação Instituto Oswaldo Cruz (ENSP/Fiocruz), in Rio de Janeiro, in November 2022, the defense of which is planned to take place in 2024.
Publication Dates
- Publication in this collection
20 Oct 2023 - Date of issue
2023
History
- Received
01 Mar 2023 - Accepted
05 July 2023