Evaluation of the National Immunization Program Surveillance System - Vaccination Record Module, Brazil, 2017

Silva, Aline Almeida da; Teixeira, Antonia Maria da Silva; Domingues, Carla Magda Allan Santos; Braz, Rui Moreira; Cabral, Cibelle Mendes

doi:10.1590/s1679-49742021000100028

Abstract

Objetivo

To evaluate the National Immunization Program Immunization Surveillance System, based on its Vaccination Record module, for Brazil in 2017.

Methods

This was a descriptive study using the Guidelines for Evaluating Public Health Surveillance Systems, published by the Centers for Disease Control and Prevention (CDC/Atlanta/GA/United States) to evaluate the attributes of simplicity, flexibility, data quality, sensitivity, timeliness and usefulness of the system for six vaccines on the child immunization schedule.

Results

The Immunization Surveillance System was considered complex in its description; flexible to changes in the immunization schedule; of poor data quality for the DTP and rotavirus vaccines; regular acceptability; high sensitivity for the BCG vaccine; untimely for the hepatitis B vaccine and useful for the purposes of the National Immunization Program.

Conclusion

The data quality, acceptability and timeliness results were not satisfactory, so that actions are needed to enhance the information system.

Keywords:
Immunization; National Immunization Program; Public Health Surveillance; Program Evaluation; Information Systems; Data Accuracy

Introduction

The mission of the National Immunization Program (PNI) is to organize the national immunization policy in Brazil, i.e., structuring and coordinating immunization actions, monitoring immunobiologicals and their effects on the population, in order to reduce morbidity and mortality due to vaccine-preventable diseases.¹1. Domingues CMAS, Teixeira AMS. Coberturas vacinais e doenças imunopreveníveis no Brasil no período 1982-2012: avanços e desafios do Programa Nacional de Imunizações. Epidemiol Serviços Saúde [Internet]. 2013 jan-mar [citado 2018 set 6];22(1):9-27. Disponível em: http://dx.doi.org/10.5123/S1679-49742013000100002
https://doi.org/10.5123/S1679-4974201300...

2. Ministério da Saúde (BR). Programa nacional de imunizações: 40 anos [Internet]. Brasília: Ministério da Saúde; 2013 [citado 2018 mar 29]. Disponível em: https://www.saude.gov.br/svs
https://www.saude.gov.br/svs... ^-³3. Temporão JG. O Programa Nacional de Imunizações (PNI): origens e desenvolvimento. Hist Ciênc Saúde Manguinhos [Internet]. 2003 [citado 2020 dez 1];10(suppl 2):601-17. Disponível em: https://doi.org/10.1590/S0104-59702003000500008
https://doi.org/10.1590/S0104-5970200300...

The Immunization Surveillance System’s information management is performed by several information systems, encompassing other technical areas, both within and outside the Ministry of Health. The National Immunization Program Information System (SI-PNI),⁴4. Ministério da Saúde (BR). Manual do Sistema de Informação do Programa Nacional de Imunizações: SI-PNI. Brasília: Ministério da Saúde ; 2016. uses three main modules to manage immunization data: Vaccination Record, listing immunization coverage reports and lists of no-shows, among other data; Post-Vaccination Adverse Events (EAPV); and Immunobiological Mobilization (MI). Besides these modules, SI-PNI provides aggregated information on multivaccination strategies, rapid vaccination coverage monitoring and campaigns. SI-PNI aims to (i) unify parallel systems, (ii) promote their intercommunication, and (iii) provide individualized data on the vaccinated population.⁵5. Silva BS, Coelho HV, Cavalcante RB, Oliveira VC, Guimarães EAA. Estudo de avaliabilidade do Sistema de Informação do Programa Nacional de Imunização. Rev Bras Enferm [Internet]. 2018 [citado 2020 dez 1];71(supl 1):660-9. Disponível em: https://doi.org/10.1590/0034-7167-2017-0601
https://doi.org/10.1590/0034-7167-2017-0... These individualized data on administered doses are found in the Vaccination Record module³3. Temporão JG. O Programa Nacional de Imunizações (PNI): origens e desenvolvimento. Hist Ciênc Saúde Manguinhos [Internet]. 2003 [citado 2020 dez 1];10(suppl 2):601-17. Disponível em: https://doi.org/10.1590/S0104-59702003000500008
https://doi.org/10.1590/S0104-5970200300... (Figure 1).

Thumbnail

Figure 1
Migration flow of the several parallel systems of the National Immunization Program, Brazil, 2017

In order to monitor the actions of the Immunization Surveillance System, adherence to SI-PNI and regular and qualified information input by municipalities is required. Municipalities that use their own systems must also inform data on SI-PNI. The information on immunization data is provided by the services that perform this activity, regardless of their public or private nature.⁶6. Ministério da Saúde (BR). Agência Nacional de Vigilância Sanitária - Anvisa. Resolução da Diretoria Colegiada RDC nº 197, de 26 de dezembro de 2017. Dispõe sobre os requisitos mínimos para o funcionamento dos serviços de vacinação humana [Internet]. Brasília: Anvisa, 2017 [citado 2020 dez 1]. Disponível em: https://sbim.org.br/legislacao/867-rdc-anvisa-n-197-26-de-dezembro-de-2017
https://sbim.org.br/legislacao/867-rdc-a...

To encourage municipalities to join SI-PNI, two ordinances published by the Ministry of Health are worth highlighting: GM/MS Ordinance No. 2.363, dated October 18, 2012, which established the National Health Fund financial transfer to states and municipalities for the purchase of computer equipment; and GM/MS Ordinance No. 2.984, dated December 27, 2016, which refers to the Health Surveillance Actions Qualification Program goals and indicators. GM/MS Ordinance No. 2.984/2016 assigned two indicators to the PNI, one of which defines the following goal: ≥80% of municipal vaccine rooms providing monthly input to SI-PNI, which is exactly the object of this study.⁷7. Brasil. Ministério da Saúde. Portaria MS/GM nº 2.363, de 18 de outubro de 2012. Institui repasse financeiro do Fundo Nacional de Saúde aos Fundos de Saúde dos Estados, Distrito Federal e Municípios, por meio do Piso Variável de Vigilância e Promoção da Saúde, para fomento na implantação do Sistema de Informação do Programa Nacional de Imunizações (SI-PNI) e Sistema de Informação de Agravos de Notificação (SINAN), no âmbito das unidades de saúde [Internet]. Diário Oficial da União, Brasília (DF), 2012 out 19 [citado 2018 ago 29]; Seção 1:24. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt2363_18_10_2012.html
http://bvsms.saude.gov.br/bvs/saudelegis... ^,⁸8. Brasil. Ministério da Saúde. Portaria MS/GM nº 2.984, de 27 de dezembro de 2016. Revisa a relação de metas e seus respectivos indicadores do Programa de Qualificação das Ações de Vigilância em Saúde (PQA-VS) a partir de 2017 [Internet]. Diário Oficial da União , Brasília (DF), 2016 dez 28 [citado 2018 jun 6]; Seção 1:109. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2016/ prt2984_27_12_2016.html
http://bvsms.saude.gov.br/bvs/saudelegis...

The need to identify the shortcomings and the potential of the Immunization Surveillance System so as to improve information on immunization in Brazil, the shortage of publications regarding SI-PNI and the absence of specific analysis, requires an understanding of the Vaccination Record module, how it works, its consistency and possibilities for improvement.

This study aimed to evaluate the Immunization Surveillance System based on an analysis of the National Immunization Program Information System Vaccine Record module for 2017.

Methods

A descriptive-evaluative study of the Immunization Surveillance System was conducted, using secondary data from the SI-PNI Vaccination Record module reports, assessed at all levels of the Brazilian National Health System (SUS), considering the 27 Federative Units (UF), the 5,570 municipalities and the 35,458 active vaccine rooms in the year 2017, adopting, as theoretical reference, the Guidelines for Evaluating Public Health Surveillance Systems, issued by the United States Centers for Disease Control and Prevention (CDC, Atlanta, GA, USA).⁹9. Centers for Disease Control and Prevention - CDC. Updated guidelines for evaluating public health surveillance systems [Internet]. Washington, D.C.: CDC; 2001 [cited 2018 Mar 29]. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm
https://www.cdc.gov/mmwr/preview/mmwrhtm...

The evaluation criteria for the Immunization Surveillance System were its qualitative attributes - simplicity, flexibility, data quality, acceptability - and its quantitative attributes - sensitivity, timeliness and completeness - explained as follows:

Simplicity
Subjective evaluation, through description of the Immunization Surveillance System, with the final classification of this attribute as being simple or complex.
Flexibility
Adequacy of the Immunization Surveillance System (i) for changes in the age range and target group of the human papillomavirus (HPV) vaccine, (ii) for the yellow fever vaccination scheme and (iii) for intercommunicability between the Vaccination Record and Post-Vaccination Adverse Event modules. Changes in the system were used as criteria according to changes in the schedule (‘Yes’ or ‘No’) and the Business Rule. The latter was considered adequate or inadequate, according to interoperability or lack of it between the modules.
This attribute’s final classification was either flexible (when scoring three or more positive criteria) or poor flexibility (when scoring less than three positive criteria).
Data quality
To evaluate the Immunization Surveillance System’s completeness, the ‘Immunized by Vaccine’ report for hepatitis B and its variables ‘mother’s name’, ‘address’, ‘phone number’ and ‘vaccine batch’ were used. This attribute’s classification scale was defined as follows: ≤5% blank, Good completeness; and >5% for poor completeness. For data inconsistency, three system reports were used: ‘Recording Errors’, which sought to identify only doses of diphtheria, tetanus and whooping cough (pertussis) (DTP) vaccine administered outside the prescribed age range; ‘List of No-shows’ for rotavirus vaccination, in which, in the variable ‘days of delay’, only >30 day delays were investigated; and ‘Immunized by Vaccine’, for the 1^st and 2^nd doses of rotavirus vaccine, in which the field ‘Inadvertent Dose’ equal to ‘Yes’ was examined for doses outside the prescribed age range. The Business Rule was classified as adequate or inadequate.
Final classification of data quality was as follows: high, moderate or low, based on the series of combinations between completeness and inconsistency (Figure 2).
Acceptability
Adherence of municipal vaccine rooms to SI-PNI was evaluated, taking into account the above mentioned ministerial ordinances.
The final rating for this attribute was: excellent, for ≥80% of vaccine rooms using the system; regular, between ≥50 and <80%; or poor, when <50% of vaccine rooms used SI-PNI.
Sensitivity
Children under one year of age receiving BCG (Bacillus Calmette-Guérin) vaccination in 2017 were considered in relation to live births registered on the Live Birth Information System (Sinasc) in 2016.
The criterion used for this attribute’s classification was the following: high sensitivity, when ≥80% of children of this age were vaccinated with BCG; or low sensitivity, when <80% of those children were immunized by the vaccine.
Timeliness
Hepatitis B vaccination timeliness within one day of life, in relation to the doses administered up to 30 days of life.
The criterion used in this classification was the following: timely system, ≥80% of the doses taken within one day of life; or untimely system, <80% of the doses administered within that time.
Usefulness
Aiming at estimating the accomplishment of the Immunization Surveillance System’s purpose, this attribute’s evaluation was based on the construction of important data for Public Health guidance and decision making. Among other aspects, usefulness considers the usability and effects of systems in decisions on public policies.⁹9. Centers for Disease Control and Prevention - CDC. Updated guidelines for evaluating public health surveillance systems [Internet]. Washington, D.C.: CDC; 2001 [cited 2018 Mar 29]. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm
https://www.cdc.gov/mmwr/preview/mmwrhtm... Although it is a component of systems assessment, usefulness may be influenced by attributes used in this assessment methodology.

Thumbnail

Figure 2
Final classification of the data quality attribute, according to the combination of completeness and inconsistency results

To evaluate the Immunization Surveillance System’s completeness and timeliness, given the errors observed in exporting the reports from the state capitals, the most populous municipality in each state was used as a replacement criterion, according to the Brazilian Institute of Geography and Statistics (IBGE), and so on, until a report was exported. For the states of Bahia, Ceará, Goiás, Mato Grosso do Sul, Minas Gerais, Pará, Paraná, Santa Catarina and São Paulo, the following municipalities were used as a reference, respectively: Feira de Santana (BA), Caucaia (CE), Aparecida de Goiânia (GO), Dourados (MS), Contagem (MG), Ananindeua (PA), Londrina (PR), Joinville (SC) and Santo André (SP).

The following vaccines were analyzed, with their selection criteria:¹⁰10. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Manual de normas e procedimentos para vacinação [Internet]. Brasília: Ministério da Saúde ; 2014 [citado 2017 jul 3]. 176 p. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf
http://bvsms.saude.gov.br/bvs/publicacoe...

11. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Nota informativa nº 94, de 2017/CGPNI/DEVIT/SVS/MST. Orientações e indicação de dose única da febre amarela [Internet]. Brasília: Ministério da Saúde ; 2017 [citado 2020 dez 1]. Disponível em: https://www.saude.mg.gov.br/images/noticias_e_eventos/000_2018/01-jan-fev-marc-abril/Boletins_AEDES/Nota Informativa 94 Com de Acordo 005.pdf
https://www.saude.mg.gov.br/images/notic...

12. Ministério da Saúde (BR). Nota informativa no 62-SEI/2017-CGPNI/DEVIT/SVS/MS. Orienta os serviços de vacinação para a otimização do uso da vacina HPV quadrivalente, com ampliação temporária da faixa etária [Internet]. Brasília: Ministério da Saúde ; 2017 [citado 2020 dez 1]. Disponível em: https://www.saude.gov.br/images/pdf/2017/agosto/18/SEI_MS-0290791-Nota-Informativa.pdf
https://www.saude.gov.br/images/pdf/2017... ^-¹³13. Teixeira AMS, Silva AA, Silva AF, Domingues CMAS, Locádio ES, Fantinato FFST, Souza LRO, et al. Avaliação dos indicadores de desempenho da vacinação do Programa Nacional de Imunizações e os desafios para elevar as coberturas vacinais no Brasil. In: Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Saúde Brasil 2019: uma análise da situação de saúde com enfoque nas doenças imunopreveníveis e na imunização [Internet]. Brasília: Ministério da Saúde; 2019 [citado 2020 dez 1]. p. 370-486. Disponível em: https://portalarquivos2.saude.gov.br/images/pdf/2019/dezembro/05/Saude-Brasil-2019-imunizacao.pdf
https://portalarquivos2.saude.gov.br/ima...

BCG and hepatitis B vaccines: indicated at birth
BCG is considered the child’s ‘gateway’ to the Immunization Surveillance System and is recommended up to <5 years of age.
Hepatitis B vaccine is indicated up to 30 days of life; preferably on the first day, while still in the maternity ward, to reduce vertical transmission infection risk.
b) Human rotavirus and DTP vaccines: age-range restriction for vaccination
Human rotavirus vaccine is indicated at 2 and 4 months of life, with deadlines:
1^st dose - to be administered in the period of ≥1 month and 15 days to ≤3 months and 15 days; and
2^nd dose - to be administered in the period of ≥3 months and 15 days to ≤7 months and 29 days.
These deadlines are to be strictly met, given the risk of intestinal intussusception.
DTP vaccine administration is indicated within ≥2 months to <7 years, due to the pertussis component, which can cause aggravation of adverse events. This vaccine, used as a booster for the vaccine known as pentavalent vaccine (DTP + Haemophilus influenzae B + hepatitis B), is recommended at 15 months of life and 4 years of age.
c) Yellow fever and HPV vaccines: changes in the immunization schedule in 2017
In the case of the yellow fever vaccine, until 2016, the regimen of one dose and one booster every ten years was followed. With effect from 2017, the following is prescribed:
Single dose - for life; and
Fractional Dose - temporary strategy, in some priority municipalities.

The change in the HPV vaccination routine, in turn, not only expanded the age range of target girls, from nine to 13 years to nine to <15 years. It also included boys from 11 to <15 years.

The study data sources consisted of SI-PNI reports, PNI technical and operational manuals, and Sinasc and IBGE data. Microsoft Excel 2013 was used for data analysis.

Results

Simplicity

The Immunization Surveillance System is a national system, fully decentralized, responsible for immunization effectiveness among the entire population, considering target group specificities, clinical indications and disease prevention and control strategies, according to the epidemiological context (Figure 3). Immunization is configured as a passive (vaccine room spontaneous demand) and active (unvaccinated people seeking it) surveillance action and it therefore involves the all three health service management levels, namely the national, state, regional, municipal and local levels.

Thumbnail

Figura 3
Immunization Surveillance System and vaccination data information flow, from vaccine administration to inputting information on the system

The Immunization Surveillance System evaluates the incidence of vaccine-preventable diseases, their morbidity and mortality, as well as detection of susceptibility, being directly integrated to wider Epidemiological Surveillance.¹⁴14. Waldman EA, Sato APS. Trajetória das doenças infecciosas no Brasil nos últimos 50 anos: um contínuo desafio. Rev Saúde Pública [Internet]. 2016 dez [citado 2020 dez 1]; 50:68. Disponível em: https://doi.org/10.1590/S1518-8787.2016050000232
https://doi.org/10.1590/S1518-8787.20160... In order to decide on changes in the immunization schedule, monitoring of national and international research results is performed. These are discussed by the National Immunization Technical Advisory Commission, an advisory body composed of PNI technicians, researchers and specialists, among others.¹1. Domingues CMAS, Teixeira AMS. Coberturas vacinais e doenças imunopreveníveis no Brasil no período 1982-2012: avanços e desafios do Programa Nacional de Imunizações. Epidemiol Serviços Saúde [Internet]. 2013 jan-mar [citado 2018 set 6];22(1):9-27. Disponível em: http://dx.doi.org/10.5123/S1679-49742013000100002
https://doi.org/10.5123/S1679-4974201300... ^,¹⁵15. Domingues CMAS, Woycicki JR, Rezende KS, Henriques CMP. Programa Nacional de Imunização: a política de introdução de novas vacinas. Rev Eletr Gestão Saúde [Internet]. 2015 out [citado 2020 dez 1]; 6(supl 4):3250-74. Disponível em: https://periodicos.unb.br/index.php/rgs/article/view/3331/3017
https://periodicos.unb.br/index.php/rgs/...

National and international laboratories can be included in the process, aiming at strengthening Brazil’s industrial complex through technology transfers and enabling Brazil to achieve sustainable production of immunobiologicals.¹⁵15. Domingues CMAS, Woycicki JR, Rezende KS, Henriques CMP. Programa Nacional de Imunização: a política de introdução de novas vacinas. Rev Eletr Gestão Saúde [Internet]. 2015 out [citado 2020 dez 1]; 6(supl 4):3250-74. Disponível em: https://periodicos.unb.br/index.php/rgs/article/view/3331/3017
https://periodicos.unb.br/index.php/rgs/...

Once changes to the routine have been incorporated, the Immunization Surveillance System requires there to be adequate infrastructure for receiving immunobiologicals, right from the federal level down to the local vaccine room (storage, inventory monitoring, information recording), in addition to trained personnel for technical support and execution of all processes, including vaccination itself, and the ability to use registration instruments.⁶6. Ministério da Saúde (BR). Agência Nacional de Vigilância Sanitária - Anvisa. Resolução da Diretoria Colegiada RDC nº 197, de 26 de dezembro de 2017. Dispõe sobre os requisitos mínimos para o funcionamento dos serviços de vacinação humana [Internet]. Brasília: Anvisa, 2017 [citado 2020 dez 1]. Disponível em: https://sbim.org.br/legislacao/867-rdc-anvisa-n-197-26-de-dezembro-de-2017
https://sbim.org.br/legislacao/867-rdc-a...

The Immunization Surveillance System features the Daily Vaccination Record Form and its 25 macrofields. Depending on vaccine indication and quantities to be administered, the number of records grows. If the system is installed at a health center, input to it should take place in the vaccine room, preferably with the vaccinated individual and/or their legal guardian present; otherwise, the completed form must be sent for input at another location, according to the local established workflow. Health centers that have their own systems must transfer the data to SI-PNI.¹⁶16. Sato APS. Programa Nacional de Imunização: sistema informatizado como opção a novos desafios. Rev Saúde Pública [Internet]. 2015 jul [citado 2019 fev 6]; 49:39. Disponível em: https://doi.org/10.1590/S0034-8910.2015049005925
https://doi.org/10.1590/S0034-8910.20150...

SI-PNI has 14 main menus and 147 submenus; of these, 81 are reports, of which 59 belong to the Vaccination Record module. All health management levels have access to the system, although only health establishments perform data entry. SI-PNI intercommunication with the National SUS Card for importing user identification data is worthy of note.⁴4. Ministério da Saúde (BR). Manual do Sistema de Informação do Programa Nacional de Imunizações: SI-PNI. Brasília: Ministério da Saúde ; 2016.

The Business Rules, included in this analysis, are a set of criteria defined by the PNI, grounded on technical-scientific information and indications for each immunobiological, thereby serving as a reference for reviewing the system in relation to data input and output. The Rules are in the form of a spreadsheet with 24 columns and 83 rows, for description of vaccines, diluents, serums and immunoglobulins, as well as vaccine administration and cost indications. Each immunobiological has a specific Business Rule, according to the natural history of the disease for which it is used to prevent infection.

Monitoring of the Business Rules is performed by the PNI’s information technical area, routinely and systematically. When errors are identified, the recommended adjustments are requested from the SUS Information Technology Department (Datasus).

Taking into account (i) the various lines of action, (ii) the various professionals and entities involved in these processes, (iii) the status of qualified and trained staff for immunization actions⁶6. Ministério da Saúde (BR). Agência Nacional de Vigilância Sanitária - Anvisa. Resolução da Diretoria Colegiada RDC nº 197, de 26 de dezembro de 2017. Dispõe sobre os requisitos mínimos para o funcionamento dos serviços de vacinação humana [Internet]. Brasília: Anvisa, 2017 [citado 2020 dez 1]. Disponível em: https://sbim.org.br/legislacao/867-rdc-anvisa-n-197-26-de-dezembro-de-2017
https://sbim.org.br/legislacao/867-rdc-a... and (iv) the requirement for continuous and systematic monitoring, the Immunization Surveillance System was classified as complex.

Flexibility

The Immunization Surveillance System, regarding HPV vaccine, proved adequate for the incorporation of a new strategy, by age group (record: ‘Yes’), and for the inclusion of a new target group (record: ‘Yes’). The system also proved to be adequate for the prescription of a new yellow fever vaccine regimen (record: ‘Yes’); however, interoperability of the Vaccination Record module and the Post-Vaccination Adverse Events module was found to be inadequate (record: ‘No’).

Therefore, a final score of three positive criteria was obtained for this attribute, and the system was classified as flexible.

Data quality

When evaluating data recording completeness for ‘mother’s name’, ‘address’ and ‘telephone’, it was found that the report does not export these variables, making it impossible to evaluate this system attribute. In order to analyze the ‘vaccine batch’ variable, 696,593 records were assessed, of which 38,219 (5.5%) were blank, so that data completeness was considered poor.

Regarding inconsistency, 27,787 of the 112,333 records contained in the ‘Recording Errors’ report were found to be within the permitted age range, so that the SI-PNI Business Rule was classified as inadequate in terms of the errors identified in the records.

Of the 3,852 records contained in the ‘List of No-shows’ report, 228 were 30 days or more overdue, and so the Business Rule was once more classified as inadequate.

Of the 4,868 records contained in the ‘Immunized by Vaccine’ report, 2,656 were first dose, with 325 records outside the eligible age range. Of these, 185 had ‘No’ recorded in the ‘Inadvertent Dose’ field. For the second dose (2,212 records), 136 records were found outside the eligible age range and of these, 82 showed had ‘No’ recorded in the ‘Inadvertent Dose’ field. The Business Rule was also classified as inadequate with regard to observation of people immunized by vaccines.

Therefore, the final classification of the data quality attribute was poor.

Acceptability

Of the 5,570 Brazilian municipalities, 569 had less than 50% of their vaccine rooms using SI-PNI; and in 762 municipalities 50 to <80% of vaccine rooms were active. Low adherence to the Immunization Surveillance System in 2017 was mostly observed in the states of Roraima and Rio de Janeiro, and in some municipalities in the Northeast region.

The system was used in ≥80% of the vaccine rooms in 4,239 (76%) municipalities, and its acceptability was classified as regular (Figure 4).

Thumbnail

Figure 4
National Immunization Program Information System acceptability by municipality, considering ≥80% of vaccination rooms using it, Brazil, 2017

Sensitivity

SI-PNI registered 2,708,944 doses of BCG vaccine in children under one year of age in 2017; according to Sinasc, there were 2,857,800 live births in 2016.

As 94.8% of the children in the Sinasc database were covered, the system was classified as highly sensitive.

Timeliness

134,531 doses of hepatitis B vaccine were administered at ≤30 days of life; of these, 106,609 were applied on ≤1 day of life, resulting in 79.2% timeliness.

Therefore, the Immunization Surveillance System was considered untimely (Table 1).

Thumbnail

Table 1
Immunization Program Surveillance System Timeliness, according to age for hepatitis B vaccination, per selected capitals or municipalities in the country, Brazil, 2017

Usefulness

The Immunization Surveillance System was found to be useful in fulfilling its purpose, allowing the identification of the population susceptible to vaccine-preventable diseases through individualized data and management of timely actions in the territory. The system not only provided information on the immunization schedule available in the public network; it also expanded the vaccine supply over the years, covering several age groups, and demonstrated its capacity to group its subsystems, despite still requiring functional adjustments.

Discussion

The Immunization Surveillance System contributes to the control, elimination or eradication of vaccine-preventable diseases in Brazil. Its evaluation, considering the PNI’s National Immunization Schedule’s six vaccines, showed that the system is complex, flexible, of low-quality regarding data on DTP and rotavirus vaccines, of low acceptability, high sensitivity for BCG, untimely for hepatitis B vaccine, and useful, in fulfilling its purpose.

This study found that the Immunization Surveillance System’s complexity is in part due to the need to monitor the epidemiological profile of vaccine-preventable diseases, to the partnership with specialists and researchers, and to the structuring of health services with adequate logistics and infrastructure, skilled and trained professionals to conduct immunization actions.³3. Temporão JG. O Programa Nacional de Imunizações (PNI): origens e desenvolvimento. Hist Ciênc Saúde Manguinhos [Internet]. 2003 [citado 2020 dez 1];10(suppl 2):601-17. Disponível em: https://doi.org/10.1590/S0104-59702003000500008
https://doi.org/10.1590/S0104-5970200300... ^,¹⁶16. Sato APS. Programa Nacional de Imunização: sistema informatizado como opção a novos desafios. Rev Saúde Pública [Internet]. 2015 jul [citado 2019 fev 6]; 49:39. Disponível em: https://doi.org/10.1590/S0034-8910.2015049005925
https://doi.org/10.1590/S0034-8910.20150... Immunization surveillance requires the participation of various sectors and levels of care throughout all its processes. It depends on, results from, and provides a large amount of information, and this makes its profile complex.¹⁷17. Machado Costa MR, Barbosa J. Situação atual da febre amarela no Brasil. Bol Eletr Epidemiol [Internet]. Brasília: Funasa; 2001 [citado 2020 dez 1]. Disponível em: http://www.funasa.gov.br
http://www.funasa.gov.br...

The system was considered flexible as it adapted to changes in the national immunization schedule prior to them being put into operation, thus providing system operators with the time to obtain and correctly input the system variables. According to the US/CDC Guidelines for Evaluating Public Health Surveillance Systems, when a system is complex this tends to make it inflexible, but this was not the case in this evaluation.⁹9. Centers for Disease Control and Prevention - CDC. Updated guidelines for evaluating public health surveillance systems [Internet]. Washington, D.C.: CDC; 2001 [cited 2018 Mar 29]. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm
https://www.cdc.gov/mmwr/preview/mmwrhtm...

The low quality of the data, evidenced by inadequate completeness and consistency, directly impacts data validation, accuracy and reliability and other aspects inherent to the necessary correctness of the information,¹⁸18. Valente NTZ, Fujino A. Atributos e dimensões de qualidade da informação nas ciências contábeis e na ciência da informação: um estudo comparativo. Perspect Ciência Inf [Internet]. 2017 abr-jun [citado 2020 dez 1];21(2):141-67. Disponível em: https://doi.org/10.1590/1981-5344/2530
https://doi.org/10.1590/1981-5344/2530... ^,¹⁹19. Lima CRA, Schramm JMA, Coeli CM, Silva MEM. Revisão das dimensões de qualidade dos dados e métodos aplicados na avaliação dos sistemas de informação em saúde. Cad Saúde Pública [Internet]. 2009 out [citado 2020 dez 1];25(10):2095-109. Disponível em: https://www.scielo.br/pdf/csp/v25n10/02.pdf
https://www.scielo.br/pdf/csp/v25n10/02.... and can interfere in the evaluation of the actual health situation and also in the feasibility of comparison with other databases.²⁰20. Correia LO, Padilha BM, Vasconcelos SML. Métodos para avaliar a completitude dos dados dos sistemas de informação em saúde do Brasil: uma revisão sistemática. Ciênc Saúde Coletiva [Internet]. 2014 [citado 2020 dez 1];19(11):4467-78. Disponível em: https://www.scielo.br/pdf/csc/v19n11/1413-8123-csc-19-11-4467.pdf
https://www.scielo.br/pdf/csc/v19n11/141... Regarding low completeness of the ‘vaccine batch’ variable, this finding may be related to the fact that it is not a mandatory variable, although its completion could assist in outbreak and/or post-vaccination adverse event identification.²¹21. Luhm KR, Waldman EA. Sistemas informatizados de registro de imunização: uma revisão com enfoque na saúde infantil. Epidemiol Serv Saúde [Internet]. 2009 jan-mar [citado 2020 dez 1];18(1):65-78. Disponível em: http://scielo.iec.gov.br/pdf/ess/v18n1/v18n1a07.pdf
http://scielo.iec.gov.br/pdf/ess/v18n1/v...

First of all, the importance of training the professionals involved in feeding the data into the information system, thus contributing to the strengthening of municipal operational capacity, must be stressed. It is also important to provide continuous training for Information Technology specialists, in keeping with their work with the Business Rules, in analyzing the data input to and exported from the information system.⁵5. Silva BS, Coelho HV, Cavalcante RB, Oliveira VC, Guimarães EAA. Estudo de avaliabilidade do Sistema de Informação do Programa Nacional de Imunização. Rev Bras Enferm [Internet]. 2018 [citado 2020 dez 1];71(supl 1):660-9. Disponível em: https://doi.org/10.1590/0034-7167-2017-0601
https://doi.org/10.1590/0034-7167-2017-0... ^,²⁰20. Correia LO, Padilha BM, Vasconcelos SML. Métodos para avaliar a completitude dos dados dos sistemas de informação em saúde do Brasil: uma revisão sistemática. Ciênc Saúde Coletiva [Internet]. 2014 [citado 2020 dez 1];19(11):4467-78. Disponível em: https://www.scielo.br/pdf/csc/v19n11/1413-8123-csc-19-11-4467.pdf
https://www.scielo.br/pdf/csc/v19n11/141...

As for low acceptability, municipalities that do not adhere to the information system can have difficulties in managing their information, mostly in evaluating the impact of surveillance actions, and in achieving indicator targets.²²22. Teixeira AMS, Mota ELA. Denominators for vaccine coverage estimates: a database study to estimate the population less than one year of age. Epidemiol Serv Saúde [Internet]. 2010 Jul-Sep [cited 2019 Feb 12];19(3):187-203. Available from: http://dx.doi.org/10.5123/S1679-49742010000300002
https://doi.org/10.5123/S1679-4974201000... However, it is important to highlight the different realities existing for health service network accessibility, especially in places in Brazil that are hard to access, which can influence this attribute’s performance.²³23. Ministério da Saúde (BR). Organização Pan-Americana da Saúde - OPAS. Fundação Oswaldo Cruz - Fiocruz. A experiência brasileira em sistemas de informação em saúde: falando sobre os sistemas de informação em saúde no Brasil [Internet]. Brasília: Ministério da Saúde ; 2009 [citado 2020 dez 1]. 2 v. (Série B. Textos Básicos de Saúde). Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/experiencia_brasileira_sistemas_saude_volume2.pdf
https://bvsms.saude.gov.br/bvs/publicaco...

The system showed itself to be sensitive, considering that immunization surveillance covered a satisfactory percentage of children under one year old, and which was above the reference value for BCG vaccination at birth. This sensitivity is attributed, albeit partially, to vaccination being recommended when the child is still in the maternity ward, and if it is not vaccinated at birth then sensitivity can also be attributed to the possibility of their being called subsequently for vaccination.¹⁰10. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Manual de normas e procedimentos para vacinação [Internet]. Brasília: Ministério da Saúde ; 2014 [citado 2017 jul 3]. 176 p. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf
http://bvsms.saude.gov.br/bvs/publicacoe...

Although the calculation used for sensitivity is the same for vaccination coverage, the sensitivity rate in this study differs from the coverage rate. The US/CDC Guidelines for Evaluating Public Health Surveillance Systems enable sensitivity to be considered on two levels: firstly, the proportion of events captured by surveillance, and secondly, the ability to identify outbreaks or monitor changes across time.⁹9. Centers for Disease Control and Prevention - CDC. Updated guidelines for evaluating public health surveillance systems [Internet]. Washington, D.C.: CDC; 2001 [cited 2018 Mar 29]. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm
https://www.cdc.gov/mmwr/preview/mmwrhtm... Within this framework, the calculation proposed in this study is seen to be adequate as it identifies the percentage of children covered by vaccination compared to the number of live births.

The divergence in BCG vaccination uptake outcome and in the timeliness of hepatitis B vaccination should be highlighted, since both vaccines are recommended simultaneously.¹⁰10. Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Manual de normas e procedimentos para vacinação [Internet]. Brasília: Ministério da Saúde ; 2014 [citado 2017 jul 3]. 176 p. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... The system’s lack of timeliness in immunizing against hepatitis B draws attention to the need for a joint assessment of BCG vaccination timeliness, seeking to identify the susceptibility of children to these vaccine-preventable diseases in the first days of life. The maximum vaccination periods for both vaccines should be considered.

The lack of timeliness of the Immunization Surveillance System needs to be discussed jointly with state and municipal health service management, in order for the issue to be solved, in the sense of decentralizing the procedure to the largest number of maternity hospitals, or, if necessary, training hospital care personnel to solve the issue.

The Immunization Surveillance System proved to be useful, because it fulfills its purposes, allows weaknesses and potentialities to be identified, and thus contributes to the improvement of its quality, especially with solid technical information to support decision making.

This evaluation component is directly associated with the relevance of its purpose, in the direct relationship between what is expected and what is detected in the information produced by the system.¹⁸18. Valente NTZ, Fujino A. Atributos e dimensões de qualidade da informação nas ciências contábeis e na ciência da informação: um estudo comparativo. Perspect Ciência Inf [Internet]. 2017 abr-jun [citado 2020 dez 1];21(2):141-67. Disponível em: https://doi.org/10.1590/1981-5344/2530
https://doi.org/10.1590/1981-5344/2530... ^,²¹21. Luhm KR, Waldman EA. Sistemas informatizados de registro de imunização: uma revisão com enfoque na saúde infantil. Epidemiol Serv Saúde [Internet]. 2009 jan-mar [citado 2020 dez 1];18(1):65-78. Disponível em: http://scielo.iec.gov.br/pdf/ess/v18n1/v18n1a07.pdf
http://scielo.iec.gov.br/pdf/ess/v18n1/v... ^,²⁴24. Tauil MC, Sato APS, Costa ÂA, Inenami M, Ferreira VLR, Waldman EA. Vaccination coverage according to doses received and timely administered based on an electronic immunization registry, Araraquara-SP, Brazil, 2012-2014. Epidemiol Serv Saúde [Internet]. 2017 Oct-Dec [cited 2020 Dez 1];26(4):835-46. Available from: https://doi.org/10.5123/S1679-49742017000400014
https://doi.org/10.5123/S1679-4974201700...

Among the study’s main limitations are (i) the use of secondary and population data from Sinasc 2016, to estimate indicators of vaccination conducted in 2017, which can underestimate or overestimate the data, (ii) consulting large databases, making it impossible to export reports which would enable analysis of a smaller number of records, and (iii) the non-use of the National Immunization Program’s Information System (SI-PNI) by some municipalities and, as a consequence, the smaller volume of records for analysis.

Although the Immunization Surveillance System is complex in structure, flexible, with low data quality for DTP and rotavirus vaccines, low acceptability, high sensitivity for BCG vaccine and untimely for hepatitis B vaccine, its evaluation should be done on a regular basis, to identify possible process failures, thus preventing their possible impact on PNI performance.

Over the years, PNI has expanded to provide several immunobiologicals.¹³13. Teixeira AMS, Silva AA, Silva AF, Domingues CMAS, Locádio ES, Fantinato FFST, Souza LRO, et al. Avaliação dos indicadores de desempenho da vacinação do Programa Nacional de Imunizações e os desafios para elevar as coberturas vacinais no Brasil. In: Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Saúde Brasil 2019: uma análise da situação de saúde com enfoque nas doenças imunopreveníveis e na imunização [Internet]. Brasília: Ministério da Saúde; 2019 [citado 2020 dez 1]. p. 370-486. Disponível em: https://portalarquivos2.saude.gov.br/images/pdf/2019/dezembro/05/Saude-Brasil-2019-imunizacao.pdf
https://portalarquivos2.saude.gov.br/ima... This noteworthy progress requires that new management designs should be prepared for PNI, based on more consistent and timely information. Preparation of joint strategies together with the Federative Units can contribute to identifying and untying critical immunization knots, in order to improve Immunization Surveillance System planning, organization and coordination.⁵5. Silva BS, Coelho HV, Cavalcante RB, Oliveira VC, Guimarães EAA. Estudo de avaliabilidade do Sistema de Informação do Programa Nacional de Imunização. Rev Bras Enferm [Internet]. 2018 [citado 2020 dez 1];71(supl 1):660-9. Disponível em: https://doi.org/10.1590/0034-7167-2017-0601
https://doi.org/10.1590/0034-7167-2017-0... ^,¹⁹19. Lima CRA, Schramm JMA, Coeli CM, Silva MEM. Revisão das dimensões de qualidade dos dados e métodos aplicados na avaliação dos sistemas de informação em saúde. Cad Saúde Pública [Internet]. 2009 out [citado 2020 dez 1];25(10):2095-109. Disponível em: https://www.scielo.br/pdf/csp/v25n10/02.pdf
https://www.scielo.br/pdf/csp/v25n10/02....

Accordingly, it is recommended that the PNI National Coordination reorganize the information system’s processes and flows, aiming to increase adherence to the Immunization Surveillance System, improve data quality, increase timely recruitment of individuals for vaccination, and improve the integration of SI-PNI with other systems’ databases.⁵5. Silva BS, Coelho HV, Cavalcante RB, Oliveira VC, Guimarães EAA. Estudo de avaliabilidade do Sistema de Informação do Programa Nacional de Imunização. Rev Bras Enferm [Internet]. 2018 [citado 2020 dez 1];71(supl 1):660-9. Disponível em: https://doi.org/10.1590/0034-7167-2017-0601
https://doi.org/10.1590/0034-7167-2017-0... ^,¹⁶16. Sato APS. Programa Nacional de Imunização: sistema informatizado como opção a novos desafios. Rev Saúde Pública [Internet]. 2015 jul [citado 2019 fev 6]; 49:39. Disponível em: https://doi.org/10.1590/S0034-8910.2015049005925
https://doi.org/10.1590/S0034-8910.20150... It is essential to encourage state and municipal health service managers to adhere to the Immunization Surveillance System; and strengthen on the job immunization surveillance network training and supervision actions, focusing on improving data quality.

For the SI-PNI it is necessary to foster the acquisition of new computer equipment, capable of processing large databases, and to work together with SUS Information Technology Department (Datasus) staff in preparing technological solutions, including reformulating the system with the aim of improving its performance as a whole and rendering it less complex for operators at all levels.

In addition, the distribution and supply of hepatitis B vaccine must be ascertained with the state and municipal coordinators, as well as the training of health center professionals to ensure timely vaccination of newborns. Periodic evaluation of the program is also recommended, using different methodologies.

The fulfillment of such recommendations can qualify the work, production processes and information flow of the PNI, from the vaccine room of the most distant location to the headquarters of its national coordination. And vice versa.

Erratum
In the article “Evaluation of the National Immunization Program Surveillance System - Vaccination Record Module, Brazil, 2017”, doi: 10.1590/S1679-49742021000100028, published on Epidemiology and Health Services, 30(1):e2019596, 2021, in the page 1:
Original text:
“Abstract
Objective: To compare structure and work process in Primary Care for implementing medical teleconsultation in municipalities in different regions and with different population sizes (100,000 inhabitants).
Methods: This was a cross-sectional study, with descriptive and bivariate analysis, using data from 2017-2018 to assess availability of computers with internet access, webcam, microphone, speaker, as well as to assess team work processes (use of Telessaúde [Telehealth], service supply and demand control center and communication flow).
Results: 30,346 primary health centers and 38,865 teams were evaluated. Presence of teleconsultation equipment in the health centers ranged from 1.2% in large northern municipalities to 26.7% in small southern municipalities. Established work process ranged from 10.7% in small northern municipalities to 39.5% in large southern municipalities. Compared to the South, medium-sized municipalities in the North (OR=0.14 - 95%CI 0.11;0.17) and Northeast (OR=0.21 - 95%CI 0.18;0.25) regions were less likely to have the necessary equipment.
Conclusion: Significant regional inequalities call for investments in Digital Health.
Keywords: Telemedicine; Remote Consultation; Information Technology; Policy Making; Cross-Sectional Studies.”
Corrected text:
“Abstract
Objective: To evaluate the National Immunization Program Immunization Surveillance System, based on its Vaccination Record module, for Brazil in 2017.
Methods: This was a descriptive study using the Guidelines for Evaluating Public Health Surveillance Systems, published by the Centers for Disease Control and Prevention (CDC/Atlanta/GA/United States) to evaluate the attributes of simplicity, flexibility, data quality, sensitivity, timeliness and usefulness of the system for six vaccines on the child immunization schedule.
Results: The Immunization Surveillance System was considered complex in its description; flexible to changes in the immunization schedule; of poor data quality for the DTP and rotavirus vaccines; regular acceptability; high sensitivity for the BCG vaccine; untimely for the hepatitis B vaccine and useful for the purposes of the National Immunization Program.
Conclusion: The data quality, acceptability and timeliness results were not satisfactory, so that actions are needed to enhance the information system.
Keywords: Immunization; National Immunization Program; Public Health Surveillance; Program Evaluation; Information Systems; Data Accuracy.”

References

¹
Domingues CMAS, Teixeira AMS. Coberturas vacinais e doenças imunopreveníveis no Brasil no período 1982-2012: avanços e desafios do Programa Nacional de Imunizações. Epidemiol Serviços Saúde [Internet]. 2013 jan-mar [citado 2018 set 6];22(1):9-27. Disponível em: http://dx.doi.org/10.5123/S1679-49742013000100002
» https://doi.org/10.5123/S1679-49742013000100002
²
Ministério da Saúde (BR). Programa nacional de imunizações: 40 anos [Internet]. Brasília: Ministério da Saúde; 2013 [citado 2018 mar 29]. Disponível em: https://www.saude.gov.br/svs
» https://www.saude.gov.br/svs
³
Temporão JG. O Programa Nacional de Imunizações (PNI): origens e desenvolvimento. Hist Ciênc Saúde Manguinhos [Internet]. 2003 [citado 2020 dez 1];10(suppl 2):601-17. Disponível em: https://doi.org/10.1590/S0104-59702003000500008
» https://doi.org/10.1590/S0104-59702003000500008
⁴
Ministério da Saúde (BR). Manual do Sistema de Informação do Programa Nacional de Imunizações: SI-PNI. Brasília: Ministério da Saúde ; 2016.
⁵
Silva BS, Coelho HV, Cavalcante RB, Oliveira VC, Guimarães EAA. Estudo de avaliabilidade do Sistema de Informação do Programa Nacional de Imunização. Rev Bras Enferm [Internet]. 2018 [citado 2020 dez 1];71(supl 1):660-9. Disponível em: https://doi.org/10.1590/0034-7167-2017-0601
» https://doi.org/10.1590/0034-7167-2017-0601
⁶
Ministério da Saúde (BR). Agência Nacional de Vigilância Sanitária - Anvisa. Resolução da Diretoria Colegiada RDC nº 197, de 26 de dezembro de 2017. Dispõe sobre os requisitos mínimos para o funcionamento dos serviços de vacinação humana [Internet]. Brasília: Anvisa, 2017 [citado 2020 dez 1]. Disponível em: https://sbim.org.br/legislacao/867-rdc-anvisa-n-197-26-de-dezembro-de-2017
» https://sbim.org.br/legislacao/867-rdc-anvisa-n-197-26-de-dezembro-de-2017
⁷
Brasil. Ministério da Saúde. Portaria MS/GM nº 2.363, de 18 de outubro de 2012. Institui repasse financeiro do Fundo Nacional de Saúde aos Fundos de Saúde dos Estados, Distrito Federal e Municípios, por meio do Piso Variável de Vigilância e Promoção da Saúde, para fomento na implantação do Sistema de Informação do Programa Nacional de Imunizações (SI-PNI) e Sistema de Informação de Agravos de Notificação (SINAN), no âmbito das unidades de saúde [Internet]. Diário Oficial da União, Brasília (DF), 2012 out 19 [citado 2018 ago 29]; Seção 1:24. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt2363_18_10_2012.html
» http://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt2363_18_10_2012.html
⁸
Brasil. Ministério da Saúde. Portaria MS/GM nº 2.984, de 27 de dezembro de 2016. Revisa a relação de metas e seus respectivos indicadores do Programa de Qualificação das Ações de Vigilância em Saúde (PQA-VS) a partir de 2017 [Internet]. Diário Oficial da União , Brasília (DF), 2016 dez 28 [citado 2018 jun 6]; Seção 1:109. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2016/ prt2984_27_12_2016.html
» http://bvsms.saude.gov.br/bvs/saudelegis/gm/2016/ prt2984_27_12_2016.html
⁹
Centers for Disease Control and Prevention - CDC. Updated guidelines for evaluating public health surveillance systems [Internet]. Washington, D.C.: CDC; 2001 [cited 2018 Mar 29]. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm
» https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm
¹⁰
Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Manual de normas e procedimentos para vacinação [Internet]. Brasília: Ministério da Saúde ; 2014 [citado 2017 jul 3]. 176 p. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf
» http://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf
¹¹
Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Nota informativa nº 94, de 2017/CGPNI/DEVIT/SVS/MST. Orientações e indicação de dose única da febre amarela [Internet]. Brasília: Ministério da Saúde ; 2017 [citado 2020 dez 1]. Disponível em: https://www.saude.mg.gov.br/images/noticias_e_eventos/000_2018/01-jan-fev-marc-abril/Boletins_AEDES/Nota Informativa 94 Com de Acordo 005.pdf
» https://www.saude.mg.gov.br/images/noticias_e_eventos/000_2018/01-jan-fev-marc-abril/Boletins_AEDES/Nota Informativa 94 Com de Acordo 005.pdf
¹²
Ministério da Saúde (BR). Nota informativa no 62-SEI/2017-CGPNI/DEVIT/SVS/MS. Orienta os serviços de vacinação para a otimização do uso da vacina HPV quadrivalente, com ampliação temporária da faixa etária [Internet]. Brasília: Ministério da Saúde ; 2017 [citado 2020 dez 1]. Disponível em: https://www.saude.gov.br/images/pdf/2017/agosto/18/SEI_MS-0290791-Nota-Informativa.pdf
» https://www.saude.gov.br/images/pdf/2017/agosto/18/SEI_MS-0290791-Nota-Informativa.pdf
¹³
Teixeira AMS, Silva AA, Silva AF, Domingues CMAS, Locádio ES, Fantinato FFST, Souza LRO, et al. Avaliação dos indicadores de desempenho da vacinação do Programa Nacional de Imunizações e os desafios para elevar as coberturas vacinais no Brasil. In: Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Saúde Brasil 2019: uma análise da situação de saúde com enfoque nas doenças imunopreveníveis e na imunização [Internet]. Brasília: Ministério da Saúde; 2019 [citado 2020 dez 1]. p. 370-486. Disponível em: https://portalarquivos2.saude.gov.br/images/pdf/2019/dezembro/05/Saude-Brasil-2019-imunizacao.pdf
» https://portalarquivos2.saude.gov.br/images/pdf/2019/dezembro/05/Saude-Brasil-2019-imunizacao.pdf
¹⁴
Waldman EA, Sato APS. Trajetória das doenças infecciosas no Brasil nos últimos 50 anos: um contínuo desafio. Rev Saúde Pública [Internet]. 2016 dez [citado 2020 dez 1]; 50:68. Disponível em: https://doi.org/10.1590/S1518-8787.2016050000232
» https://doi.org/10.1590/S1518-8787.2016050000232
¹⁵
Domingues CMAS, Woycicki JR, Rezende KS, Henriques CMP. Programa Nacional de Imunização: a política de introdução de novas vacinas. Rev Eletr Gestão Saúde [Internet]. 2015 out [citado 2020 dez 1]; 6(supl 4):3250-74. Disponível em: https://periodicos.unb.br/index.php/rgs/article/view/3331/3017
» https://periodicos.unb.br/index.php/rgs/article/view/3331/3017
¹⁶
Sato APS. Programa Nacional de Imunização: sistema informatizado como opção a novos desafios. Rev Saúde Pública [Internet]. 2015 jul [citado 2019 fev 6]; 49:39. Disponível em: https://doi.org/10.1590/S0034-8910.2015049005925
» https://doi.org/10.1590/S0034-8910.2015049005925
¹⁷
Machado Costa MR, Barbosa J. Situação atual da febre amarela no Brasil. Bol Eletr Epidemiol [Internet]. Brasília: Funasa; 2001 [citado 2020 dez 1]. Disponível em: http://www.funasa.gov.br
» http://www.funasa.gov.br
¹⁸
Valente NTZ, Fujino A. Atributos e dimensões de qualidade da informação nas ciências contábeis e na ciência da informação: um estudo comparativo. Perspect Ciência Inf [Internet]. 2017 abr-jun [citado 2020 dez 1];21(2):141-67. Disponível em: https://doi.org/10.1590/1981-5344/2530
» https://doi.org/10.1590/1981-5344/2530
¹⁹
Lima CRA, Schramm JMA, Coeli CM, Silva MEM. Revisão das dimensões de qualidade dos dados e métodos aplicados na avaliação dos sistemas de informação em saúde. Cad Saúde Pública [Internet]. 2009 out [citado 2020 dez 1];25(10):2095-109. Disponível em: https://www.scielo.br/pdf/csp/v25n10/02.pdf
» https://www.scielo.br/pdf/csp/v25n10/02.pdf
²⁰
Correia LO, Padilha BM, Vasconcelos SML. Métodos para avaliar a completitude dos dados dos sistemas de informação em saúde do Brasil: uma revisão sistemática. Ciênc Saúde Coletiva [Internet]. 2014 [citado 2020 dez 1];19(11):4467-78. Disponível em: https://www.scielo.br/pdf/csc/v19n11/1413-8123-csc-19-11-4467.pdf
» https://www.scielo.br/pdf/csc/v19n11/1413-8123-csc-19-11-4467.pdf
²¹
Luhm KR, Waldman EA. Sistemas informatizados de registro de imunização: uma revisão com enfoque na saúde infantil. Epidemiol Serv Saúde [Internet]. 2009 jan-mar [citado 2020 dez 1];18(1):65-78. Disponível em: http://scielo.iec.gov.br/pdf/ess/v18n1/v18n1a07.pdf
» http://scielo.iec.gov.br/pdf/ess/v18n1/v18n1a07.pdf
²²
Teixeira AMS, Mota ELA. Denominators for vaccine coverage estimates: a database study to estimate the population less than one year of age. Epidemiol Serv Saúde [Internet]. 2010 Jul-Sep [cited 2019 Feb 12];19(3):187-203. Available from: http://dx.doi.org/10.5123/S1679-49742010000300002
» https://doi.org/10.5123/S1679-49742010000300002
²³
Ministério da Saúde (BR). Organização Pan-Americana da Saúde - OPAS. Fundação Oswaldo Cruz - Fiocruz. A experiência brasileira em sistemas de informação em saúde: falando sobre os sistemas de informação em saúde no Brasil [Internet]. Brasília: Ministério da Saúde ; 2009 [citado 2020 dez 1]. 2 v. (Série B. Textos Básicos de Saúde). Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/experiencia_brasileira_sistemas_saude_volume2.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/experiencia_brasileira_sistemas_saude_volume2.pdf
²⁴
Tauil MC, Sato APS, Costa ÂA, Inenami M, Ferreira VLR, Waldman EA. Vaccination coverage according to doses received and timely administered based on an electronic immunization registry, Araraquara-SP, Brazil, 2012-2014. Epidemiol Serv Saúde [Internet]. 2017 Oct-Dec [cited 2020 Dez 1];26(4):835-46. Available from: https://doi.org/10.5123/S1679-49742017000400014
» https://doi.org/10.5123/S1679-49742017000400014

Publication Dates

Publication in this collection
26 Mar 2021
Date of issue
2021

History

Received
03 Feb 2020
Accepted
09 Oct 2020

Este é um artigo publicado em acesso aberto sob uma licença Creative Commons

Attribute	Classification/cut-off points
Simplicity	Simple or complex	Evaluated subjectively, considering the Immunization Surveillance System description.
Flexibility	Flexible or not very flexible	It was evaluated whether the information system adapted to the changes in the Immunization Schedule. As a cut-off point, three or more positive criteria were considered.
Data quality	High	Good Completeness; Business Rule classified as adequate in the three inconsistency criteria.
	Moderate	Good Completeness; Business Rule rated as adequate in one or two inconsistency criteria.
	Moderate	Poor Completeness; Business Rule rated as adequate in two or three inconsistency criteria.
	Low	Good Completeness; Business Rule rated as inadequate in three inconsistency criteria.
		Poor Completeness; Business Rule rated as adequate in only one inconsistency criterion
		Poor Completeness; Business Rule rated as inadequate in all three inconsistency criteria.
Acceptability	Excellent, regular or bad	Vaccine rooms' adherence to SI-PNI was evaluated. The cut-off points were: ≥80%; ≥50 to <80%; and <50%.
Sensitivity	High or low	Children under one year of age, covered by BCG vaccination, compared to the number of to live births. The cut-off point was 80%.
Timeliness	Timely or untimely	Timeliness of hepatitis B vaccination in up to one day of life in relation to the doses applied up to 30 days of life. The cut-off point was 80%.
Usefulness	A classification of this evaluation component was not made, as it was considered that the systems are useful, but can be improved.	It was evaluated based on the system's purpose.

Municipality	Doses administered ≤30 days	Timely dose ≤24h	Timeliness (%)
Rio Branco	6,729	5,422	80.6
Maceió	2,584	108	4.2
Macapá	8,254	4,226	51.2
Parintins	1,887	1,582	83.8
Feira de Santana	4,827	2,083	43.2
Caucaia	574	381	66.4
Brasília	11,773	10,447	88.7
Vitória	3,149	2,402	76.3
Aparecida de Goiânia	1,094	375	34.3
São Luís	204	14	6.9
Cuiabá	442	256	57.9
Dourados	3,092	2,540	82.1
Contagem	4,924	2,588	52.6
Ananindeua	2,787	1,735	62.3
João Pessoa	1,804	915	50.7
Londrina	3,738	3,706	99.1
Recife	3,267	1,891	57.9
Teresina	10,426	8,691	83.4
Rio de Janeiro	21,103	19,201	91.0
Natal	2,318	1,234	53.2
Porto Alegre	4,956	4,850	97.9
Porto Velho	4,718	3,757	79.6
Boa Vista	5,192	5,127	98.7
Joinville	12,340	11,999	97.2
Santo André	6,624	6,374	96.2
Aracaju	1,719	1,257	73.1
Palmas	4,006	3,448	86.1
27 municipalities	134,531	106,609	79.2

Saúde Pública

Saúde Pública

Evaluation of the National Immunization Program Surveillance System - Vaccination Record Module, Brazil, 2017

Abstract

Objetivo

Methods

Results

Conclusion

Introduction

Methods

Results

Simplicity

Flexibility

Data quality

Acceptability

Sensitivity

Timeliness

Usefulness

Discussion

Erratum

References

Publication Dates

History