ABSTRACT

The COVID-19 pandemic generated a large volume of scientific productions with different quality levels. The speed with which knowledge was produced and shared worldwide imposed on health management the challenge of seeking ways to identify the best available evidence to support its decisions. In response to this challenge, the Department of Science and Technology of the Brazilian Ministry of Health started offering a service to produce and provide scientific knowledge addressing priority public health issues in the pandemic scenario. Drug treatments, non-pharmacological measures, testing, reinfection and immunological response, immunization, pathophysiology, post-COVID syndrome and adverse events are among the topics covered. In this article, we discuss the strengths and lessons learned, as well as the challenges and perspectives that present a real example of how to offer the best scientific evidence in a timely manner in order to assist the decision-making process during a public health emergency.

Coronavirus; Implementation Science; Health Communication; Information Dissemination; Public Health

INTRODUCTION

The public health crisis caused by the new coronavirus (COVID-19) has increased interest in access to the best evidence concerning policies, practices, and personal decisions. To support timely decision making, evidence synthesis experts summarized available research in a short period of time. The challenge was even greater due to the huge amount of publications available on COVID-19 in preprint articles and the speed with which scientific studies were conducted in the period¹1. Majumder MS, Mandl KD. Early in the epidemic: impact of preprints on global discourse about COVID-19 transmissibility. Lancet Glob Health. 2020 May;8(5):e627-30. https://doi.org/10.1016/S2214-109X (20)30113-3
https://doi.org/10.1016/S2214-109X (20)3... .

Evidence syntheses are summaries with interpretations of individual research, which answer specific questions within a larger context of knowledge²2. Tricco AC, Tetzlaff J, Moher D. The art and science of knowledge synthesis. J Clin Epidemiol. 2011 Jan;64(1):11-20. https://doi.org/10.1016/j.jclinepi.2009.11.007
https://doi.org/10.1016/j.jclinepi.2009.... ,³3. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet. 2017 Jul;390(10092):415-23. https://doi.org/10.1016/S0140-6736 (16)31592-6
https://doi.org/10.1016/S0140-6736 (16)3... and are essential for providing the highest level of evidence. However, their elaboration period, in many cases, exceeds the time to inform decisions during public health crises⁴4. Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017 Feb;7(2):e012545. https://doi.org/10.1136/bmjopen-2016-012545
https://doi.org/10.1136/bmjopen-2016-012... . In these circumstances, rapid evidence syntheses are recommended⁵5. World Health Organization. Alliance for Health Policy and Systems Research. Rapid reviews to strengthen health policy and systems: a practical guide Geneva. World Health Organization; 2017.,⁶6. Garritty C, Gartlehner G, Nussbaumer-Streit B, King VJ, Hamel C, Kamel C, et al. Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. J Clin Epidemiol. 2021 Feb;130:13-22. https://doi.org/10.1016/j.jclinepi.2020.10.007
https://doi.org/10.1016/j.jclinepi.2020.... .

There is a growing body of literature on the use of on-demand evidence rapid response services and the limitations of their methods compared to other types of syntheses⁶6. Garritty C, Gartlehner G, Nussbaumer-Streit B, King VJ, Hamel C, Kamel C, et al. Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. J Clin Epidemiol. 2021 Feb;130:13-22. https://doi.org/10.1016/j.jclinepi.2020.10.007
https://doi.org/10.1016/j.jclinepi.2020.... . In this article, we report the initiative of a Brazilian team to provide the best scientific evidence, via a rapid response service within a federal agency, the Ministry of Health (MS)¹²12. Brasil. Decreto nº 9.795, de 17 de maio de 2019. Aprova a Estrutura Regimental e o Quadro Demonstrativo dos Cargos em Comissão e das Funções de Confiança do Ministério da Saúde, remaneja cargos em comissão e funções de confiança, transforma funções de confiança e substitui cargos em comissão do Grupo-Direção e Assessoramento Superiores - DAS por Funções Comissionadas do Poder Executivo - FCPE. Diário Oficial União. 2019 May 20.,¹³13. Brasil. Decreto nº 11.098, de 20 de junho de 2022. Aprova a Estrutura Regimental e o Quadro Demonstrativo dos Cargos em Comissão e das Funções de Confiança do Ministério da Saúde e remaneja e transforma cargos em comissão e funções de confiança. Diário Oficial União. 2022 Jun 20.. During the first two years of the pandemic, the team conducted around 90 studies, covering 20 technical areas. The lessons learned and the challenges overcome during these studies are presented below.

Facilitators in the Process of Responding to Demands for Evidence

There was an experienced and multidisciplinary team dedicated to work on evidence syntheses. This team created a portfolio that functioned as a service charter, presenting the types of evidence rapid responses products that could be developed, their objectives, limitations, elaboration deadlines, and the necessary request flows¹⁴14. Ministério da Saúde (BR). Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde. Departamento de Ciência e Tecnologia. Serviço de produção de evidências para apoio à tomada de decisão: portfólio de produtos. Brasília, DF: Ministério da Saúde; 2019. (Figure 1). The team also knew how many members were needed to complete a synthesis within a specific deadline, as well as to determine the choice of the type of synthesis to be prepared, the type of study design to be included and the evidence sources¹⁴14. Ministério da Saúde (BR). Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde. Departamento de Ciência e Tecnologia. Serviço de produção de evidências para apoio à tomada de decisão: portfólio de produtos. Brasília, DF: Ministério da Saúde; 2019..

The continuous relationship between those preparing the syntheses and the decision makers was important to generate trust and help in refining the main doubts regarding the fight against the pandemic, and also in the definition of the eligibility criteria and results of interest for research. Research demands were co-created by the team and consumers, which strengthened the service, characterizing it as a collaborative knowledge translation¹⁵15. Cvitanovic C, Shellock RJ, Mackay M, van Putten EI, Karcher DB, Dickey-Collas M, et al. Strategies for building and managing 'trust' to enable knowledge exchange at the interface of environmental science and policy. Environ Sci Policy. 2021 Sep;123:179-89. https://doi.org/10.1016/j.envsci.2021.05.020
https://doi.org/10.1016/j.envsci.2021.05... . These decision makers, in general, were from final technical areas of the Ministry of Health and carried out the action in different ways, involving primary and specialized health care and mainly health surveillance—epidemiological, laboratory and worker health.

With the aim of aligning expectations and optimizing time, one established a workflow, which consisted of sequential steps: receiving demand via the Electronic Information System (SEI) and through direct contact with the area coordination; alignment meeting between the team and decision makers; study development; preparation of the communication product, and demand delivery. Each of them had its own flows and schedules adapted to the demanding areas’ needs.

Frequently, the demanding areas’ time to align research questions was reduced, but efforts were directed at clarifying the question, target audience and the usefulness of research results. Furthermore, this strategy contributed to increasing the engagement of decision makers in some steps of synthesis preparation, that is, it was a co-creation process, which could facilitate the evidence absorption in decision making.

Requests were prioritized based on: response urgent need; level of decision-making, often measured by the hierarchical position of the demanding area; team’s knowledge and technical capacity regarding the content, and evidence availability, verified by a preliminary database search. The team was also proactive in searching for immediate and long-term priorities, anticipating what might be needed.

The types of evidence products developed varied in relation to the scope (public health measures, clinical management, health systems arrangements, and economic and social responses), time required for response and availability of the team, which made some adjustments and methodological shortcuts to deliver products faster without scientific rigor reduction (Table 1)⁵5. World Health Organization. Alliance for Health Policy and Systems Research. Rapid reviews to strengthen health policy and systems: a practical guide Geneva. World Health Organization; 2017.. As science evolved quickly in the pandemic scenario, some syntheses were periodically updated. Changes were presented to the decision maker in a quick and transparent manner, through a text box with highlights entitled “Main updates to this version” at the beginning of the synthesis. The main evidence products included rapid systematic reviews (34%), abstract summaries (25%), and reference inventories (23%) (Table 2) ⁷7. Hartling L, Guise JM, Hempel S, Featherstone R, Mitchell MD, Motu'apuaka ML, et al. Fit for purpose: perspectives on rapid reviews from end-user interviews. Syst Rev. 2017 Feb;6(1):32. https://doi.org/10.1186/s13643-017-0425-7
https://doi.org/10.1186/s13643-017-0425-... ,¹⁴14. Ministério da Saúde (BR). Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde. Departamento de Ciência e Tecnologia. Serviço de produção de evidências para apoio à tomada de decisão: portfólio de produtos. Brasília, DF: Ministério da Saúde; 2019..

To make it usable, better packaging of knowledge was necessary. Besides the traditional format, an additional version was prepared by a scientific communication and information design team, responsible for making the language and format appropriate and accessible to the target audience. Scientific jargon was transformed into plain language, besides investment in visual summaries, such as infographics (Figure 2).

A knowledge intermediary was constantly used to transfer the evidence generated, in order to promote greater absorption of knowledge by potential users¹⁶16. Cochrane Consumer Network. Covid-19 consumer rapid response group. London: Cochrane Consumer Network; 2021 [cited 2021 Mar]. Available from: https://consumers.cochrane.org/news/covid-19-consumer-rapid-response-group
https://consumers.cochrane.org/news/covi... .

It was a neutral communicator—like a necessary channel—between the generator and the potential knowledge user. People operating as intermediaries can pave the way for the absorption and potential implementation of scientific knowledge in policy due to their credibility²⁰20. Paing PY, Kyaw ZL, Schojan M, Traill T, Thura S, Tin N, et al. Development of a knowledge broker group to support evidence-informed policy: lessons learned from Myanmar. Health Res Policy Syst. 2021 Dec;19(1):153. https://doi.org/10.1186/s12961-021-00806-x
https://doi.org/10.1186/s12961-021-00806... .

An example of a situation in which the evidence synthesis indicated that it was contrary to the use of a medicine and was not followed by some Brazilian managers was the case of hydroxychloroquine in the treatment for COVID-19²⁴24. Andrade KR, Carvalho VK S, Farinasso CM, Lima AA, Silva RB, Wachira VK, et al. Terapia medicamentosa para infecções por coronavírus em humanos: revisão sistemática rápida. Cienc SaUde Coletiva. 2020Sep;25(9):3517-54. https://doi.org/10.1590/1413-81232020259.14242020
https://doi.org/10.1590/1413-81232020259... . In contrast, an example of implementation of the prepared syntheses was the summary of abstracts on mapping of definitions of post-COVID-19 syndrome, which contributed to the adoption of terminology related to post-COVID conditions and the definition of a new ICD (International Statistical Classification of Diseases and Health-Related Problems)²⁵25. Ministério da Saúde (BR). Secretaria Extraordinária de Enfrentamento à covid-19. Nota técnica nº 31/2021. Brasília, DF: Ministério da Saúde; 2021 [cited 2021 Mar]. Available from: https://www.gov.br/saude/pt-br/coronavirus/vacinas/plano-nacional-de-operacionalizacao-da-vacina-contra-a-covid-19/notas-tecnicas/2021/nota-secovid-gab-secovid-ms.pdf/view
https://www.gov.br/saude/pt-br/coronavir... .

A daily report of scientific evidence on pharmacological treatment for COVID-19 was established as a means of dissemination. The studies were screened, summarized, and classified by methodological design, as well as assessed for their methodological quality and risk of bias, using appropriate tools for each type of study, and, finally, made available in a dashboard (Figure 3). In this way, abstracts of preprint technical-scientific publications from indexed journals on the efficacy, safety and effectiveness of medicines and biological products used for COVID-19 treatment and prevention were made available daily. At the beginning of the pandemic, when there was a surge in publications and media coverage, the aim of this initiative was to promote access to evidence in a qualified, easy, and reliable manner.

There were sufficient organizational resources to respond to requests from decision makers. Part of the team used personal computers and Internet connections since they were working remotely; nevertheless, the department had the capacity for data storage and access to information sources upon payment of a license to use indexed journals publications.

The team sought to reduce the duplication of efforts in conducting syntheses by searching for ongoing research initiatives worldwide that addressed Brazilian decision makers’ doubts. For example, the group accessed the COVID-19 Evidence Network to support Decision-making (COVID-END) and the Brazilian Evidence-Informed Policy Network (Evipnet-Brasil). A prior protocol was also drawn up and, when possible, registered on public platforms.

Challenges and Perspectives in Preparing Evidence Syntheses

Using evidence synthesis assists the decision-making process, but it is not sufficient to ensure evidence-informed decision making. Political and economic interests interfere in the process, and, therefore, the presence of institutional leadership that values the use of evidence facilitates its adoption²⁷27. Shearer JC, Lavis J, Abelson J, Walt G, Dion M. Evidence-informed policymaking, and policy innovation in a low-income country: does policy network structure matter? Evid Policy. 2018;14(3):381-401. https://doi.org/10.1332/174426418X15330477583836
https://doi.org/10.1332/174426418X153304... . Furthermore, uncertainty surrounded a scenario of incipient knowledge.

There was a kind of “evidence explosion” and many organizations began to promote themselves by producing documents based on it. There was no methodological consensus regarding the synthesis and evaluation of this evidence, and decision makers were not always able to differentiate a systematic review from a manual selection of studies.

It was common for several working groups in the country to develop the same themes. It is necessary to improve coordination and national communication tools, and mobilization must prioritize themes and disseminate results, avoiding duplication of efforts in preparing evidence syntheses.

The team was asked to deal with different sources of information, as there were policymakers interested in identifying evidence related to institutional documents from different countries to understand their experience in managing the pandemic.

Translation of evidence synthesis or documents prepared in other languages into Portuguese was a resource that was little used or took a long time. There was also a lack of an online platform to produce living documents and indicate which research efforts became obsolete.

Researchers used to scientific production understand how evidence evolves, so changes are seen as part of the process. However, for the lay public this process can be confusing and cause distrust. Thus, one issue faced was the lack of strategies to deal with studies that report results from the beginning of the pandemic, which could no longer be relevant, given the knowledge of the most recent COVID-19 situation. For example, a rapid review of social distancing in the transmission of the new coronavirus was maintained and studies continued to be published, reporting results from the beginning of the pandemic. Nevertheless, as much more was known about virus transmission, infection and prevention measures, these studies might not provide relevant value.

One of the department’s biggest challenges was engaging social actors interested in the topics of evidence syntheses, especially due to limited deadlines. Such involvement could help identify health needs and research problems, besides enabling assessments and finding interpretation. For example, they can provide their perspective on the issue, the results, the analysis interpretation, and the summary in plain language. Most are interested in various areas of research, but not everyone has experience in evidence synthesis. A useful strategy would be to provide those interested with adequate training on evidence-based health concepts and terms²⁸28. Han G, Mayer M, Canner J, Lindsley K, Datar R, Le J, et al. Development, implementation and evaluation of an online course on evidence-based healthcare for consumers. BMC Health Serv Res. 2020 Oct;20(1):928. https://doi.org/10.1186/s12913-020-05759-5
https://doi.org/10.1186/s12913-020-05759... .

It is noteworthy that given the resources available, the team always sought to make the most assertive decisions for the scenario, but other efforts also need to be encouraged. Besides contributions to the preparation of syntheses, it is necessary to inform the population about the results of the syntheses and recognize reliable scientific evidence. Initiatives in this area can contribute to the democratization of science and improvement of critical thinking about information published on the Internet. Furthermore, although evidence is fundamental to informing decisions, it is not the only factor considered in decision making. Nevertheless, one believes that the lack of alignment between the spheres of government may have compromised the implementation of the syntheses.

FINAL CONSIDERATIONS

An important step towards the application of scientific knowledge in real time in Brazil was taken during the COVID-19 pandemic. It is insufficient to invest in selected research, and it is also necessary to encourage the use and creation of living online platforms. For this purpose, we must work in a network to avoid overlapping efforts and non-dissemination of the knowledge produced. More research is needed on how to present and prioritize results and how to engage the stakeholders in rapid syntheses to ensure all perspectives are achieved. Thus, the need to institutionalize the transparent and systematic use of evidence in decision-making processes stands out. This path taken by the group during the pandemic can be considered a proposal for an innovative model regarding the use of evidence syntheses to support health decision-making.

REFERENCES

Publication Dates

History