ABSTRACT

Objective.

To understand how replacement of decision-makers (DMs) serving as principal investigators (PI) or co-PIs on research teams may affect the feasibility and value of embedded implementation research (EIR) used to improve health policies, programs, and services in Latin America and the Caribbean.

Methods.

This was a descriptive qualitative study based on 39 semistructured interviews with 13 embedded research teams selected by financing agencies to explore team composition, interaction among members, and research results. Interviews were conducted at three points during the study period from September 2018 to November 2019; data were analyzed from 2020 to 2021.

Results.

Research teams were found to be operating in one of three situations: (i) permanent core team (no change) with either active DM or inactive DM participation; (ii) replacement of DM–PI or co-PI that did not affect EIR research; and (iii) replacement of DM–PI that affected EIR.

Conclusions.

To ensure EIR continuity and stability, research teams should include high-level DMs together with more technical staff performing essential implementation activities. This structure could improve collaboration among professional researchers and ensure greater embeddedness of EIR to strengthen the health system.

Keywords
Operational research; decision making; health policy; implementation science; Latin America; Caribbean Region

RESUMEN

Objetivo.

Comprender de qué manera la sustitución de los responsables de tomar decisiones (RTD) que se desempeñan como investigadores principales (IP) o coinvestigadores principales (coIP) en los equipos de investigación puede incidir en la viabilidad y el valor de la investigación integrada en materia de ejecución (IIME) utilizada con el objetivo de mejorar las políticas, los programas y los servicios de salud en América Latina y el Caribe.

Métodos.

Este estudio cualitativo descriptivo se basó en 39 entrevistas semiestructuradas realizadas a 13 equipos de investigación integrada seleccionados por organismos de financiación con el objetivo de estudiar la composición de los equipos, la interacción entre sus miembros y los resultados de la investigación. Las entrevistas se realizaron en tres ocasiones durante el período de estudio, que se extendió de septiembre del 2018 a noviembre del 2019; los datos se analizaron entre el 2020 y el 2021.

Resultados.

Se encontró que los equipos de investigación se desempeñaban en una de las siguientes tres situaciones: a) un equipo central permanente (sin cambios), sea con participación activa del RTD o sin participación activa del mismo; b) sustitución del RTD–IP o coIP, sin consecuencias para la IIME; y c) sustitución del RTD–IP, con consecuencias para la IIME.

Conclusiones.

Para asegurar la continuidad y estabilidad de la IIME, los equipos de investigación deberían incluir RTD de alto nivel junto con más personal técnico que lleve a cabo actividades esenciales en materia de ejecución. Esta estructura podría mejorar la colaboración entre los investigadores profesionales y asegurar una mayor integración de la IIME con la finalidad de fortalecer el sistema de salud.

Palabras clave
Investigación operative; toma de decisiones; política de salud; ciencia de la implementación; América Latina; Región del Caribe

RESUMO

Objetivo.

Entender como a substituição de decisores atuando como investigadores principais ou co-investigadores em equipes de pesquisa pode afetar a viabilidade e o valor da pesquisa de implementação incorporada (EIR, na sigla em inglês), método usado para aprimorar as políticas, programas e serviços de saúde na América Latina e no Caribe.

Métodos.

Estudo qualitativo e descritivo, baseado em 39 entrevistas semiestruturadas com 13 equipes de pesquisa incorporadas, selecionadas por agências de fomento, para explorar a composição das equipes, a interação entre os membros e os resultados das pesquisas. As entrevistas foram realizadas em três momentos durante o período de estudo, de setembro de 2018 a novembro de 2019. Os dados foram analisados de 2020 a 2021.

Resultados.

Verificou-se que as equipes de pesquisa se encontravam em uma das seguintes três situações: (i) equipe essencial permanente (sem alteração), com participação ativa ou inativa do decisor; (ii) substituição do decisor (investigador principal ou co-investigador) não afetou a pesquisa EIR; e (iii) substituição do decisor (investigador principal ou co-investigador) afetou a pesquisa EIR.

Conclusões.

Para garantir a continuidade e estabilidade de pesquisas de EIR, as equipes de pesquisa devem incluir decisores de alto nível, juntamente com pessoal de perfil mais técnico, para realizar atividades essenciais de implementação. Tal estrutura poderia melhorar a colaboração entre pesquisadores profissionais e garantir uma incorporação ainda maior da EIR para fortalecer o sistema de saúde.

Palavras-chave
Pesquisa operacional; tomada de decisões; política de saúde; ciência da implementação; América Latina; Região do Caribe

Governments and international agencies have used several approaches to narrow the distance between research and decision-making (¹1. Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ. 2006;84(8):620-8. doi:10.2471/blt.06.030312
https://doi.org/10.2471/blt.06.030312... –³3. Koon AD, Rao KD, Tran NT, Ghaffar A. Embedding health policy and systems research into decision-making processes in low- and middle-income countries. Health Res Policy Sys. 2013;104(12):501-9. doi:10.1258/jrsm.2011.110285
https://doi.org/10.1258/jrsm.2011.110285... ). Efforts have been made to approach knowledge translation as more than a simple process (²2. Greenhalgh T, Sietsewieringa S. Is it time to drop the “knowledge translation” metaphor? A critical literature review. J Royal Soc Med. 2011;104(12):501-9. doi:10.1258/jrsm.2011.110285
https://doi.org/10.1258/jrsm.2011.110285... ,⁴4. Kothari AR, Bickford JJ, Edwards N, Dobbins MJ, Meyer M. Uncovering tacit knowledge: a pilot study to broaden the concept of knowledge in knowledge translation. BMC Health Serv Res. 2011;1(198). doi:10.1186/1472-6963-11-198
https://doi.org/10.1186/1472-6963-11-198... ,⁵5. Crosschild C, Huynh N, De Sousa I, Bawafaa E, Brown H. Where is critical analysis of power and positionality in knowledge translation? Health Res Policy Sys. 2021;19(92). doi:10.1186/s12961-021-00726-w
https://doi.org/10.1186/s12961-021-00726... ) and to move toward a more participative dynamic. Stemming from the Changing Mindsets publication (⁶6. World Health Organization. Changing Mindsets: Strategy on Health Policy and Systems Research. Geneva: WHO; 2012.), embedded research and integrated knowledge translation, coproduction of knowledge, and other concepts are/focused on the collaboration of decision-makers (DMs) and researchers (⁷7. Becerril-Montekio V, García-Bello LA, Torres-Pereda P, Alcalde-Rabanal J, Reveiz L, Langlois ÉV. Collaboration between health system decision makers and professional researchers to coproduce knowledge: a scoping review. 2022;19(92). doi:10.1186/s12961-021-00726-w
https://doi.org/10.1186/s12961-021-00726... ). These concepts are at the center of important theoretical discussions and practical endeavors in the contexts of low- and middle-income countries, mostly in Asia and Latin America and the Caribbean (LAC) (⁸8. Olivier J, Whyle E, Gilson L. Embedded Health Policy and Systems Research: A Rapid Scoping Review. Report for the Alliance for Health Policy and Systems Research. 2018;1-20.

9. Ghaffar A, Langlois EV, Rasanathan K, Peterson S, Adedokun L, Tran NT. Strengthening health systems through embedded research. Bull World Health Organ. 2017;95(2):87. doi:10.2471/BLT.16.189126
https://doi.org/10.2471/BLT.16.189126...

10. Langlois EV, Tran NT, Ghaffar A, Reveiz L, Becerra-Posada F. Embedding research in health policy and systems in the Americas. Rev Panam Salud Publica. 2017;41:e68. doi:10.26633/RPSP.2017.68
https://doi.org/10.26633/RPSP.2017.68...

11. Tran N, Langlois EV, Reveiz L, Varallyay I, Elias V, Mancuso A, et al. Embedding research to improve program implementation in Latin America and the Caribbean. Rev Panam Salud Publica. 2017;41:e75. doi:10.26633/RPSP.2017.75
https://doi.org/10.26633/RPSP.2017.75... -¹²12. Rasanthan K, Tran NT, Johnson H, Hafeez A, Peterson S, Ghaffar A. Realizing the potential of embedded implementation research: lessons from Pakistan. J Glob Health. 2020;10(2):1-3. doi:10.7189/jogh.10.020104
https://doi.org/10.7189/jogh.10.020104... ).

Research financing agencies have fostered investigation oriented by the criterion that one of the most effective ways to improve health interventions is by engaging the implementers (mainly high-level DMs) in research processes aimed to unveil what is and is not working as expected (¹³13. Olivier J, Scott V, Molosiwa D GL. Embedded systems approaches to health policy and systems research. In: Savigny D, Blanchet K, Adam T, eds. Applied Systems Thinking for Health Systems Research: a Methodological Handbook. New York: McGraw-Hill Education; 2017.,¹⁴14. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Sys. 2020;18(64). doi:10.1186/s12961-020-00579-9
https://doi.org/10.1186/s12961-020-00579... ). This basic assumption has been the essential theoretical standpoint on which international organizations and financing agencies have promoted research involving health DMs as leaders of embedded implementation research (EIR) projects. Their participation as principal investigators (PIs) of implementation research projects has been considered imperative to closing the know-do gap (¹⁴14. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Sys. 2020;18(64). doi:10.1186/s12961-020-00579-9
https://doi.org/10.1186/s12961-020-00579...

15. Langlois EV, Mancuso A, Elias V, Reveiz L. Embedding implementation research to enhance health policy and systems: a multi-country analysis from ten settings in Latin America and the Caribbean. Heal Res Policy Syst. 2019; 1(95). doi:10.1186/s12961-019-0484-4
https://doi.org/10.1186/s12961-019-0484-... -¹⁶16. Ghaffar A, Gupta A, Kampo A, Swaminathan S. The value and promise of embedded research. Health Res Policy Sys. 2021;19(suppl 2):99. doi:10.1186/s12961-021-00744-8
https://doi.org/10.1186/s12961-021-00744... ). These efforts have been based on the participation of teams of DMs and professional researchers from the formulation of the research projects, hoping to obtain their effective involvement throughout all the process. Nevertheless, the uneven development and results of these experiences demand that we consider how DMs and professional researchers collaborate at different levels of interest and engagement. Recent literature has paid little attention to the particular ways in which these two stakeholders collaborate, a dynamic that depends on issues such as relative power and position and the stability of the health services, all of which affect the research process itself (⁵5. Crosschild C, Huynh N, De Sousa I, Bawafaa E, Brown H. Where is critical analysis of power and positionality in knowledge translation? Health Res Policy Sys. 2021;19(92). doi:10.1186/s12961-021-00726-w
https://doi.org/10.1186/s12961-021-00726... ). The authors of this research article have documented that in the context of low- and middle-income countries, where DM positions are subject to unstable circumstances, replacement is not unusual (¹⁷17. García-Cerde R, Becerril-Montekio V, Langlois É, Reveiz L, Alcalde-Rabana J, Torres-Pereda P. Embedded implementation research determinants in Latin American Health Systems. Rev Saude Publica. 2021;23(55):16. doi:10.11606/s1518-8787.2021055003027
https://doi.org/10.11606/s1518-8787.2021... ). However, replacement has not been sufficiently considered as a factor possibly influencing how DMs participate in collaborative projects nor how it may be affecting EIR results.

Analysis of actual collaboration between implementers and researchers in EIR and knowledge coproduction in recent experiences has proposed the following three categories: formal collaboration, mutual learning, and consolidation of previous collaboration (¹⁸18. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Experiencias y retos del Centro de Apoyo Técnico en la investigación integrada en los servicios en América Latina y el Caribe. Rev Panam Salud Publica. 2021;45:e19. doi:10.26633/RPSP.2021.19
https://doi.org/10.26633/RPSP.2021.19... ):

Formal Collaboration is characterized by the establishment of a compromise between the DM collaborating with researchers (generally from academic institutions) who propose and lead the research, benefitting both parties but lacking a longer or deeper commitment. While this modality usually renders good quality products, it does not necessarily improve the health staff’s capacity to produce and use research results.

Mutual Learning is based on effective collaboration to coproduce knowledge from the beginning to the end of the research process. Decision-makers and health staff acquire new research capabilities, and researchers learn to include the DM’s experience and knowledge of the terrain. In the end, both parties share ownership of the research products, which greatly favors implementation and utilization.

Consolidation of Previous Collaboration implies existing relationships between both parties, generally based on institutional arrangements that guarantee close and horizontal cooperation. The other two modalities may serve as a starting point for its development.

Current available literature highlights the advantages of EIR, and there is evidence of its success (¹¹11. Tran N, Langlois EV, Reveiz L, Varallyay I, Elias V, Mancuso A, et al. Embedding research to improve program implementation in Latin America and the Caribbean. Rev Panam Salud Publica. 2017;41:e75. doi:10.26633/RPSP.2017.75
https://doi.org/10.26633/RPSP.2017.75... ,¹⁴14. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Sys. 2020;18(64). doi:10.1186/s12961-020-00579-9
https://doi.org/10.1186/s12961-020-00579... ,¹⁵15. Langlois EV, Mancuso A, Elias V, Reveiz L. Embedding implementation research to enhance health policy and systems: a multi-country analysis from ten settings in Latin America and the Caribbean. Heal Res Policy Syst. 2019; 1(95). doi:10.1186/s12961-019-0484-4
https://doi.org/10.1186/s12961-019-0484-... ). Nevertheless, it cannot be assumed that the presence of a DM in the research process will inherently coproduce knowledge nor improve implementation. Along with the theoretical development of the possibilities that EIR offers, a technical exploration of how research teams are organized to collaborate and how replacement of DMs can affect EIR execution and results is still needed.

This article presents the findings of an empirical analysis of the experience of 13 research teams that each included a high-level DM as PI and a professional researcher as co-PI and that received financing to develop a 12-month EIR project to advance in the Sustainable Development Goals (SDG) (¹⁹19. Pan American Health Organization. Call for Applications: Embedding Research for the Sustainable Development Goals. Accessed 17 March 2023. Available from https://www.paho.org/en/topics/communicable-diseases/communicable-disease-research-and-partnerships-program/special-0
https://www.paho.org/en/topics/communica... ). This study stems from the aforementioned basic collaboration modalities between DMs and professional researchers. The objective of this study was to identify how the replacement of core members of the research team influenced the feasibility and value of EIR in the coproduction of evidence to improve the implementation of health policies, programs, and services in Latin America and the Caribbean.

MATERIALS AND METHODS

Study Design

This descriptive qualitative study was developed as part of the authors’ participation in the Technical Support Center (TSC) (¹⁸18. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Experiencias y retos del Centro de Apoyo Técnico en la investigación integrada en los servicios en América Latina y el Caribe. Rev Panam Salud Publica. 2021;45:e19. doi:10.26633/RPSP.2021.19
https://doi.org/10.26633/RPSP.2021.19... ) in the Embedding Research to Advance in the Sustainable Development Goals (ER-SDG) initiative launched in 2018 by the Pan American Health Organization, the Alliance for Health Systems Research, and the Special Program for Research and Training in Tropical Diseases (¹⁹19. Pan American Health Organization. Call for Applications: Embedding Research for the Sustainable Development Goals. Accessed 17 March 2023. Available from https://www.paho.org/en/topics/communicable-diseases/communicable-disease-research-and-partnerships-program/special-0
https://www.paho.org/en/topics/communica... ). The TSC provided external guidance to the teams for developing their implementation research protocols and throughout their development, but did not participate in their research activities.

Study Participants and Tools

This study included all core team members involved in 13 research projects in 11 countries in Latin America and the Caribbean from 2018 to 2019. Following the ER-SDG call, this study included DMs responsible for target health services or programs who were acting as PIs (although not necessarily trained in research activities) paired with an experienced professional researcher who served as a co-PI.

Data were obtained from 39 semistructured interviews (three per team) that were performed at three successive periods during the research process. Thirteen interviews were held in September 2018, at the onset of the study; 13 in July 2019, during the execution of the research projects; and 13 at the end, in November 2019.

The interviews sought to acknowledge and understand the teams’ experience with the embedding of research during the different stages of the initiative. The interviews were conducted by TSC members who specialized in health policy and systems research and had no previous working relationship with the interviewees.

Statistical Analysis

From the aforementioned collaboration modalities, data indicated the importance of core team permanence vs transitions, mainly the influence of DM replacement (¹⁸18. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Experiencias y retos del Centro de Apoyo Técnico en la investigación integrada en los servicios en América Latina y el Caribe. Rev Panam Salud Publica. 2021;45:e19. doi:10.26633/RPSP.2021.19
https://doi.org/10.26633/RPSP.2021.19... ). Analysis of the interviews divided the information into six categories with the following codes: (a) context in which the embedded research was conducted; (b) team composition; (c) development of the research project; (d) findings of the research project; (e) results utilization; and (f) assessment of the advice received from the TSC. This article reports on categories (b), (c), and (d).

The interviews were audio-recorded, transcribed verbatim, and coded and analyzed by three researchers (VBM, PTP, LAGB) using Atlas.ti, version 7.5.4 (Berlin, Germany), and interpretative triangulation was performed. The analysis was performed in Spanish; when necessary, the testimonies were translated into English for this article. Striving to render a clear and comprehensive report, we followed the COREQ (²¹21. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Heal Care. 2007;19(6):349-57. doi:10.1093/intqhc/mzm042
https://doi.org/10.1093/intqhc/mzm042... ) guidelines for items pertinent to this study.

Ethical approval

The protocol of the overall study was approved by the Ethics Committee of the National Institute of Public Health of Mexico (No. CI/2019/238). All participants provided oral and written informed consent. To preserve participant privacy, the research teams were assigned a two-part code composed of an acronym for the collaboration modality and a consecutive number: FC 1 to 6 for formal collaboration, ML 1 and 2 for mutual learning, and CPC 1 to 5 for the consolidation of the previous collaboration.

RESULTS

The Table presents the research teams establishing a relationship between the three collaboration modalities (¹⁸18. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Experiencias y retos del Centro de Apoyo Técnico en la investigación integrada en los servicios en América Latina y el Caribe. Rev Panam Salud Publica. 2021;45:e19. doi:10.26633/RPSP.2021.19
https://doi.org/10.26633/RPSP.2021.19... ) and the permanence of DMs and co-PIs throughout the EIR process. Using this general classification, it then discusses the ways in which the replacement of the core members of the teams influenced the execution and results of the research project.

Collaboration Modalities

As shown in the Table, six teams developed their research project on the basis of a formal collaboration between the DM and the professional researchers. Two more benefited from a mutual learning relationship, and in five instances, the DM and researchers consolidated previous collaboration experiences. Replacement of a DM engaged as the PI or project leader was not uncommon; one team even underwent two such changes. In some cases, the co-PI was also replaced, which was particularly relevant since the co-PI was also a high-level DM.

Most of the DMs’ replacements occurred in teams classified as formal collaborations, and only one happened in a context of consolidation of previous collaboration. Finally, we found that the DM’s area of influence, national or subnational (provincial/state), was not relevant regarding their permanence.

Association of DM Permanence with Research Execution and Results

On a few teams, the replacement of the DMs did not substantially affect the development of the research projects. However, on teams that retained their DM, the DM’s actual participation was not always aligned with the initial commitment to being the PI.

Based on evaluation of team performance and product analysis, the TSC identified three main situations regarding DM–PI and co-PI permanence or replacement related to the collaboration modalities. These three situations were corroborated by the data analysis of the interview information as:

1. Permanent core team (no DM/co-PI replacements) with either:

   1. (a)

      Active DM participation, or

   2. (b)

      Limited DM participation

2. DM or co-PI replacement that did not affect EIR

3. DM replacement that affected EIR

Situation 1: Permanent Core Team

Active DM participation. Notwithstanding particular differences, among the teams that did not experience changes, the direct involvement of the DMs ensured the swift development of the main, if not all, the research activities. These DMs actively participated in identifying the health system problem to be addressed, defining the research question, and elaborating the research protocol. They directed or were aware of the field work, analyzed and discussed the results with the research team, and read, commented on, and approved the final report being presented to the financing agency. One researcher’s testimony describes this:

Being perfectly aware of their formal and actual positions in the research process, the DM and the co-PI of a team that kept its original structure even perceived their roles as interchangeable. They produced a sound final report with high probabilities for uptake to improve the program’s implementation. The co-PI recognized this as follows:

In one case, with several researchers participating on the core team as co-PIs, successful collaboration with the DM became institutionalized. As members of this team affirmed, the existing legal framework ensured that the eventual replacement of a DM could not affect the research project.

The effective commitment of the DM also guaranteed regular and efficient communication with the whole research team and ensured a successful execution of the research process, as expressed by a DM who remained the PI:

Another DM expressed that the effective participation of the DM from the initial stages of the project was particularly relevant in ensuring its pertinence:

Limited DM participation. When a DM’s involvement was limited to a formal presence, mainly devoted to facilitating the research activities based on the trust placed on the co-PIs, this role was sufficient to ensure the development of valuable research products. As one interviewee stated:

Situation 2: DM or co-PI Replacement Did Not Affect EIR

Some teams were able to perform all research activities in a swift manner despite changes to their DM or co-PI. In these cases, either the replacements occurred during the early stages of the project or the complete core team was replaced a couple of weeks before the research protocol development workshop. Nevertheless, the participants argued that because their proposal had been previously developed within the Ministry of Health (MoH), the changes did not affect the project. On another team, the person initially committed as co-PI also held a relevant decision-making position in the MoH but left the MoH and the project shortly after the selection of grantees. At the same time, the PI held a high-level position in an institution linked to the national MoH but had little influence on decision-making. Nevertheless, a local DM successfully replaced the first co-PI and assumed an active role throughout the project, bringing it to completion as noted:

DM Replacement Affected EIR

The experience of the teams in which the DM was replaced during the first months and either continued the research activities based on a formal collaboration or lost all contact with the new DM revealed a key finding: replacement of the DM hindered the development of EIR. The research activities and the delivery of a final product were made possible only by the effort of the co-PIs and their collaborators inside or outside the health services organizations. This effort included the finalization of the research protocol and all research activities, data analysis, and final report elaboration, together with the intention to keep and restore the closest possible relationship with the health authorities. Yet, these situations raise questions concerning the actual embeddedness of research and the coproduction of knowledge as observed by an interviewee:

On another team, the replacement of the DM forced the co-PI to grapple with changes in several actors’ positions and roles inside the MoH. The necessary contacts to proceed with research activities were flawed and the whole EIR project was compromised at different times, which generated delays and uncertainty:

In another case, the response to changes in the health services’ most relevant posts was different. A lower-level DM holding a secondary role on the initial research team managed to make the necessary adjustments, and eventually led the research:

Another team went through several replacements of health authorities at the highest and intermediate levels. After the arrival of the third DM, this team’s research was essentially completed by the co-PI, an experienced academic researcher. This case once again raises doubts about the embeddedness of research, as expressed in this statement:

DISCUSSION

These findings highlight that DM replacement, given their high-level positions, hindered the execution of EIR projects and the coproduction of knowledge in several ways. Even though the replacement of DMs is something normal and generally associated with democratic succession of the ruling administration by a newly elected one, developing democracies do not always go through these changes smoothly; the transition may jeopardize the development of important health policies and programs. In these contexts, politically and even technically less relevant initiatives, such as EIR efforts, are easily subject to disruptions and need to be dealt with carefully (¹⁷17. García-Cerde R, Becerril-Montekio V, Langlois É, Reveiz L, Alcalde-Rabana J, Torres-Pereda P. Embedded implementation research determinants in Latin American Health Systems. Rev Saude Publica. 2021;23(55):16. doi:10.11606/s1518-8787.2021055003027
https://doi.org/10.11606/s1518-8787.2021... ).

EIR has been fostered by financing agencies whose calls demand the participation of high-level DMs as leaders of the research teams (¹⁰10. Langlois EV, Tran NT, Ghaffar A, Reveiz L, Becerra-Posada F. Embedding research in health policy and systems in the Americas. Rev Panam Salud Publica. 2017;41:e68. doi:10.26633/RPSP.2017.68
https://doi.org/10.26633/RPSP.2017.68... ,¹⁹19. Pan American Health Organization. Call for Applications: Embedding Research for the Sustainable Development Goals. Accessed 17 March 2023. Available from https://www.paho.org/en/topics/communicable-diseases/communicable-disease-research-and-partnerships-program/special-0
https://www.paho.org/en/topics/communica... ,²²22. Marten R, Reveiz L, Aslanyan G, Perez F, Ghaffar A. The value and impact of embedded implementation research: insights from Latin America and the Caribbean. Rev Panam Salud Publica. 2021;45:e110. doi:10.26633/RPSP.2021.110
https://doi.org/10.26633/RPSP.2021.110... ). Recent literature underscores that the involvement of these health DMs as PIs is the foundation of the potential of EIR to strengthen health systems and ensure pertinent research results’ uptake (¹¹11. Tran N, Langlois EV, Reveiz L, Varallyay I, Elias V, Mancuso A, et al. Embedding research to improve program implementation in Latin America and the Caribbean. Rev Panam Salud Publica. 2017;41:e75. doi:10.26633/RPSP.2017.75
https://doi.org/10.26633/RPSP.2017.75... ,¹³13. Olivier J, Scott V, Molosiwa D GL. Embedded systems approaches to health policy and systems research. In: Savigny D, Blanchet K, Adam T, eds. Applied Systems Thinking for Health Systems Research: a Methodological Handbook. New York: McGraw-Hill Education; 2017.

14. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Sys. 2020;18(64). doi:10.1186/s12961-020-00579-9
https://doi.org/10.1186/s12961-020-00579...

15. Langlois EV, Mancuso A, Elias V, Reveiz L. Embedding implementation research to enhance health policy and systems: a multi-country analysis from ten settings in Latin America and the Caribbean. Heal Res Policy Syst. 2019; 1(95). doi:10.1186/s12961-019-0484-4
https://doi.org/10.1186/s12961-019-0484-... -¹⁶16. Ghaffar A, Gupta A, Kampo A, Swaminathan S. The value and promise of embedded research. Health Res Policy Sys. 2021;19(suppl 2):99. doi:10.1186/s12961-021-00744-8
https://doi.org/10.1186/s12961-021-00744... ). Certain studies even develop relevant theoretical issues such as “context” and the “embedded research team attributes” (¹⁴14. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Sys. 2020;18(64). doi:10.1186/s12961-020-00579-9
https://doi.org/10.1186/s12961-020-00579... ). One article does mention that, in one of three study cases, the structure of the EIR team changed because of the reassignment of the co-PIs to new regions (²³23. Varallyay I, Bennet S, Keneddy C, Ghaffar A, Peters D. How does embedded implementation research work? Examining core features through qualitative case studies in Latin America and the Caribbean. Health Policy Plan. 2020;35(suppl 2):ii98-111. doi:10.1093/heapol/czaa126
https://doi.org/10.1093/heapol/czaa126... ). Yet the importance of how the mobility of PIs and co-PIs influence the EIR teams' performance in real-world experiences has not been sufficiently addressed.

The teams in which previous collaboration relationships between DMs and researchers were consolidated or in which new mutual learning partnerships were established based on effective collaboration between DMs and researchers, generally did not experience core team members’ replacements and offered the best examples of coproduction of knowledge with high possibilities of utilization. On the other hand, where the DMs’ participation was essentially formal, core team members’ replacements were frequent, limiting the real coproduction of knowledge. The combination of formal collaboration and core team replacements made it even more difficult to fully meet the goal of the ER-SDG initiative: health system and personnel capacity strengthening through EIR to advance toward the SDGs (¹⁸18. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Experiencias y retos del Centro de Apoyo Técnico en la investigación integrada en los servicios en América Latina y el Caribe. Rev Panam Salud Publica. 2021;45:e19. doi:10.26633/RPSP.2021.19
https://doi.org/10.26633/RPSP.2021.19... ).

As previous research demonstrates (¹⁸18. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Experiencias y retos del Centro de Apoyo Técnico en la investigación integrada en los servicios en América Latina y el Caribe. Rev Panam Salud Publica. 2021;45:e19. doi:10.26633/RPSP.2021.19
https://doi.org/10.26633/RPSP.2021.19... ), not only are high-level authorities quite mobile, which generates problems for the execution of the research efforts, sometimes the commitment of high-level DMs is limited to a merely formal participation. In these cases, the formal support that DMs can ensure for the execution of research along with the professionalism of academic researchers can produce rigorous scientific results. Nevertheless, questions arise about research being sufficiently embedded to strengthen the health system. No doubt it has been brought close to the interests, and maybe the needs, of DMs and the whole system; however, have the capacities of the people responsible of health programs and services really been upgraded? Do they identify themselves as coproducers of the knowledge that is relevant for their best performance?

These results led to an essentially technical recommendation. Together with a high-level DM, research teams should include someone in a lower-level position, who is relevant to the day-to-day implementations, and thus, less subject to biased replacements. Engaging them from the beginning may also help to ensure a more effective collaboration with external co-PIs. Even when a project stems from the academic interests of professional researchers, the inclusion of middle-level officials may foster the consideration of pertinent and relevant research questions.

Limitations

The main limitation of this study was that it focused on the development of the ER-SDG initiative and was secondarily informed by the previous EIR experience (⁹9. Ghaffar A, Langlois EV, Rasanathan K, Peterson S, Adedokun L, Tran NT. Strengthening health systems through embedded research. Bull World Health Organ. 2017;95(2):87. doi:10.2471/BLT.16.189126
https://doi.org/10.2471/BLT.16.189126...

10. Langlois EV, Tran NT, Ghaffar A, Reveiz L, Becerra-Posada F. Embedding research in health policy and systems in the Americas. Rev Panam Salud Publica. 2017;41:e68. doi:10.26633/RPSP.2017.68
https://doi.org/10.26633/RPSP.2017.68... -¹¹11. Tran N, Langlois EV, Reveiz L, Varallyay I, Elias V, Mancuso A, et al. Embedding research to improve program implementation in Latin America and the Caribbean. Rev Panam Salud Publica. 2017;41:e75. doi:10.26633/RPSP.2017.75
https://doi.org/10.26633/RPSP.2017.75... ). Some of its elements may be applied to understanding efforts in low and middle-income countries; however, the findings mainly pertain to the Latin American and Caribbean context and cannot be generalized. In addition, author involvement in the initiative was essential to closely following-up on the research teams. Yet this involvement may have involuntarily biased their performance and our evaluation of it given that we earnestly worked to ensure their success.

Conclusions

Building on the theoretical assumption that the participation of DMs fosters the coproduction of pertinent knowledge, the identification of how PI and co-PI mobility influences the results of this kind of initiative highlights key aspects of the concrete implementation of the theoretical model of EIR. It underscores the need to revisit essential technical criteria concerning the composition of the research teams to consider them as candidates, and eventually, be selected to receive financing.

Deepening research on the way EIR is performed in the real-world is necessary to maximizing the effective allocation of resources. In the context of reduced financing for health policy and systems research (²⁴24. Becerra-Posada F, dos Santos Boeira L, García-Godoy B, Lloyd E, Martínez-Sánchez H, O’Donnell C, et al. Politics and political determinants of health policy and systems research funding in Latin America and the Caribbean. Public Heal Res Pract. 2021;31(4):1-7. doi:10.17061/phrp3142120.
https://doi.org/10.17061/phrp3142120... ), the role of international agencies becomes ever more essential to fostering the production of relevant evidence to inform decision-making. We propose that a more complex core research team, including at least three rather than two stakeholders (a high-level DM, a technically relevant implementer who can replace the DM, and a professional researcher) would bring better results from EIR initiatives, strengthening the health system with upgraded research capacity and larger evidence uptake possibilities.

Disclaimer.

Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the RPSP/PAJPH and/or the Pan American Health Organization (PAHO).

Acknowledgement.

The authors are grateful to Ludovic Reveiz, Freddy Perez, and Vanessa Elias (Pan American Health Organization) and Etienne Langlois (World Health Organization) for their support during the Embedding Research to Advance the Sustainable Goals initiative (ER-SDG). We thank the decision-makers, the researchers, and the health staff who, as members of the ER-SDG teams from Latin America and the Caribbean, agreed to participate in this research on their collaboration experiences.

REFERENCES

Publication Dates

History