ABSTRACT

Objectives.

This paper describes and analyzes embedded implementation research and the empirical processes of planning for utilization, strategies to promote utilization and the sustainability of utilization of results from research led by decision-makers in Latin America and the Caribbean.

Methods.

This qualitative, descriptive and interpretive study is based on the findings from semistructured interviews with members of teams working under the Embedding Research for the Sustainable Development Goals initiative (2018–2019) as well as their responses to a self-assessment follow-up questionnaire 1 year after the project was completed.

Results.

Altogether 13 teams from 11 countries participated in the Initiative. Nine teams had a core team composed of a decision-maker as the principal investigator assisted by a researcher as co-principal investigator. Four teams included more than one co-principal investigator; and in five teams, the originally assigned principal investigator was replaced. There was an interesting relationship between the expected utilization of research results, the utilization strategies, the sustainability of research uptake and the teams’ collaboration modalities. When decision-makers and co-principal investigators were active participants, the intention to use the results and strategies for utilization were clearly oriented to improve implementation. In teams with basically a formal collaboration between the two principals, plans for utilization were unclear or focused on producing academic knowledge. The participation of implementers below the rank of principal investigator decision-maker may be relevant.

Conclusions.

Embedded implementation research is an innovative tool that may foster the utilization of research and strengthen health programs and services. Considering the internal dynamics of such research teams will enhance planning and strategies for research utilization as well as the sustainability of practical and actionable findings.

Keywords
Implementation science; health services research; Latin America; Caribbean region

RESUMEN

Objetivos.

En este artículo se describe y analiza la incorporación de la investigación sobre implementación y los procesos empíricos de planificación de su utilización, las estrategias para promoverla y la sostenibilidad de la utilización de los resultados de la investigación dirigida por los responsables de la toma de decisiones en América Latina y el Caribe.

Métodos.

Este estudio cualitativo, descriptivo e interpretativo se basa en los resultados de entrevistas semiestructuradas a miembros de equipos que trabajan en el marco de la iniciativa Incorporación de la investigación para avanzar en el cumplimiento de los Objetivos de Desarrollo Sostenible (2018-2019), así como en sus respuestas a un cuestionario de autoevaluación de seguimiento un año después de la finalización del proyecto.

Resultados.

En la iniciativa participaron un total de 13 equipos de 11 países. Nueve equipos disponían de un equipo central formado por un responsable de la toma de decisiones como investigador principal, que contaba con la colaboración de un investigador como coinvestigador principal. Cuatro equipos tenían más de un coinvestigador principal, y en cinco equipos se sustituyó al investigador principal asignado inicialmente. Se observó una relación interesante entre la utilización prevista de los resultados de la investigación, las estrategias de utilización, la sostenibilidad de la adopción de la investigación y las modalidades de colaboración de los equipos. Cuando había una participación activa de los responsables de la toma de decisiones y los coinvestigadores principales, el objetivo del uso de los resultados y las estrategias de utilización estaban claramente orientados a la mejora de su implementación. En los equipos en los que había una colaboración básicamente formal entre los dos directores, los planes de utilización no eran claros o se centraban en la producción de conocimiento académico. La participación de responsables de la puesta en práctica con cargos inferiores a los de investigador principal con capacidad de decisión puede ser un factor de interés.

Conclusiones.

La incorporación de la investigación sobre implementación es una herramienta innovadora que puede fomentar la utilización de la investigación y fortalecer los programas y servicios de salud. La consideración de la dinámica interna de estos equipos de investigación mejorará la planificación y las estrategias para su utilización, así como la sostenibilidad de los resultados prácticos y aplicables.

Palabras clave
Ciencia de la implementación; investigación sobre servicios de salud; América Latina; Región del Caribe

RESUMO

Objetivos.

Este artigo descreve e analisa a pesquisa com implementação incorporada e processos empíricos de planejamento da aplicação, estratégias de promoção do uso e sustentabilidade da utilização dos resultados de pesquisas lideradas por tomadores de decisão da América Latina e do Caribe.

Métodos.

Este estudo qualitativo, descritivo e interpretativo baseia-se nos achados de entrevistas semiestruturadas com membros de equipes que trabalharam no âmbito da iniciativa “Incorporação de Pesquisas para os Objetivos de Desenvolvimento Sustentável” (2018–2019), além de suas respostas a um questionário de autoavaliação aplicado um ano após a conclusão do projeto.

Resultados.

Treze equipes de 11 países participaram da iniciativa. Nove equipes contavam com uma equipe central composta por um tomador de decisão como investigador principal, auxiliado por um pesquisador como coinvestigador principal. Quatro equipes incluíram mais de um coinvestigador principal; e cinco equipes substituíram o investigador principal designado inicialmente. Houve uma relação interessante entre a aplicação esperada dos resultados da pesquisa, as estratégias de uso, a sustentabilidade da adoção da pesquisa e as modalidades de colaboração das equipes. Quando os tomadores de decisão e os coinvestigadores principais eram participantes ativos, a intenção de usar os resultados e as estratégias de utilização estavam claramente orientadas para o aprimoramento da implementação. Nas equipes em que a colaboração entre os dois investigadores principais era basicamente uma formalidade, os planos de utilização não estavam claros ou se concentravam na produção de conhecimento acadêmico. A participação de implementadores abaixo da posição de tomador de decisão do pesquisador principal pode ser importante.

Conclusões.

A pesquisa com implementação incorporada é uma ferramenta inovadora que pode promover a aplicação das pesquisas e fortalecer os programas e serviços de saúde. Levar em conta a dinâmica interna dessas equipes de pesquisa aprimorará o planejamento e as estratégias de utilização da pesquisa, bem como a sustentabilidade de achados práticos e implementáveis.

Palavras-chave
Ciência da implementação; pesquisa sobre serviços de saúde; América Latina; Região do Caribe

Since the 2004 Ministerial Summit on Health Research (¹1. Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder A, et al. Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet. 2004;364(4):900–6., ²2. Bennett S, Frenk J, Mills A. The evolution of the field of health policy and systems research and outstanding challenges. Health Res Policy Syst. 2018;16(1):43.), hosted by Mexico, it has been widely acknowledged that to effectively improve a population’s health status, it is necessary to narrow the gap between knowledge production and decision-making (²2. Bennett S, Frenk J, Mills A. The evolution of the field of health policy and systems research and outstanding challenges. Health Res Policy Syst. 2018;16(1):43.–⁵5. Martens P, Roos N. When health services researchers and policy makers interact: tales from the tectonic plates. Healthc Policy. 2005;1(1):72–84.). Relevant efforts are being made in what the literature mostly identifies as either embedded research or integrated knowledge translation (⁶6. Vindrola-Padros C, Pape T, Utley M, Fulop NJ. The role of embedded research in quality improvement: a narrative review. BMJ Qual Saf. 2017;26(1):70-80.–⁹9. World Health Organization. Strategy on health policy and systems research: changing the mindset. Geneva: World Health Organization; 2012. https://iris.who.int/handle/10665/77942
https://iris.who.int/handle/10665/77942... ).

One of these recent endeavors is the Embedding Research for the Sustainable Development Goals initiative, launched in 2018 by the Pan American Health Organization, the Alliance for Health Policy and Systems Research and the Special Programme for Research and Training in Tropical Diseases. The Initiative’s call for proposals clearly stated, “Our conception of embedded research involves leadership of the key stakeholders involved in policy generation and/or program management…” to foster “…the integration of scientific inquiry into the implementation problem-solving process, along with programs, policy and system level improvements in an iterative and continuous manner” (¹⁰10. Pan American Health Organization. Call for applications: Embedding Research for the Sustainable Development Goals [Internet]. Washington (DC): PAHO; 2018 [cited 2024 May 22]. Available from: https://www.paho.org/en/topics/communicable-diseases/communicable-disease-research-and-partnerships-program/special-0
https://www.paho.org/en/topics/communica... ). Based on the essential idea that implementation research is at its best when informed by what implementers know about their own activity (¹¹11. Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. BMJ. 2013;347:f6753.), the 2018 call requested that research proposals included a decision-maker or implementer as the principal investigator and a research expert from academia or a health institution as a co-principal investigator. Proposals were to focus on a program-, policy- or system-level problem related, but not limited, to Sustainable Development Goals (SDGs) 3 (i.e. Ensure healthy lives and promote well-being for all at all ages) and 10 (i.e. Reduce inequality within and among countries), providing a solid basis for their further development as implementation research protocols, with the participation of a regional technical assistance center (TAC). The inclusion of a TAC, an independent group of professional researchers with longstanding experience in implementation research, ensured that teams would produce sound results for embedded implementation research (¹⁰10. Pan American Health Organization. Call for applications: Embedding Research for the Sustainable Development Goals [Internet]. Washington (DC): PAHO; 2018 [cited 2024 May 22]. Available from: https://www.paho.org/en/topics/communicable-diseases/communicable-disease-research-and-partnerships-program/special-0
https://www.paho.org/en/topics/communica... , ¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41.).

Thirteen teams from 11 countries in Latin America and the Caribbean were selected from more than 200 proposals that met the two initial demands for embedded research and implementation research focusing on SDGs. Teams also had to meet six evaluation criteria (¹⁰10. Pan American Health Organization. Call for applications: Embedding Research for the Sustainable Development Goals [Internet]. Washington (DC): PAHO; 2018 [cited 2024 May 22]. Available from: https://www.paho.org/en/topics/communicable-diseases/communicable-disease-research-and-partnerships-program/special-0
https://www.paho.org/en/topics/communica... ), mainly to ensure that there was evidence of effective engagement to support evidence-informed decision-making and to show their willingness to use the results to improve policies, programs or systems. This paper focuses on this willingness, aligned with the Initiative’s fifth objective, to “integrate research findings from the health policy and systems research study in order to improve the implementation of health programs/policies and functioning of the overall health system” (¹⁰10. Pan American Health Organization. Call for applications: Embedding Research for the Sustainable Development Goals [Internet]. Washington (DC): PAHO; 2018 [cited 2024 May 22]. Available from: https://www.paho.org/en/topics/communicable-diseases/communicable-disease-research-and-partnerships-program/special-0
https://www.paho.org/en/topics/communica... ).

Although there is abundant literature highlighting the advantages of embedded research (⁹9. World Health Organization. Strategy on health policy and systems research: changing the mindset. Geneva: World Health Organization; 2012. https://iris.who.int/handle/10665/77942
https://iris.who.int/handle/10665/77942... , ¹³13. Gagliardi AR, Dobrow MJ. Identifying the conditions needed for integrated knowledge translation (IKT) in health care organizations: qualitative interviews with researchers and research users. BMC Health Serv Res. 2016;16:256.–¹⁷17. Ghaffar A, Langlois EV, Rasanathan K, Peterson S, Adedokun L, Tran NT. Strengthening health systems through embedded research. Bull World Health Organ. 2017;95(2):87.), there is a paucity of studies focusing on the development of these research efforts in relation to: (i) how decision-makers and researchers collaborate; (ii) the influence of the composition of a research team and its collaboration modalities on the pertinence and excellence of their results; (iii) how results from embedded research are initially intended to and have actually been used, depending on items i and ii (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41., ¹⁸18. García-Cerde R, Becerril-Montekio V, Langlois E, Reveiz L, Alcalde-Rabanal J, Torres-Pereda P. Embedded implementation research determinants in Latin American health systems. Rev Saude Publica. 2021;55:16.–²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77.).

Although all teams in this study were initially integrated as specified in the call for proposals, with a core team that included principal and co-principal investigators, previous studies have observed that the actual dynamics, composition and collaboration modalities of the teams were diverse (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41.). Additionally, in some cases projects were carried out by a larger team that included other staff from the participating research or health institution. These particularities not only ensured or hindered the pertinence and excellence of the projects but also marked the overall experience of the embedded implementation research (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41., ¹⁸18. García-Cerde R, Becerril-Montekio V, Langlois E, Reveiz L, Alcalde-Rabanal J, Torres-Pereda P. Embedded implementation research determinants in Latin American health systems. Rev Saude Publica. 2021;55:16., ²⁰20. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Replacement of core team members on embedded implementation research teams: experiences from Latin America and the Caribbean. Rev Panam Salud Publica. 2023;47:e82.).

We have identified three collaboration modalities to describe how members of the core team relate: (i) basically formal (i.e. predominant participation by professional researchers), (ii) mutual learning (i.e. active participation by decision-makers, with both the principal and co-principal investigators sharing ownership of the research products) and (iii) consolidation of previous collaboration (i.e. through horizontal collaboration that strengthens the capacities of the decision-makers and the researchers working with them) (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41.).

Science utilization studies refer to different scales (²²22. Landry R, Amara N, Lamari M. Climbing the ladder of research utilization: evidence from social science research. Sci Commun. 2001;22(4):396-422.) and utilization processes (²³23. Cherney A, McGee TR. Utilization of social science research: results of a pilot study among Australian sociologists and criminologists. J Sociol. 2011;47(2):144-62.). The offer and demand model of scientific production sheds some light on the general model in which embedded implementation research can be placed, since it refers to the integration of decision-makers, who are the final users of the research produced (²²22. Landry R, Amara N, Lamari M. Climbing the ladder of research utilization: evidence from social science research. Sci Commun. 2001;22(4):396-422.). Despite embedded implementation research placing decision-makers at the center of the production of knowledge, thus making the adoption of results more probable, several authors have highlighted different challenges and contradictions on what was initially seen as a direct path to utilization (²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77., ²⁴24. Varallyay NI, Bennett SC, Kennedy C, Ghaffar A, Peters DH. How does embedded implementation research work? Examining core features through qualitative case studies in Latin America and the Caribbean. Health Policy Plan. 2020;35(Suppl. 2):ii98-111.).

The debate about the actual integration of knowledge into health systems underscores the roles of decision-makers and the research team (²⁴24. Varallyay NI, Bennett SC, Kennedy C, Ghaffar A, Peters DH. How does embedded implementation research work? Examining core features through qualitative case studies in Latin America and the Caribbean. Health Policy Plan. 2020;35(Suppl. 2):ii98-111.), the contradictions of scientific knowledge itself and the organizational environment in which the knowledge is supposed to be used (²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77.). The literature points out that the mere presence of decision-makers does not suffice because they need enough power to execute the proposed changes (²⁴24. Varallyay NI, Bennett SC, Kennedy C, Ghaffar A, Peters DH. How does embedded implementation research work? Examining core features through qualitative case studies in Latin America and the Caribbean. Health Policy Plan. 2020;35(Suppl. 2):ii98-111.), and their presence on the research team has to be carefully handled in terms of the use of power, possible bias and their particular interests as part of the system (²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77.). The balance of people working on the team is key to correctly conducting a study and generating actionable, scientifically sound results (²⁵25. Puga F, Stevens KR, Patel DI. Adopting best practices from team science in a healthcare improvement research network: the impact on dissemination and implementation. Nurs Res Pract. 2013;2013 814360.).

Using embedded implementation research is more complex than just making a decision or involving a decision-maker. Important input chains might be out of the decision-maker’s control, and there may even be contradictions between the research results and decision-making itself. Since it is not always possible to apply results exactly as they are formulated, adopting the results of embedded implementation research requires an appropriate mix of fidelity and flexibility (²³23. Cherney A, McGee TR. Utilization of social science research: results of a pilot study among Australian sociologists and criminologists. J Sociol. 2011;47(2):144-62.).

We know little about the utilization of such research and the real impact of research on everyday health practices (²³23. Cherney A, McGee TR. Utilization of social science research: results of a pilot study among Australian sociologists and criminologists. J Sociol. 2011;47(2):144-62.). In the field of embedded implementation research, studies are more focused on aspects such as the co-production of knowledge, while evidence about its utilization is sparse (²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77., ²⁴24. Varallyay NI, Bennett SC, Kennedy C, Ghaffar A, Peters DH. How does embedded implementation research work? Examining core features through qualitative case studies in Latin America and the Caribbean. Health Policy Plan. 2020;35(Suppl. 2):ii98-111.). Research utilization has been classified as instrumental (i.e. providing technical support), conceptual (i.e. providing theoretical support) and symbolic (i.e. providing ideological support) (²³23. Cherney A, McGee TR. Utilization of social science research: results of a pilot study among Australian sociologists and criminologists. J Sociol. 2011;47(2):144-62.). This paper focuses on its instrumental, or technical, use to resolve particular implementation problems.

This paper describes and analyses the empirical process of how the 13 teams working under the Embedding Research for the Sustainable Development Goals initiative intended to use their results, the general scope of the team, the utilization strategies they proposed and their sustainability, and if and how the results were finally used.

METHODS

Study design and participants

This qualitative, descriptive and interpretive study is based on semistructured interviews conducted with core members of the 13 teams from 11 countries participating in the Initiative from September 2018 to December 2019 and on responses to a follow-up self-assessment questionnaire with open-ended questions administered in November 2020. Each core team included at least two people: a decision-maker acting as the principal investigator and a professional researcher acting as a co-principal investigator. In what we designate as larger teams, two or more people also assisted the co-principal investigator. Most interviews were conducted with only the principal or co-principal investigator, but in 14 of the 39 interviews both of them participated. The TAC developed the follow-up questionnaire to gather information about the utilization of results in November 2020, 1 year after the end of the Initiative, and emailed it to each core team. Responses were received from 10 principal or co-principal investigators, representing 9 of the teams

Instruments and data collection

Three semistructured interviews were conducted with each team, for a total of 39 interviews. The first interviews, in September 2018, explored four issues: (i) previous knowledge about embedded research, (ii) the composition of the research team, (iii) the context and operational conditions under which the embedded implementation project was conducted, and (iv) the considerations for utilizing the results of the research. The second interviews, in July 2019, focused on the concepts underpinning the embedded research and experience of the research. The third interviews, in November 2019, explored issues similar to those in the first interview and added themes about (i) fieldwork experience, (ii) institutional support and how resources were managed by the financing agencies and (iii) the respondent’s perceptions of the assistance provided by the TAC. The follow-up questionnaire, emailed in November 2020, focused on utilization of the research results and the sustainability of embedded research activities. Only 9 of 13 teams responded to this questionnaire.

Research team

The interviews were conducted by three members of the TAC who had proven experience in public health and qualitative research (VBM, PTP, JAR). TAC members were completely independent from the financing agencies and had no conflicts of interest or working relationship with the people or institutions participating in the teams (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41.). The anonymized data were stored in a restricted-access TAC Google Drive.

Data analysis

A thematic analysis of the data (²⁶26. Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods. 2006;5(1):80-92.) was made using axial coding (²⁷27. Marshall C, Rossman GB. Designing qualitative research. 6th ed. Thousand Oaks (CA): Sage; 2014.). Responses from the interviews and the emailed follow-up questionnaire were transcribed verbatim in Spanish and in English. The interviews and questionnaires in Spanish were translated to English for this paper.

The researchers developed a codebook for the responses that included definitions and examples, based mainly on an iterative process, using the interview guides in a deductive manner, as well as by using the collected data in an inductive process to enable the generation of new codes, as necessary. Data were organized using Atlas-ti software (Berlin, Germany). Three researchers independently coded and interpreted the data (VBM, PTP, LAGB). Once fully coded, the interviews were read again to identify new subthemes and findings. Using interpretive triangulation, the coding team reached consensus regarding the main themes. Disagreements were resolved through discussion and a second revision of the transcripts. The main findings reached theoretical saturation and are considered to be central results (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41.). Emergent or exceptional results are also reported (²⁸28. Meadows LM, Morse JM. Constructing evidence within a qualitative project. In: Morse JM, Swanson JM, Kuzel AJ, editors. The nature of qualitative evidence. Thousand Oaks (CA): Sage; 2001. p. 187–201.). All findings are supported by interviewees’ responses.

Twenty-seven codes were generated. This paper examines three main codes directly related to utilization of the results of the embedded implementation research: scope, strategies and sustainability. Scope was used to explore the initial vision of participants regarding how they intended to use their research results. The strategies code explored how to promote the results of the research and achieve utilization. Sustainability was measured using data obtained from the follow-up questionnaire at 1 year after the completion of the Initiative, and it focused on the actual utilization of research after the end of the projects and elicited information about whether it had been possible to maintain embedded research activities throughout that time.

Responses are presented using codes that identify three kinds of participants: principal investigators, coded as PIs (i.e. decision-makers); co-principal investigators, coded as co-PIs (i.e. the professional researchers); and members of the larger research team (coded as larger team). Each quote corresponds to one person’s answers. The origin of the responses is identified as pre-, mid- or post-, in alignment with the three interviews, and as follow up, for the follow-up questionnaire. Teams are identified according to their anonymized collaboration modality and the number assigned to the team: basically formal (i.e. BF-Tx), mutual learning (i.e. ML-Tx) and consolidation of previous collaboration (i.e. CC-Tx).

Ethics

The research protocol received clearance from the Research Committee of the National Institute of Public Health of Mexico (Proyecto Secundario Comité de Investigación: 1454). Participants signed a mandatory informed consent letter, and this has been properly stored by the authors. The names of people, cities and specific health conditions have been anonymized to better preserve confidentiality.

RESULTS

Thirteen teams from 11 countries participated in the Embedding Research for the SDGs initiative and this study. Those countries are Argentina, Bolivia (Plurinational State of), Brazil, Colombia, the Dominican Republic, Ecuador, Guatemala, Guyana, Haiti, Paraguay and Peru. Nine had a core team composed of a decision-maker as the principal investigator assisted by a researcher as a co-principal investigator. Four teams were larger and included more co-principal investigators; in five teams, the originally assigned principal investigator was replaced (²⁰20. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Replacement of core team members on embedded implementation research teams: experiences from Latin America and the Caribbean. Rev Panam Salud Publica. 2023;47:e82.).

Expected utilization of research results

In teams in which decision-makers and the co-principal investigators collaborated actively and harmoniously, classified either as mutual learning or consolidation of previous collaboration, the intention to use research results was clearly expressed and oriented towards improving programs or services. In contrast, on teams in which the relationship between the principal and co-principal investigators was characterized as basically formal collaboration, initial ideas about utilizing results were less oriented towards improving health programs or services and more focused on producing academic knowledge (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41.).

Two responses show the clear aim of utilizing research to improve programs or services or, at least, its consideration as a foundation for decision-making.

Contrasting with these statements, teams with basically formal collaboration expressed that their projects would generate the basis for carrying out more research.

Members of basically formal teams mentioned that their main plan for utilizing their research was to publish scientific articles.

Strategies to promote research utilization

Active and engaged collaboration between principal and co-principal investigators resulted in clearly formulated strategies.

Lots of diffusion, a lot of lobbying the authorities, a lot of knowledge management…policy briefs, for example, conversations for analysis and reflection during which we will be able to present our results and tell them why [the results] are important and how they will help to improve the success of treatment. — PI-post-CC-T1

In teams in which academics led the research, the strategies were not clear, envisioned producing more research and scientific papers or were nonexistent.

Research utilization and sustainability

Responses from the principal investigator of a team classified as consolidation of previous collaboration describe how the research results were used.

A similar response pinpoints other instances of utilization.

An emergent finding highlighted another relevant issue influencing the whole research process: the replacement of health services staff, mainly at the highest decision-making levels, at different times during the projects (¹⁸18. García-Cerde R, Becerril-Montekio V, Langlois E, Reveiz L, Alcalde-Rabanal J, Torres-Pereda P. Embedded implementation research determinants in Latin American health systems. Rev Saude Publica. 2021;55:16.).

Another emergent finding signaled that the presence of implementers with lower decision-making responsibilities compared with principal investigators but with longer experience in a program can help ensure there is improvement in a program.

Table 1 illustrates the different collaboration modalities used between decision-makers and professional researchers in terms of the expected utilization of the research, the strategies used and the actual utilization of research.

DISCUSSION

This study shows a link between the composition of research teams, their research utilization strategies, their actual use of research and the sustainability of embedded research. On some teams collaboration between decision-makers and researchers was associated with the ensuing use of research results, while other teams fell short of this goal. Emergent findings also highlight the effect of the replacement of decision-makers on the utilization of results and how members of a team who have less influence on decision-making may promote the uptake of results.

Although some studies underscore the importance of the composition of the research team (¹⁹19. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Syst. 2020;18(1):64., ²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77., ²⁴24. Varallyay NI, Bennett SC, Kennedy C, Ghaffar A, Peters DH. How does embedded implementation research work? Examining core features through qualitative case studies in Latin America and the Caribbean. Health Policy Plan. 2020;35(Suppl. 2):ii98-111.), they do not fully consider its eventual consequences on the utilization of results from embedded implementation research. However, our results suggest that research teams are dynamic and may or may not consolidate over time. Analyzing the experience of teams that were unable to coalesce may be a rich source of information for future similar endeavors.

As mentioned above, the mere presence of a decision-maker on a research team is not sufficient (²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77.). This person must have real agency to enact decisions, be aware of the everyday processes and be able to ensure that the research results are timely for the programmatic structure of the institution (¹⁹19. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Syst. 2020;18(1):64.).

This study found no evidence that research teams complied with one of the two main criteria defined by the financing agency for the selection of proposals – that is, to include evidence of effective engagement to support evidence-informed decision-making. Before the teams were selected and even at the outset of the Initiative, only a formal declaration in a proposal could be considered as evidence of such engagement.

If those who submit proposals try to avoid this problem, it might undermine the trust of the financing agency. More importantly, it might close the door to the establishment of completely new collaborations between decision-makers and professional researchers who cannot demonstrate evidence of effective engagement before a new project begins. This is not a flaw that can be attributed to the financing agency but a reality that needs to be properly dealt with in the future.

As we have highlighted elsewhere, the inclusion of technical personnel with relevant skills as implementers in the research team may ensure the teams’ better commitment and also continuity when high-level decision-makers are subject to politically motivated replacement (¹⁸18. García-Cerde R, Becerril-Montekio V, Langlois E, Reveiz L, Alcalde-Rabanal J, Torres-Pereda P. Embedded implementation research determinants in Latin American health systems. Rev Saude Publica. 2021;55:16., ²⁰20. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Replacement of core team members on embedded implementation research teams: experiences from Latin America and the Caribbean. Rev Panam Salud Publica. 2023;47:e82.). Technical personnel may ensure there is direct knowledge of the chains of command, that actions are timely and provide insight into the particular ways in which power affects a policy’s or program’s implementation (²¹21. Phillips J, MacLeod B, Kachura P. Bugs in the bed: addressing the contradictions of embedded science with agile implementation research. Glob Health Sci Pract. 2021;9(1):55-77., ²²22. Landry R, Amara N, Lamari M. Climbing the ladder of research utilization: evidence from social science research. Sci Commun. 2001;22(4):396-422.). Thus, we propose teams that initially function based on formal relationships may evolve into those that embrace mutual learning and consolidation in the long run.

Concerning the willingness to use results to improve policies, programs or systems, our findings show more clearly that teams classified as using mutual learning or consolidation of previous collaboration have more potential to use research results, may have the necessary strategies to promote this use, and may actually and effectively use the results of the research (¹²12. Becerril-Montekio V, Torres-Pereda P, García-Bello LA, Alcalde-Rabanal J. Embedding research into health services in Latin America and the Caribbean: experiences and challenges of the technical support center. Rev Panam Salud Publica. 2021;45:e41.). Conversely, teams based on basically formal collaboration have less potential to utilize research results, have no or only vague strategies to promote the research and, finally, do not utilize the results of the embedded implementation research. This means that while on the one hand decision-makers may have been stimulated to participate in research activities, on the other, academic researchers may not necessarily feel as committed to using research results or take actions to use them.

There are three difficulties with not utilizing results from embedded implementation research: (i) public money spent by cooperating agencies might not be helping to solve the implementation problems it is meant to address; (ii) local and international workforces are dedicating time and resources to projects that are not efficacious; and (iii) opportunities to stimulate and reaffirm the value of research and ensure capacity-building are being missed where they are most needed.

This study reflects on how teams engaging in embedded implementation research envision, plan and use the results of their research, and our findings may guide future similar endeavors. The main limitation of our study is that it is based on an analysis of the 2018 Embedding Research for the SDGs initiative, so it does not offer enough evidence that could be generalized to other similar research experiences. Yet it does provide several examples of what may happen in low- and middle-income countries, particularly in Latin American and the Caribbean. Another limitation is the authors’ involvement as members of the TAC, which may have biased some of our findings.

Conclusions

Embedded implementation research is certainly a useful and innovative tool for closing the gap between research and strengthening health systems, programs and services. Yet it still needs to be and must be tested and improved. Ensuring there is real collaboration among implementers and researchers, and, thus, considering the internal dynamics of embedded implementation research teams will enhance planning and strategies for research utilization as well as the sustainability of practical and actionable findings to advance this methodology and improve initiatives aimed at health systems strengthening.

Disclaimer.

Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the Revista Panamericana de Salud Pública/Pan American Journal of Public Health or the Pan American Health Organization.

Acknowledgements.

We thank the decision-makers, the researchers and the health staff who, as members of the Embedding Research for the SDGs teams from Latin America and the Caribbean, agreed to participate in this research about their collaboration experiences.

REFERENCES

Publication Dates

History