ABSTRACT

The objectives of this study were to identify and analyze the contributions of the Research Management Committee for access to outcomes related to implementation, in a research conducted in the mental health field. It is a qualitative study that adopted as a methodological resource the accomplishment of a focus group and the data collection of the minutes of the meetings held by the Committee. By means of the study, it was possible to identify that the Research Management Committee favored the participation of the interested parties in several aspects of the research; made possible the evaluation and monitoring of the meaning and feasibility of the research in the field of study, from the perspective of those who live the experience of daily work and care; and favored access to some outcomes of implementation research in a continuous and more meaningful way for those who would benefit from it. In this sense, the researchers and agencies involved in this type of study are suggested to adopt this tool as a possibility to make the research a more dialogic and potentially transformative process.

KEYWORDS
Methods; Health evaluation; Program evaluation

Introduction

Among the main advances in the study of public policies, the global growth of implementation research¹1 Peters DH, Adam T, Alonge O, et al. Implementation research: what it is and how to do it. Br. j. sports med. 2014; 48:731-736. stands out. As the final stage of a series of events related to translational science, this type of research corresponds to the scientific study of the many processes that guide the translation of public policies into actions or effective changes in the practice of individuals or organizations²2 Proctor EK, Landsverk J, Aarons G, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm. policy ment. health. 2009; 36(1):24-34..

The scope of this particular type of research considers many aspects of implementation, including the factors affecting it, its results, at individual and collective levels, and also the establishment of strategies for promoting the sustainability of the proposal and its wide use. Thus, it is summarized that the intention of this type of research is to understand how interventions work in ‘real-world’ scenarios¹1 Peters DH, Adam T, Alonge O, et al. Implementation research: what it is and how to do it. Br. j. sports med. 2014; 48:731-736.^,33 Powell BJ, Proctor E, Glass JE. A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions. Res soc work pract. 2014; 24(2):192-212..

In this sense, the context in which interventions take place plays a central role in implementation research. Since its focus is on bringing to light contextual elements that drive intervention to success or failure rather than trying to control context conditions or remove their influence, implementation research seeks to understand and work with the actual conditions of context¹1 Peters DH, Adam T, Alonge O, et al. Implementation research: what it is and how to do it. Br. j. sports med. 2014; 48:731-736..

It is emphasized that working with the context implies, as well, working with the actors that shape it. In addition to the macropolitical, economic and legal aspects, it is noteworthy that, especially at local levels, the role played by governments, workers and users, as well as how they interact, make them responsible for shaping the institutional environment, which may function both as facilitator and as a barrier to the implementation of policies and interventions⁴4 Palinkas LA, Horwitz SM, Green CA, et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm. policy ment. health. 2015; 42(5):533-544..

Considering the abovementioned assumptions, Proctor et al. (2011)⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76. proposed a theoretical model to guide the practice of implementation research in order to score their outcomes at different levels: outcomes related to implementation; service-related outcomes; and user-related outcomes. Each of these consists of a series of items that should be scored in the evaluation process. It is emphasized, here, items related to the first construct: acceptability; adoption; adequacy; costs; viability; penetration; and sustainability³3 Powell BJ, Proctor E, Glass JE. A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions. Res soc work pract. 2014; 24(2):192-212.^,55 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

The literature recommends that these items should be accessed through mixed methods that consider both quantitative and qualitative approach elements⁶6 Palinkas LA, Aarons GA, Horwitz S, et al. Mixed method designs in implementation research. Adm. policy ment. health. 2011; 38(1):44-53.. Additionally, there is the need for making access to the items a cross-cutting process, since, in addition to some items requiring evaluation at various stages of implementation, some of them will interact with each other and change over time⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76.. These changes need to be noticeable to the leaders of the process, who must also establish strategies to provide feedback to both implementing agencies, workers and users to encourage learning and the incorporation of change⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

In this sense, the research delineation ‘Innovative strategies to qualify the mental health network in a medium-sized city of São Paulo: an implementation research’ sought to include, as a cross-sectional component of the study, a Research Management Committee (RMC).

The aforementioned research concerns a partnership between the city hall of a medium-sized municipality in the state of São Paulo and a public health and mental health research laboratory at a public university in the same state. This is an ongoing research, funded by the Department of Science and Technology of the Ministry of Health, through Public notice MS-SCTIE-Decit/CNPq nº 12/2018, whose general objective is to execute and evaluate the impact of device implementation of network integration in the qualification of mental health care in the municipality.

It is noteworthy that, currently, one of the most critical issues in research on mental health services is the distance between what is recognized as good community mental health care practices and the reality that is experienced in these services by workers and users²2 Proctor EK, Landsverk J, Aarons G, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm. policy ment. health. 2009; 36(1):24-34.. In this sense, the field of mental health is a particularly fertile ground for the expansion of implementation research, which today is one of the main advances in the study of public policies at the global level²2 Proctor EK, Landsverk J, Aarons G, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm. policy ment. health. 2009; 36(1):24-34.^,33 Powell BJ, Proctor E, Glass JE. A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions. Res soc work pract. 2014; 24(2):192-212..

One of the main highlights of implementation researches is that the outcomes proposed in this type of study are particularly useful in identifying when an intervention does not achieve the expected success, whether this is because it was ineffective in the new scenario (intervention failure) or if a good intervention was implemented incorrectly (failed implementation)⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

Thus, conducting an Implementation Research in the context of network integration devices can be useful in helping to shed light on one of the most important gaps in mental health care in the Country: the persistence of a number of deadlocks and challenges for effectiveness of matrix support as a network articulation mechanism, even after a decade of its incorporation in health networks⁷7 Treichel CAS, Onocko-Campos RT, Campos GWS. Impasses e desafios para consolidação e efetividade do apoio matricial em saúde mental no Brasil. Interface (Botucatu). 2019; 23:1-18..

It is noteworthy that, although shared⁸8 Smith SM, Allwright S, O'Dowd T. Effectiveness of shared care across the interface between primary and specialty care in chronic disease management. Cochrane database syst. rev. 2007; 18(3):1-10. or collaborative⁹9 Bower P, Gilbody S, Richards D, et al. Collaborative care for depression in primary care. Making sense of a complex intervention: systematic review and meta-regression. Br J Psychiatry. 2006; 189:484-493. proposals, similar to matrix support, are consolidated around the world as a good practice in mental health care in Brazil, the success of these arrangements still bumps into problems that can be related to the implementation process. A recent literature review⁷7 Treichel CAS, Onocko-Campos RT, Campos GWS. Impasses e desafios para consolidação e efetividade do apoio matricial em saúde mental no Brasil. Interface (Botucatu). 2019; 23:1-18. showed aspects such as: lack of clear delineations for the practice of matrix-based strategies; deficiency in the training and qualification of professionals; and absence of institutionalized spaces, with systematic meetings of professionals to discuss cases and joint evaluation of the progress of activities.

The adoption of the RMC, in turn, was based on the understanding that this would be a useful tool in monitoring the outcomes proposed for implementation research. The assumption was that aspects such as constant assessments of the context and interested parties engagement needed to access outcomes would be favored by this tool as it is a strategy for fostering stakeholder participation in planning, implementation and in project evaluation.

It is noteworthy that this methodology has already been used in the participatory elaboration of indicators for mental health evaluation¹⁰10 Furtado JP, Onocko-Campos RT, Moreira MIB, et al. A elaboração participativa de indicadores para a avaliação em saúde mental. Cad. Saúde Pública. 2013; 29(1):102-110. and has favored important advances in order to give greater depth to the participation of workers in the research process. Another important record of its use was in a research that sought to implement the use of instruments capable of qualifying the use of psychiatric drugs among users of community mental health services¹¹11 Passos E, Otanari TMC, Emerich BF, et al. O Comitê Cidadão como estratégia cogestiva em uma pesquisa participativa no campo da saúde mental. Ciênc. Saúde Colet. 2013; 18(10):2919-2928.. In this experience, it was also found a significant increase in the scope of the participatory process, since, in addition to workers, users of services were also included in the process.

In the research that originated the present study, the RMC includes researchers linked to the educational institution, local managers and workers, allocated to the mental health services of the municipality, as well as members of the municipal management directly involved in the coordination of mental health services.

Linked to the educational institution are: the research coordinator, physician, doctor in public health; and three doctoral students in public health, one nurse, one psychologist and one occupational therapist. To these, it’s their duty to provide the methodological operationalization of research and the relationship with the development agency.

The members of the RMC linked to the municipal government, appointed by the municipal health department through an internal selection process, are five psychologists and two nurses, as follows: two coordinators of Specialized Care, two coordinators of the Psychosocial Care Centers of the municipality (one of the type II and one type Alcohol and Drugs – AD), two workers from the Central Specialties Ambulatory and one worker from the Child Care Center (which provides mental health care to children and adolescents). To these, they have the role of evaluating and giving feedback on the proposed operation, as well as communicating expectations, fears and suggestions of other workers regarding the services in which they are allocated.

The operation of the RMC takes place through systematic meetings – monthly or biweekly – in which representatives of the services involved in the research process meet to reflect on the meanings and usefulness of research for the field, thus, deliberating on how it will be led. All participants can express themselves freely and are encouraged to share their impressions. The meetings last approximately three hours and usually use the following script: update on the progress of the research, with possible presentation of results obtained since the last meeting; definition of meeting guidelines; discussion about the topics under discussion, proposing directives for the follow-up of the work; and consensus and agreement on upcoming activities.

In this study, the objectives were to identify and analyze the contributions of the RMC to access to implementation-related outcomes in research conducted in the field of mental health. Therefore, it is expected to provide information that can guide the decision of researchers and agencies involved in this type of study, regarding the adoption of the tool in question.

Material and methods

In this study, it was intended to identify the contributions of the RMC, both from the objective point of view, from the deliberations and adaptations operated by the Committee, and from the subjective point of view, from the perceptions of its members. Therefore, a qualitative study was conducted, which adopted as a methodological resource the realization of focus group and survey of data recorded in the minutes of the meetings held by the RMC.

By the time of the focus group, four meetings had already been held with the presence of all members of the RMC, as well as three previous meetings held with part of the team at the project design stage. The characterization of RMC members can be seen in chart 1. The meetings of the RMC were held monthly or bimonthly, depending on the period, and were held at both the educational institution and the services to which the members were linked. All minutes of these meetings (seven) were taken into account for this study.

The focus group itself was conducted after the fifth meeting of the Committee and was coordinated by a nurse, doctoral student in public health, and accompanied by a reporter/observer, psychologist, also doctoral student in public health, who took notes and assisted in conducting the group. Both the coordinator and the reporter/observer were linked to the RMC through the educational institution, where the session was held.

The group was held in a room of adequate size and place, without noise or interference, ensuring privacy and facilitating debate. All members of the Committee were present, seated around a table, with no predefined seats, so that everyone could have good eye contact, being at equal distance from the group coordinator. The session lasted approximately one hour and followed the following script: (1) Opening; (2) Clarification on the dynamics of the debate; (3) Establishment of the setting; (4) Debate; (5) Synthesis of previous moments; and (6) Closing.

For the debate, the coordinator used questions defined in a script built in conjunction with the research coordinator. The purpose of the questions was to bring out each member’s perceptions of their participation in the RMC. They were encouraged to respond individually and also interact with each other in order to explore and clarify individual and shared perspectives. The session lasted approximately one hour and was recorded with the consent of the participants, expressed by signing the Informed Consent Form (ICF).

To obtain the ICF, the participants were informed about the study objectives and the right to give up participating at any time, as well as the guarantee of anonymity. In this sense, in the transcripts, the names of the participants were replaced by letters and numbers. It is noteworthy that the present study was submitted and approved, under opinion nº 3.065.312, by the Ethics Committee of the Faculty of Medical Sciences of the State University of Campinas, following the Brazilian rules and guidelines for research regulation involving human subjects – Resolution of the National Health Council nº 466/2012, in addition to the provisions of the Declaration of Helsinki.

The analysis materials for this study were the transcription of the focus group audio and the registers, in minutes, of the previous meetings. It was decided to categorize the results according to the contributions of the RMC to explore each of the items that make up the construction of outcomes related to implementation, namely: acceptability; adoption; adequacy; costs; feasibility; penetration; and sustainability⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

For this categorization, the authors were guided by the thematic categorical content analysis model¹²12 Minayo MCS. O desafio do conhecimento: pesquisa qualitativa em saúde. 5. ed. São Paulo: Hucitec-Abrasco; 1998.. That is, the analysis was conducted in stages, the first related to the breakdown of the texts into units and categories, and the second, related to the analytical grouping through the organization of messages, from the predefined categories. In the second moment, each item was discussed in order to establish a consensus among the researchers as to its relevance in the proposed category.

Results and discussions

The increase in implementation researches occurs at the moment when the recognition of evidences and experiences validated by research results as qualifiers of construction and management of health policies¹³13 Campbell DM, Redman S, Jorm L, et al. Increasing the use of evidence in health policy: practice and views of policy makers and researchers. Aust. New Zealand health policy. 2009; 6:21.. Therefore, especially in the Brazilian context, it is important to highlight the role of the university in providing information that can guide decision-making in the organization and practice of services¹⁴14 Angulo-Tuesta A, Santos LMP, Iturri JA. Processos e desafios da interação entre pesquisa e política na perspectiva dos pesquisadores. Ciênc. Saúde Colet. 2018; 23(1):7-15..

The perspective that the interaction between research agents and health administrators may contribute to better implement policies, favoring their effectiveness and/or performance, is recurrent in the literature¹³13 Campbell DM, Redman S, Jorm L, et al. Increasing the use of evidence in health policy: practice and views of policy makers and researchers. Aust. New Zealand health policy. 2009; 6:21.

14 Angulo-Tuesta A, Santos LMP, Iturri JA. Processos e desafios da interação entre pesquisa e política na perspectiva dos pesquisadores. Ciênc. Saúde Colet. 2018; 23(1):7-15.^-1515 Lemay M, Sá C. The use of academic research in public health policy and practice. Res. Evaluat. 2014; 23(1):79- 88.. However, it should be taken into account that the establishment of active and effective communication between such actors is still an important challenge.

In this sense, the RMC is thought as a strategy that can contribute to reducing the distance between researchers and those who live the daily work and health care in the services studied. One of the Committee’s main strengths is to promote a systematic meeting between the team linked to the research institution and the managers and workers of the municipality. In these meetings, the joint evaluation of the activities, with presentation and discussion of the results raised so far, favors the appropriation of data immediately and intelligibly to those who will use them to produce effective changes in the work organization.

Following, some of the contributions of the RMC to the development of the research are presented more specifically, from the items that make up the outcome assessment construct about the implementation proposed by Proctor et al. (2011)⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76., namely: acceptability; adoption; adequacy; costs; feasibility; penetration; and sustainability.

Acceptability

Acceptability is the perception among interested parties of implementation – managers, workers, and users – that a particular treatment, service, practice, or innovation is acceptable, palatable, or satisfactory. Poor acceptability is a major barrier to the success of an intervention. Its variation, however, is constant, and occurs according to the experience of the subjects, throughout the implementation process⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

From the point of view of municipal management and services, the proposed implementation of network articulation devices showed, from the outset, strong indications of good acceptability. This aspect can be attributed to the fact that the proposal was based on a joint diagnosis, which effectively involved actors from both spheres.

However, it must be taken into account that, although a health intervention heavily relies on a management that supports the proposal, the success of implementation is not restricted to decision-making processes. Although the guarantee of an ethical-political framework is essential to guide the production of health, what effectively changes the indicators is the practical engagement of those who develop the day-to-day work of services.

It should be considered that, even if there is a managerial option for change, it will only be possible through its acceptability and adoption among the care professionals, who are, in fact, those who execute the proposal. The following statement illustrates a situation that warns of the institutionalization of practices as a barrier to overcome and something that may undermine acceptability:

However, the action of the RMC as a proposal diffuser in the network, since its planning, has been shown as a strategy to promote the acceptability of the proposal among the target population. Because they are understood as representatives to the intervention team, RMC members act as important field preparers in their home services, and this can significantly contribute to the acceptability of the proposal, as illustrated in the following statement:

It is noteworthy that, while identifying and reporting the existence of points of low acceptability, the RMC points to strategies that can assist in the promotion of this item, such as, for example, by investing in individuals who exert influence in the environments where they are inserted.

The above speech illustrates very well how the RMC meets the suggestions of Young, Mendizabal (2009)¹⁶16 Young J, Mendizabal E. Helping researchers become policy entrepreneurs: how to developed engagement strategies for evidence-bases policy. London: ODI Briefing Paper; 2009., who, among other things, synthesize that it is necessary to identify the main actors within the research context, as well as being able to synthesize compelling and simple stories, from the research results, to work effectively with all interested parties.

Adoption

Adoption is defined as the intention, initial decision or action to employ an evidence-based innovation or practice. This item, which is measured from the perspective of workers and managers, can also be termed as ‘absorption’⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

It is noteworthy that, as in the case of acceptability, because the construction of the proposal was based on a joint diagnosis, the adoption of the proposal was always considered as very good. It is noteworthy that, from the central management point of view, there was an initiative of the municipal health secretary, to seek the research team to articulate a partnership in the intervention.

Moreover, it is noteworthy that, although in studies conducted in Australia, in the state of New South Wales¹³13 Campbell DM, Redman S, Jorm L, et al. Increasing the use of evidence in health policy: practice and views of policy makers and researchers. Aust. New Zealand health policy. 2009; 6:21., most health managers had shown interest in contacting researchers to inform policy formulation and management processes in Brazil, this is still a very rare move. Thus, it is necessary to recognize that the research that originated this study is in a favorable context, but uncommon in the Country.

Regarding services, through the RMC, it was possible, furthermore, to identify an articulation for the adoption that has been spreading through the points of the network, making the adoption process not only in the sense of management (workers), but from a cross-sectional point of view, with greater involvement of professionals for the adoption of the intervention. The speech below illustrates this phenomenon:

From the adoption point of view, the meeting of the RMC to evaluate the activities was also an important moment for the group to establish a position of engagement in the proposal, demonstrating, therefore, the tenacity of this item in the course of the intervention.

Adequacy

Adequacy refers to the perception of the interested parties that an innovation or evidence-based practice is appropriate to address a specific issue or problem. In some literatures, this item is treated as a component of acceptability, however, Proctor et al. (2011)⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76. advocate maintaining the distinction, since, such intervention can be perceived as appropriate, but not acceptable in a given context, and vice versa.

Measuring this item is especially useful for recognizing barriers to implementation with workers. Its measurement makes it possible to identify whether there is alignment of the proposal with what is understood by the workers as the institution’s mission, or even with their skills, functions and work expectations⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

One of the situations mediated by the RMC that illustrates well its performance regarding this item occurred in the planning phase of the intervention. Initially, there was the perspective of the articulation of the network only from the Family Health Support Centers (Nasf), however, the RMC identified that the municipal network demanded a more transversal action, which contemplated the promotion of articulation actions with fronts both primary care and specialized services. The following statement illustrates a recap of this episode:

Another direction of the research evaluated based on the perceptions of the RMC was after the project was submitted to the evaluation of external experts, in order to refine the study methodology. There was an understanding, by the experts, that, for logistical reasons, it would be interesting to restrict intervention to primary care services that operate within the logic of the Family Health Strategy (FHS). However, the RMC pointed out that this change would be in the opposite direction of what the municipal network workers understood as ideal and expected. Considering the conflicts that a change in this direction could cause, especially regarding the decrease in acceptability, it was decided to use only the FHS services as places to monitor the impact of the intervention, but with the inclusion of all primary care services in this intervention.

This perspective concerns an essential feature of the involvement of the interested parties in the research process: the depth of participation. This is characterized by the degree of intervention of the agents in the process of formulation, survey and analysis of the procedures involved in the research¹⁷17 Cousins JB, Chouirnard JA. Participatory evaluation up close: an integration of research-based knowledge. Charlotte: Age Publishing; 2012..

Deepening the participation of the actors in the research processes, especially those in which they are the beneficiaries themselves, is to help the academy move increasingly from the prescriptive role to a place of joint construction and ecology of knowledge.

Costs

Costs relate to the budgetary impact of the implementation process. Its assessment should take into account both the costs of the intervention itself as well as the budget variance relative to the implementation strategy used, and the costs of where the intervention will be implemented. Measuring this item is particularly useful for conducting comparative studies on the cost of different interventions and implementation strategies⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

Although access to this item is achieved especially through very specific technical tools, such as budget and management reports, some nuances could be observed in the meetings of the RMC. Through the group, it was possible to identify a culture of centralization of investments in services of greater quantitative appeal, which would be questioned from the intervention.

Facing the above speech, it can be said that perhaps the RMC can also contribute to the evaluation of this item, as regards the characterization of the health budget, as regards the allocation priority. Although it is a subjective datum, the perception of actors on how the budget pact is given can help in understanding the political game that marks the implementation of health policies of the municipality.

Feasibility

This item concerns the extent to which an evidence-based innovation or practice can be successfully performed or used within a given scenario or organization. This is an item that can be worked through both the implementation process and retrospectively to explain the success or failure of an initiative⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

It is emphasized that, although there is a relationship between feasibility and adequacy, the two items are conceptually distinct. This is because an intervention may, for example, be appropriate for the configuration of a service – being compatible with the mission of the service –, but may not be feasible, due to the need for resources or training unavailable.

So far, the RMC has played the key role in guiding the methodological options of the intervention with regard to feasibility. Regarding the training offered, for example, the size and number of classes offered, as well as the determination of vacancies per service in each class, were aspects marked by the RMC’s weightings. In this way, the aim was to make training feasible, without there being detrimental missteps to the service work process.

Loyalty

The loyalty item is defined as the degree to which an intervention has been implemented as prescribed in its original protocol or planning. It is effectively the extent to which: (1) an intervention was adhered to between workers and users; (2) there was coverage/availability of the intervention on the intended scenario; and (3) the quality with which the intervention protocol was adhered to by workers and users⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

As this is a process commonly verified in more advanced stages of the research, no significant evidence of action of the RMC regarding this item has yet been found. However, the results obtained through the RMC for the other items give evidence that it will be a potent instrument, especially for the evaluation of adherence by workers and users of the intervention protocol.

Penetration

It is defined as the integration of a practice within a service configuration and its subsystems, and concerns one of the facets of intervention institutionalization. In literature, there are records of the measurement of this item, both from the perspective of how many users eligible for the intervention/use of innovation went through it/used it and from the perspective of the proportion of workers who used a given protocol, among those who were trained to follow it⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

It is noteworthy that, as discussed in the topic related to adoption, the municipal management initiative, of seeking research agents for the qualification of health management and practice can strongly favor the penetration of the planned intervention. Therefore, this movement marks a process of investment and prioritization of mental health policy in the municipality.

This way, as it is a much-awaited moment, the beginning of the research activities was configured as a catalyst for necessary changes in the context of the service network. Aspects that illustrate this perspective relate to evidence of the penetration of the intervention that could be observed and reported by group members, from the first articulations, in order to gather data from the health network.

Apart from aspects observed in the network, it is noteworthy that the RMC is also located as an actor of change and new ways of organizing services in this network. Aspects related to the penetration of the intervention could be observed, even in moments of self-assessment of the members, regarding their participation in the RMC, as illustrated by the following statements:

It is worth highlighting that, as illustrated in the statements above, the RMC has also functioned as a thermometer, in the sense of reflecting, in the experience of its members, the process of transformation that is already happening in the daily routine of services. This aspect also contributes to the understanding of the impact of actions developed so far.

Sustainability

It is defined as the extent to which a newly implemented innovation or practice is maintained or institutionalized within the ongoing and stable operations of a service or organization. It is noteworthy that sustainability results can be related, conceptually and empirically, to the penetration item, especially from the perspective that greater penetration can contribute to long-term sustainability⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

Within the organizational culture, sustainability occurs through the political and practical spheres, and is marked by three stages, which determine institutionalization: (1) transition (the transition from a single temporary funding event to permanent funding); (2) repetitive reinforcement cycle of the importance of the intervention, through its inclusion in procedures and organizational behaviors; and (3) niche saturation (the extent to which an intervention is integrated into all subsystems of an organization)⁵5 Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. policy ment. health. 2011; 38(2):65-76..

In the research in question, the RMC, besides a strategy of articulation and participation of the interested parties, was also adopted as a strategy that promotes the sustainability of the intervention. This interface was thought, especially, in the perspective of establishing a routine of reinforcement of the intervention, through the continuing education activities that will be promoted by the members of the RMC after the conclusion of the research, when it will become a Permanent Education Committee (PEC).

In this sense, it is noteworthy that the inclusion of professionals directly linked to the assistance of users in the composition of the RMC, as well as a potent reorientation of practices, may be an important sustainability strategy. Faced with the instability of the Brazilian sanitary political context, historically associated with administrative discontinuity and the dilemmas, practices and contradictions that arise in public administration with each change of government and change of leaders¹⁸18 Nogueira FA. Continuidade e Descontinuidade Administrativa em Governos Locais: Fatores que sustentam a ação pública ao longo dos anos [tese]. São Paulo: Fundação Getúlio Vargas; 2006. 139 p., the perspective that top professionals are appropriate regarding good care practices may favor the continuity of such proposals, even in the face of possible institutional disinterest.

It was understood, furthermore, that, after the experiences and training with the research team, the workers that made up the RMC would be able to sustain the group’s functioning autonomously, replicating the training activities with the network workers who were covered by the intervention. It is noteworthy that this perspective has been discussed with the municipal management since the planning of the project, culminating in the signing of a term that, in addition to the release of professionals, signed the commitment of management, to ensure budget means and availability for Committee maintenance after the field research team has left the field.

This process was important, both to motivate the engagement of RMC members and to strengthen them in stimulating teams. It is emphasized that the performance of interventions requires an important mobilization and reorganization of the teams, so that their discontinuity is often a source of frustration. This situation is illustrated in the following statement:

Both in the above speech and in other moments, it was possible to identify that the members of the RMC stand as articulators of the sustainability of the proposal, indicating that it is a tool that can contribute both to the access of this item and to its promotion.

Therefore, as stated in the excerpts above, it can be said that the RMC has been an important mechanism for assigning meaning to the workers’ implication so that, at the end of the research, the actions triggered remain permanent, favoring, this way, the articulated mental health care network.

Conclusions

Faced with the demand for implementation research, to consider the context and its constituent individuals as key elements to understand the success or failure of interventions, as well as the need to use strategies to evaluate the outcomes that are able to provide feedback, while encouraging learning and adaptation, the RMC has proved to be a useful device for this field of research.

Among the main contributions of the RMC were: the promotion of the participation of the interested parties in various aspects of the research, giving greater depth to the participation of these actors; the possibility, from the perspective of those who live the daily work and care, to evaluate and monitor aspects such as the meaning and feasibility of research for the field of study; and the ability to have access to implementation research outcomes on an ongoing and more meaningful basis for those who will benefit from it.

As for the outcomes of the Implementation Research worked on in this study, it should be noted that the RMC constituted, in addition to a tool of access to some of the items, one of its promotion strategies, especially with regard to acceptability, adoption, penetration and sustainability. In this sense, it is suggested to the researchers and agencies involved in this type of study to adopt this tool as a possibility to make the research a more dialogic and potentially transformative process.

It is worth mentioning as a limitation of this study the perspective that the evaluation process of the RMC activities took place at an early stage of the research, and through a single focus group. However, the memories retrieval of the group’s performance through the minutes of the meetings enriched the results, which were able to illustrate the contributions that this tool presented in the reported research, up to now.

References

Publication Dates

History