Abstract
The study aims to describe the validation process of the Continuing Health Education Policy (PEPS) logical model in the Psychosocial Care Centers (CAPS) in Barreiras, Bahia, Brazil. In two rounds, this evaluability study on specialists distributed in the health training four-square model, including management, education/research, service, and social control, was conducted through the Delphi consensus technique. The items were consensual when they reached more than 80% of agreement among the participants, confirmed by the analysis of the median and the interval between the first and the third quartiles. The results reflect the validated logical model content in five technical components: Planning, Political Articulation, Coordination, Technical Qualification, and Live Work in Care Production. The structure and processes required to carry out the EPS actions at the CAPS and the expected results in the short, medium, and long terms were described in the logical model. Validating the operational aspect of the intervention contributes to organizing and adopting operability and fertility criteria in a formative assessment that aims to strengthen the practices of an educational policy in mental health toward comprehensive care.
Key words:
Continuing education; Health policies; Validation studies; Mental health; Mental health services
Introduction
Health work is loaded with subjectivity in its relational essence and adds the use of technologies during the health-disease-care process11 Santos DS, Mishima SM, Merhy EE. Processo de trabalho na Estratégia de Saúde da Família: potencialidades da subjetividade do cuidado para reconfiguração do modelo de atenção. Cien Saude Colet 2018; 23(3):861-870.. It is not a simple, unidirectional, positivist, or superficial process22 Rézio LA, Conciani ME, Queiroz MA. O processo de facilitação de Educação Permanente em Saúde para formação em saúde mental na Atenção Primária a Saúde. Interface (Botucatu) 2020; 24:e200113. when looking at mental health care. It does not follow the path of symptoms-diagnosis-therapy-cure because the symptom circulates and is never eliminated33 Lancetti A, Amarante PDC. Saúde mental e saúde coletiva. In: Campos GWS, Minayo MCS, Akerman M, Drumond Júnior M, Carvalho YM, organizadores. Tratado de saúde coletiva. São Paulo: Hucitec; 2008. p. 615-634.. Its object of intervention requires transformations in the organization of services44 Brasil. Ministério da Saúde (MS). Portaria nº 3.088, de 23 de dezembro de 2011. Dispõe sobre a Rede de Atenção Psicossocial para pessoas com sofrimento ou transtorno mental e com necessidades decorrentes do uso de crack, álcool e outras drogas, no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 23 dez..
Regarding mental health care services, the Psychosocial Care Network (RAPS) mission is to articulate the different points of health care for people with distress, mental disorder, or drug use, as follows: Primary Health Care (PHC), in which the PHC Units (UBS), Extended Family Health Centers, Street offices and Recreational and Culture Centers are allocated; Specialized Care, with Psychosocial Care Centers (CAPS) in their modalities; Urgent and Emergency Care; Transient Home Care; Hospital Care; and Deinstitutionalization Strategies such as Home Therapy Services and the Volta para Casa (Back Home) Program44 Brasil. Ministério da Saúde (MS). Portaria nº 3.088, de 23 de dezembro de 2011. Dispõe sobre a Rede de Atenção Psicossocial para pessoas com sofrimento ou transtorno mental e com necessidades decorrentes do uso de crack, álcool e outras drogas, no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 23 dez..
The training of professionals to work in mental health services is in constant construction due to the challenges of the Anti-asylum Struggle, Deinstitutionalization, Psychosocial Care, and frequent attacks on the Brazilian Psychiatric Reform (RPB)55 Rosa MSG, Souza AC, Abrahão AL, Marques D. Inovações na formação em saúde: O programa de educação pelo trabalho - saúde mental. Rev Portuguesa Enferm Saude Mental 2016; 4:39-44.,66 Rézio LA, Fortuna CM, Borges FA. Pistas para educação permanente em saúde mental na atenção básica guiada pela socioclínica Institucional. Rev Latino AM Enferm 2019; 27:e3204.. Labor management and health education deserve much attention from managers and institutions to meet the needs of the population77 Machado MH, Ximenes Neto FRG. Gestão da educação e do trabalho em saúde no SUS: trinta anos de avanços e desafios. Cien Saude Colet 2018; 23(6):1971-1979.. As a policy, work management allows us to think strategically and ensure the primary requirements for valuing health professionals, and the qualification of health workers is an essential strategy for strengthening the Unified Health System (SUS), which far exceeds the search for diagnoses, care, treatment, prognosis, etiology and prophylaxis of diseases and illnesses88 Machado MH, Vieira ALS, Oliveira E. Gestão, trabalho e educação em saúde: perspectivas teórico-metodológicas. In: Baptista TWF, Azevedo CS, Machado CV, organizadores. Políticas, planejamento e gestão em saúde: abordagens e métodos de pesquisa. Rio de Janeiro: Fiocruz; 2015. p. 294-321..
It is crucial to develop conditions to meet the health needs of people, groups, and populations, sectoral management, and social health control99 Ceccim RB, Feuerwerker LCM. O quadrilátero da formação para área da saúde: ensino, gestão, atenção e controle social. Physis 2004; 14(1):41-65.. Thus, we can resize the development of people’s autonomy and the influence on the formulation of care policies. Continuing Health Education (EPS) is a tool for transforming practices in the daily life of health services1010 Ceccim RB. Educação permanente em saúde: desafio ambicioso e necessário. Interface (Botucatu) 2005; 9(16):161-177.. Its primary principle for training health professionals1111 Brasil. Ministério da Saúde (MS). Portaria GM/MS nº 1.996, de 20 de agosto de 2007. Diretrizes para a implementação da Política Nacional de Educação Permanente. Diário Oficial da União 2007; 21 ago. is the National Continuing Health Education Policy (PNEPS). In this sense, the PNEPS provides that evaluation processes can follow the EPS action design stages, the analysis of results, and the formulation of a value judgment about the scope of the formulated purposes1212 Brasil. Presidência da República. Avaliação de políticas públicas: guia prático de análise ex post, volume 2. Brasília: Casa Civil da Presidência da República; 2018..
An integrative review by Mattos et al.1313 Mattos MP, Campos HMN, Gomes DR, Ferreira L, Carvalho RB, Esposti CDD. Educação permanente em saúde nos Centros de Atenção Psicossocial: revisão integrativa de literatura. Saude Debate 2020; 44(127):1271-1293. points out the need for continuing education actions in the CAPS and the perpetuation of practices geared to the traditional and fragmented model of health actions, which hampers the inclusion of psychosocial practices in the setting of public health policies. In this context, the evaluation is a device that contributes to the renewal of health services and programs, where the EPS practices and developing groups with higher horizontality stand out. This strategy attaches the evaluation of the commitment to intervene in a particular reality1414 Furtado JP, Campos GWS, Oda WY, Onocko-campos T. Planejamento e Avaliação em Saúde: entre antagonismo e colaboração. Cad Saude Publica 2018; 34(7):e00087917..
Thus, evaluation processes acquire fundamental political functions by developing instruments that overcome traditional hospital models1515 Wetzel C, Kantorski LP. Avaliação de serviços em saúde mental no contexto da reforma psiquiátrica. Texto Contexto Enferm 2004; 13(4):593-598.. Few evaluation initiatives that include mental health services1616 Costa PHA, Colugnati FAB, Ronzani TM. Avaliação de serviços em saúde mental no Brasil. revisão sistemática da literatura. Cien Saude Colet 2015; 20(10):3243-3253.,1717 Oliveira MAF, Cestari TY, Pereira MO, Pinho PH, Gonçalves RMDA, Claro HG. Processos de avaliação de serviços de saúde mental: uma revisão integrativa. Saude Debate 2014; 38(101):368-378. are available within the SUS management. Evaluability studies involving the training four-square model (management, education/research, service, and social control) and whose references are the attributes of CAPs to evaluate the EPS were not identified in the literature. The evaluability study can be used as a pre-assessment at some stage of developing a program and throughout its cycle1818 Trevisan MS, Walser TM. Evaluability assessment: improving evaluation quality and use. New York: SAGE Publications; 2015..
In the meantime, it enhances the likelihood of assessing which results are helpful for the program and the professionals involved. The stages of the evaluability plan include a) documentary analysis to clarify the program’s objectives and goals; b) interview with key informants; c) modeling the intervention; d) hosting the workshop to agree with the logical model with those interested; and e) formulating evaluative questions1919 Leviton LC, Khan LK, Rog D, Dawkins N, Cotton D. Evaluability assessment to improve public health polices, programs, and pratices. Annu Rev Public Health 2010; 31:213-233.. In this context, it is noteworthy that modeling the Continuing Health Education Policy (PEPS) developed in the CAPS in Brazilian settings facilitates understanding the initiative in its rationality and logical chain from the operationalization of intervention and the essential components of the policy and its surrounding context2020 Hartz Z. Modelizar as intervenções. In: Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. Avaliação: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 61-74..
Understanding the relevance of promoting staff training spaces that allow workers to advance the transformation of daily practices and institutionalize the assessment of care and work processes developed in the CAPS to produce a change in formation, we should recognize the need for new “toolboxes” for knowledge production2121 Feuerwerker LCM. Educação na saúde - educação por profissionais de saúde - um campo de saber e de práticas sociais em construção. Rev Bras Educ Med 2007; 31(1):3-4.. This action allows feedback to reverse or minimize the obstacles and enhance the care provided2222 Almeida PF, Escorel S. Da avaliação em saúde à avaliação em Saúde Mental: gênese, aproximações teóricas e questões atuais. Saude Debate 2001; 25(58):35-47.. Thus, this study aims to describe the validation of the PEPS logical model in the CAPS in Barreiras, Bahia, Brazil.
Methods
This evaluability study for validating the PEPS logical model in the CAPS in Barreiras, Bahia, Brazil. The CAPS surveyed were CAPS II and CAPS Alcohol and Drugs (CAPS AD). This course was supported by a mixed approach2323 Ferreira L, Ribeiro MS, Oliveira LZ, Szpilman ARM, Esposti CDD, Cruz MM. Validação do modelo lógico de implementação da Política de Educação Permanente em Saúde na Atenção Primária. Trab Educ Saude 2020; 18(2):e0026294., using quantitative and qualitative techniques. The research is one of the stages of a doctoral thesis that addresses EPS actions in the CAPS from a participatory perspective.
The logical model (LM) is a tool to systematize and communicate the causal relationships between available resources, activities performed, and expected intervention results2020 Hartz Z. Modelizar as intervenções. In: Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. Avaliação: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 61-74.. Expressed through a graphic design, it explains the sequence of events with a synthesis of the main components of the intervention2020 Hartz Z. Modelizar as intervenções. In: Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. Avaliação: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 61-74.. The LM does not pretend to be a universally applicable truth, especially regarding Brazilian municipalities’ social, economic, and cultural diversity. Therefore, it is essential to incorporate new information and viewpoints about convictions, experiences, and knowledge2424 Vitorino SAS, Cruz MM, Barros DC. Validação do modelo lógico teórico da vigilância alimentar e nutricional na atenção primária a saúde. Cad Saude Publica 2017; 33(12):e00014217. in the modeling to increase the internal validity of the model itself and strengthen the analytical replication of this case study and research designs that depart from theory to empiria2020 Hartz Z. Modelizar as intervenções. In: Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. Avaliação: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 61-74.,2525 Yin RK. Estudo de Caso: planejamento e métodos. Porto Alegre: Bookman; 2015..
The preparation of the LM considered the implementation of the PNEPS from 2007, a period of reformulation of the policy guidelines through the Ministry of Health Ordinance (MS) n° 1.996/20071111 Brasil. Ministério da Saúde (MS). Portaria GM/MS nº 1.996, de 20 de agosto de 2007. Diretrizes para a implementação da Política Nacional de Educação Permanente. Diário Oficial da União 2007; 21 ago.. Official documents referring to the Municipal and State Management process of the SUS, represented by the Municipal Health Plan, State Health Plan, the Annual Municipal Management Report, the Annual State Management Report, the Regionalization Master Plan of the State of Bahia and State Continuing Health Education Plan. We performed a bibliographic review1313 Mattos MP, Campos HMN, Gomes DR, Ferreira L, Carvalho RB, Esposti CDD. Educação permanente em saúde nos Centros de Atenção Psicossocial: revisão integrativa de literatura. Saude Debate 2020; 44(127):1271-1293., referring to the intervention and documentary analysis to understand and identify its objectives and activities in Ordinance MS N° 1.174/20052626 Brasil. Ministério da Saúde (MS). Portaria nº 1.174/GM, de 7 de julho de 2005. Destina incentivo financeiro emergencial para o Programa de Qualificação dos Centros de Atenção Psicossocial - CAPS e dá outras providências. Diário Oficial da União 2005; 8 jul., Ordinance MS n° 854/20122727 Brasil. Ministério da Saúde (MS). Portaria nº 854, de 22 agosto de 2012. Altera a Tabela de Procedimentos, Medicamentos, Órteses, Próteses e Materiais Especiais do Sistema Único de Saúde. Diário Oficial da União 2012; 23 ago., Technical Note MS n° 11/ 20192828 Brasil. Ministério da Saúde (MS). Nota Técnica nº 11/2019-CGMAD/DAPES/SAS/MS. Esclarecimentos sobre a mudança na Política Nacional de Saúde Mental e nas Diretrizes da Política Nacional sobre Drogas. Diário Oficial da União 2019; 4 fev., and PNEPS1111 Brasil. Ministério da Saúde (MS). Portaria GM/MS nº 1.996, de 20 de agosto de 2007. Diretrizes para a implementação da Política Nacional de Educação Permanente. Diário Oficial da União 2007; 21 ago. to build the LM. The intervention was described and delimited through a data extraction roadmap, and aspects of thematic content analysis were used for document analysis2929 Minayo MCS. Pesquisa social: teoria, método e criatividade. Petrópolis: Vozes; 2016. and the identification of the technical components of the intervention.
Sampling was by convenience and adopted having academic experience or professional practice with EPS or mental health in Barreiras or the State of Bahia as an inclusion criterion to achieve consensus. Twenty experts representing management, education/research, service, and social control were selected and invited to participate in the study, as follows: a municipal mental health coordinator; ten CAPS health professionals; five representatives of higher education institutions (IES) (teachers and researchers); a representative of the technical area of mental health of the Regional Health Center; a representative of the Care Management Directorate of the Health Secretariat of the State of Bahia (SESAB); and two patient representatives in the Municipal Health Council. In the first contact, we explained the study’s objective, the technique to be used, and the importance of participation. One of the twenty selected experts refused to participate in the survey.
Figure 1 presents the steps of the LM construction and validation process. In the first round, held in September 2020, (1) an introductory text on the study objectives, (2) the Informed Consent Form, (3) the described modeling of the intervention in the evaluation process, (4) the first version of LM of the PEPS in the CAPS in Barreiras (Figure 2), and (5) the instructions for completing the instrument and returning the material were made available.
The second instrument was sent in October 2020 only to the 19 participants who responded to the first instrument. The document consisted of a brief consolidation of the first instrument, with (1) the description of the LM incorporating the first round’s inputs and (2) the modified visual scheme, and (3) an instrument for analyzing the relevance of the items (Figure 1). Regarding the PEPS in the municipal CAPS, a part of the theory was presented through the LM concerning structure, process, and results to be analyzed and classified by experts regarding relevance to items, into “yes” or “no”, and regarding the relevance of the items, through Likert scale (Figure 1). The first research instrument was organized into nine blocks, 1 referring to resources, 2, 3, 4, 5, and 6 to activities, and 7, 8, and 9 to short-, medium- and long-term results.
The following components related to PEPS in the CAPS were defined and inserted in the LM: planning, political articulation, coordination, technical qualification, and live work in care production. The “Planning” component comprises documents and plans that must be prepared by the stakeholders and the institutions involved in the planning and implementation of the PEPS in the CAPS.
The “political articulation” component refers to the existence of structured stakeholders and institutions responsible for the EPS in the CAPS, which are articulated with each other in a collaborative network for developing the policy. The “coordination” component addresses the availability and use of resources for developing the PEPS in the CAPS. The “technical qualification” component comprises guidance and actions for developing health education. The “live work in care production” component comprises the alignment between the stakeholders involved to reorient conceptions about the health space, the production of meanings, and their interrelationships.
In the first round, the 19 completed and returned instruments were encoded, and the answers were entered in a Microsoft Excel spreadsheet. The data were then analyzed through descriptive statistics before being presented in the next round. There was consensus on the item that received at least 80% of the votes “yes” in the relevance aspect2323 Ferreira L, Ribeiro MS, Oliveira LZ, Szpilman ARM, Esposti CDD, Cruz MM. Validação do modelo lógico de implementação da Política de Educação Permanente em Saúde na Atenção Primária. Trab Educ Saude 2020; 18(2):e0026294.,2424 Vitorino SAS, Cruz MM, Barros DC. Validação do modelo lógico teórico da vigilância alimentar e nutricional na atenção primária a saúde. Cad Saude Publica 2017; 33(12):e00014217.. In this first round, the participants judged the relevance of each item and could suggest and freely discuss the subject if they disagreed with what was presented. The participants’ suggestions were submitted to a qualitative analysis using thematic content analysis aspects2929 Minayo MCS. Pesquisa social: teoria, método e criatividade. Petrópolis: Vozes; 2016.. The irrelevant items were reformulated or excluded, and the considerations of the participants were listed and categorized, resulting in the inclusion or reformulation of the items in the second instrument (Figure 1).
In the second round, participants were asked to analyze the visual scheme again and their description and respond to the relevance of the instrument items in the nine blocks. The relevance level was assigned to each item, based on the four-point Likert scale, to determine its relevance value: (0) not at all relevant; (1) slightly relevant; (2) relevant; and (3) very relevant. The data were entered into the Statistical Packages for the Social Sciences (SPSS) software, version 21. Then, the descriptive analysis was performed, calculating the sum, median, and interquartile range of items. There was consensus on the relevance of items whose final scores reached more than 80% of the maximum available value and had a high level of agreement, confirmed by the analysis of the median response and interquartile range2323 Ferreira L, Ribeiro MS, Oliveira LZ, Szpilman ARM, Esposti CDD, Cruz MM. Validação do modelo lógico de implementação da Política de Educação Permanente em Saúde na Atenção Primária. Trab Educ Saude 2020; 18(2):e0026294.,2424 Vitorino SAS, Cruz MM, Barros DC. Validação do modelo lógico teórico da vigilância alimentar e nutricional na atenção primária a saúde. Cad Saude Publica 2017; 33(12):e00014217..
A period of 30 days was established in the first and second rounds to return the instruments duly completed, and the participants answered on time and sent the signed ICF with the instruments. The final result of the validation process was sent to participants to inform them. The Research Ethics Committee of the Federal University of Western Bahia approved this project under Opinion n° 4.022.337, of May 12, 2020.
Results
Using the two-round Delphi consensus technique, this study achieved the consensus of the relevance analysis and relevance of each LM item by experts, organized in nine blocks on structure, process, and results related to the PEPS in the CAPS in Barreiras, Bahia, Brazil. Nineteen of the 20 experts initially selected participated in the first round and 18 in the second round, representing all groups: management; education/research; service; and social control.
Regarding the analyzed items, those related to PEPS resources in the CAPS obtained the following values regarding the total sum reached, the median and interquartile range, respectively: 54, 3, 0 (human resources); 51, 3, 0 (financial resources); 49, 3, 0 (material resources). Table 1 presents the items related to the processes, represented by the activities and their respective outputs, organized into five technical components: planning, political articulation, coordination, technical qualification, and live work in care production.
The marking in Table 1 corresponds to the item considered irrelevant by the participants, that is, the one that reached the sum of points ≤ 43.2 points, confirmed by the median < 3 and interquartile range ≥ 1, and this item was reformulated in the final LM since it was considered relevant concerning PEPS activities in the municipal CAPS. Table 2 shows the items related to the results and impacts of PEPS in the CAPS in the municipality of Barreiras, Bahia.
Applying the Delphi consensus technique with the experts modified the initial LM and allowed a better understanding of the essential components of the policy in the local context. In the first round, based on experts’ contributions regarding the items’ pertinence, the items related to activities, outputs, and results of all the components closest to local governance in conducting the PEPS in CAPS were included and reformulated.
In the second round, the experts considered all items related to the structure relevant. Of the items related to the technical component of Political Articulation concerning outputs, the item referring to instances of regional agreement such as the Education-Service Integration Committee (CIES) and the Education-Health Public Action Organizational Contract (COAPES) established in conducting education in the CAPS, were not considered relevant. All items referring to the results and impact of the five technical components were considered relevant by the experts.
Finally, Figure 3 presents the PEPS LM in the CAPS in Barreiras, Bahia, validated through the Delphi consensus technique, structured according to the triad structure, process, and result, according to five components (planning, political articulation, coordination, technical qualification, and live work in care production). At the end of the validation process, we noticed that the main changes made to the LM consisted of including items that reflect the structuring, conduction, and participation of the PEPS in the municipal CAPS and, consequently, its structuring at the state and regional levels.
Discussion
The adequate evaluation of the PEPS in the CAPS requires building its LM. Its design consists of the first step for planning the evaluation with the understanding of the intervention’s operationalization2020 Hartz Z. Modelizar as intervenções. In: Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. Avaliação: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 61-74.. In this context, this PEPS evaluability study sheds light on its structure, operationalization, and direction in achieving the intended results and impacts on the qualification of professionals in the CAPS. In the context of this research, the LM stands out as a potentially helpful instrument for organizing training in the CAPS as a public policy that allows building local and regional spaces with the capacity to increase the qualification of health teams and their intersectoral partners. However, the setting is scarcely explored, and studies have yet to influence its macropolitical context3030 Dantas CR, Oda AMGR. Cartografia das pesquisas avaliativas de serviços de saúde mental no Brasil (2004-2013). Physis 2014; 24(4):1127-1179.. The components of structure, process, and results that guided the LM elaboration will also drive the discussion of the results of this study.
We highlight the relevance of structural components for developing the PEPS in the CAPS since they provide fundamental requirements for its operation, such as physical, human, and material resources. Infrastructure must result from a multisectoral effort with health to develop or continue to multiply and apply the EPS3131 Miccas FL, Batista SHSS. Educação permanente em saúde: metassíntese. Rev Saude Publica 2014; 48(1):170-185. to successfully develop the policy, planning, material, managerial, and human and financial resources. In the context of the CAPS, ambiance arises within a broader theoretical orientation, which refers to the treatment given to the physical setting that is a social, professional, and interpersonal relationship space, which provides welcoming, resolute, and human care3232 Kantorski LP, Coimbra VCC, Silva ENF, Guedes AC, Cortes JM, Santos F. Avaliação qualitativa de ambiência num Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2059-2066..
Therefore, this validation process moves toward breaking the illness processes created by institutionalization3333 Ronchi JP, Avellar LZ. Ambiência do atendimento de crianças e adolescentes em um CAPSi. Psicol Rev 2015; 21(2):379-396.. Planning, political articulation, coordination, technical qualification, and live work in care production technical components were all considered relevant. In the technical component of planning, intersectoriality is a dimension valued in public policies. It contributes to articulating knowledge and experiences for planning and evaluating policies, programs, and projects to ensure cooperative results in complex situations3434 Inojosa RM. Intersetorialidade e a configuração de um novo paradigma organizacional. RAP 1998; 32(2):35-48.. In this path, the political articulation component was fundamental for developing the EPS Policy in the CAPS. It shows the diversity of social stakeholders, the challenges, and the need for negotiation and political agreement. The EPS logic is decentralized, bottom-up, multidisciplinary, and transdisciplinary since the PNEPS management is shared99 Ceccim RB, Feuerwerker LCM. O quadrilátero da formação para área da saúde: ensino, gestão, atenção e controle social. Physis 2004; 14(1):41-65..
The inter-institutional articulation breaks the rule of single and hierarchical verticality of organizational flows. In the meantime, with the Administrative Reform in Bahia in 2014, the Regional Health Directorates (DIRES) were extinguished, and the Regional Health Centers (NRS) were established. The latter became responsible for centralizing regional management3535 Santos AM, Santos FRF, Paiva JAC. Processo de regionalização na Bahia, Brasil: desafios para inovação organizacional. Rev Gerencia Politicas Salud 2020; 19. DOI: 10.11144/Jave riana.rgps19.prbb
https://doi.org/10.11144/Jave riana.rgps... . In Brazil, health regionalization is an organizational strategy supported by SUS guidelines and manifests the need to establish healthcare networks in a complex political process involving different stakeholders3636 Santos AM, Assis MMA. Processo de regionalização da saúde na Bahia: aspectos políticos-institucionais e modelagem dos territórios sanitários. G&DR 2017; 13(2):400-422.. In the context of maintenance and municipal leadership and, simultaneously, strengthening of interdependence relationships and shared health responsibility, in this study, the Regional Interagency Committee (CIR) is a potential forum for discussing and building inducing strategies to advance deinstitutionalization and non-institutionalization and the inexistence of mental health care gaps3737 Macedo JP, Abreu MM, Fontenele MG, Dimenstein M. A regionalização da saúde mental e os novos desafios da Reforma Psiquiátrica Brasileira. Saude Soc 2017; 26(1):155-170..
The coordination component’s activities encourage, monitor, and strengthen the qualification of health workers. PNEPS funding decentralization, a shared responsibility, stumbles upon obstacles, as the “public machine” operates well within health actions and “gets stuck” when they are education actions3838 Ferraz F, Backes UMS, Matinez FJM, Feuerwerker LCM, Lino MM. Gestão de recursos financeiros da educação permanente em saúde: desafio das comissões de integração ensino-serviço. Cien Saude Colet 2013; 18(6):1683-1693.. Thus, to recover the funding flow and strengthen the PNEPS through transfers of resources from the Ministry of Health, Ordinance n° 3.194 of November 28, 20173939 Brasil. Ministério da Saúde (MS). Portaria nº 3.194, de 28 de novembro de 2017. Dispõe sobre o Programa para o Fortalecimento das Práticas de Educação Permanente em Saúde no Sistema Único de Saúde - PRO EPS-SUS. Diário Oficial da União 1990; 29 nov., provides for the Program for Strengthening EPS Practices in the SUS. Thus, around R$ 70 million were transferred to states and municipalities to conduct educational activities, observing regional and local needs4040 Gonçalves CB, Pinto ICM, França T, Teixeira CF. A retomada do processo de implementação da Política Nacional de Educação Permanente em Saúde no Brasil. Saude Debate 2019; 43(Esp.):12-23..
In the technical qualification component, the EPS and interprofessional education (EIP) are chosen to overcome the asylum model, deconstruct individualized, fragmented, mechanized practices, and build mental health care. The CAPS need a new work and care type, challenging the teams, as most professionals need to be trained in the psychosocial care paradigm4141 Sousa FMS, Severo AKS, Félix-Silva AV, Amorim AKMA. Educação interprofissional e educação permanente em saúde como estratégia para a construção de cuidado integral na Rede de Atenção Psicossocial. Physis 2020; 30(1):e300111.. This situation requires training strategies such as EIP, which enable building learning through the interaction of subjects with their knowledge, feelings, attitudes, beliefs, and customs based on democratized relationships4242 Mattos MP, Gomes DR, Silva MM, Trindade SMC, Oliveira ERA, Carvalho RB. Prática interprofissional colaborativa em saúde coletiva à luz de processos educacionais inovadores. Rev Baiana Saude Publica 2019; 43(1):271-287.. Moreover, we highlight the fragility of academic training and the need to resume reflection on the EPS concept. Thus, total appropriation of its concept is relevant, proposing its alignment, which facilitates managing and planning EPS actions4343 Gonçalves CB, Pinto ICM, França T, Teixeira CF. A retomada do processo de implementação da Política Nacional de Educação Permanente em Saúde no Brasil. Saude Debate 2019; 43( Esp.):12-23..
The final technical component, live work in care production, highlights items that must be supported by a commitment that it is possible to make a difference, even in situations where, in principle, nothing moves. The team moves towards thinking about another way to act, mobilizing affections, creating deviations, and other unthinkable care possibilities4444 Merhy EE, Feuerwerker LCM, Cerqueira P. Da repetição à diferença: construindo sentidos com o outro no mundo do cuidado. In: Franco TB, Ramos VC, organizadores. Semiótica, afecção e cuidado em saúde. São Paulo: Hucitec; 2010; p. 60-75.. Negotiation occurs through mutual affectations in the territory of caring actions. This territory is influenced by live work in action4545 Merhy E. Saúde: cartografia do trabalho vivo. São Paulo: Hucitec; 2002., which takes on a leading role in the work process and takes on different shapes based on the user’s need and the worker’s offer in an intense movement that accompanies birth or death4646 Chagas MS, Abrahão AL. Produção do cuidado em saúde centrado no trabalho vivo: existência de vida no território da morte. Interface (Botucatu) 2017; 21(63):857-867..
Thus, good encounters generate happy passions and favor and expand the potency of action between the bodies whose relationship is one of composition. On the other hand, bad encounters generate sad passions in which the potency of one is opposed to the other4747 Almeida SA, Merhy EE. Micropolítica do trabalho vivo em saúde mental: composição por uma ética antimanicomial em ato. Psicol Politica 2020; 20(47):65-75.. Deleuze4848 Deleuze G. Spinoza e a filosofia prática. São Paulo: Escuta; 2002. explains that actions are active affections. Through them, we know the constitutive relationships between bodies; what is pleasing or displeasing, and, therefore, the actions would generate ideas adequate to our reality. Thus, the items of each technical component described in this LM contribute toward PNEPS being adjusted to local contexts2323 Ferreira L, Ribeiro MS, Oliveira LZ, Szpilman ARM, Esposti CDD, Cruz MM. Validação do modelo lógico de implementação da Política de Educação Permanente em Saúde na Atenção Primária. Trab Educ Saude 2020; 18(2):e0026294.. From this perspective, health work reveals that its reorganization must also include implementing educational and training processes. EPS is a valuable strategy for reorganizing network services and building participative management.
The intended objectives of the intervention were presented as outputs, results, and impacts, which refer to short-, medium- and long-term results, respectively. Regarding outputs, almost all items were considered relevant by experts, except for regional agreement bodies such as the regional CIES and COAPES in conducting the EPS in the CAPS. Some states carried out in several Brazilian states report deficiencies in physical space and logistical support to ensure the functioning of the CIES, besides the inexistent EPS sector in the organization chart of the State Health Secretariats and the lack of resources (personnel, per diem, transportation) for the EPS actions3838 Ferraz F, Backes UMS, Matinez FJM, Feuerwerker LCM, Lino MM. Gestão de recursos financeiros da educação permanente em saúde: desafio das comissões de integração ensino-serviço. Cien Saude Colet 2013; 18(6):1683-1693.,4040 Gonçalves CB, Pinto ICM, França T, Teixeira CF. A retomada do processo de implementação da Política Nacional de Educação Permanente em Saúde no Brasil. Saude Debate 2019; 43(Esp.):12-23.,4949 França T, Medeiros KR, Belisario SA, Garcia AC, Pinto ICM, Castro JL, Pierantoni CR. Política de Educação Permanente em Saúde no Brasil: a contribuição das Comissões Permanentes de Integração Ensino-Serviço. Cien Saude Colet 2017; 22(6):1817-1828.. However, the proposals in PNEPS1111 Brasil. Ministério da Saúde (MS). Portaria GM/MS nº 1.996, de 20 de agosto de 2007. Diretrizes para a implementação da Política Nacional de Educação Permanente. Diário Oficial da União 2007; 21 ago. aim to encourage regional policy management and inter-institutional participation through the CIES and reflect on the articulation between the several institutions that comprise the EPS99 Ceccim RB, Feuerwerker LCM. O quadrilátero da formação para área da saúde: ensino, gestão, atenção e controle social. Physis 2004; 14(1):41-65. four-square model.
Establishing the regional CIES as a member of the local COAPES management committee4040 Gonçalves CB, Pinto ICM, França T, Teixeira CF. A retomada do processo de implementação da Política Nacional de Educação Permanente em Saúde no Brasil. Saude Debate 2019; 43(Esp.):12-23. is recommended to strengthen the PNEPS. COAPES is a continuous articulation instrument that signals the intentionality of several stakeholders in building an educational model that values the education-service-community integration required for health undergraduate and residency courses5050 Brasil. Ministério da Saúde (MS). Política Nacional de Educação Permanente em Saúde: o que se tem produzido para o seu fortalecimento? Brasília: MS; 2018.. It proposes a flow to guide and formalize intersectoral and interinstitutional relationships. Its adoption shows several advantages for states, municipalities, IES, workers, and students, as it qualifies them for the SUS demands and collaborates for its improvement4040 Gonçalves CB, Pinto ICM, França T, Teixeira CF. A retomada do processo de implementação da Política Nacional de Educação Permanente em Saúde no Brasil. Saude Debate 2019; 43(Esp.):12-23.,5050 Brasil. Ministério da Saúde (MS). Política Nacional de Educação Permanente em Saúde: o que se tem produzido para o seu fortalecimento? Brasília: MS; 2018.. COAPES should transcend the bureaucratic limits and develop a permanent negotiation to defend the qualification of care and training in the SUS in articulation with the EPS premises5151 Brasil. Ministério da Educação (MEC). Ministério da Saúde (MS). Portaria Interministerial nº 1.127, de 4 de agosto de 2015. Institui as diretrizes para a celebração dos Contratos Organizativos de Ação Pública Ensino Saúde (COAPES), para o fortalecimento da integração entre ensino, serviços e comunidade no âmbito do Sistema Único de Saúde (SUS). Diário Oficial da União 2015; 5 ago..
All items referring to the results and impact of the EPS Policy on the CAPS in Barreiras, Bahia, were relevant. When recognizing the potential of education in transforming practices, a vital competency is developing pedagogical methodologies that allow dialogical relationships and broaden the perspective of changing practices and organizing services4242 Mattos MP, Gomes DR, Silva MM, Trindade SMC, Oliveira ERA, Carvalho RB. Prática interprofissional colaborativa em saúde coletiva à luz de processos educacionais inovadores. Rev Baiana Saude Publica 2019; 43(1):271-287.,5252 Mattos MP, Campos HMN, Queiroz BF, Santos EJ, Cunha RSB, Gomes DR. Tecendo redes de educação construtivista em deontologia farmacêutica: formação e dispositivos ativos na arte de ensinar. Interface (Botucatu) 2020; 24:e190567.. Constructivist training in SUS management was identified as a catalyst for this process5252 Mattos MP, Campos HMN, Queiroz BF, Santos EJ, Cunha RSB, Gomes DR. Tecendo redes de educação construtivista em deontologia farmacêutica: formação e dispositivos ativos na arte de ensinar. Interface (Botucatu) 2020; 24:e190567.,5353 Cunha M, Campos HMN, Mattos MP. A formação construtivista de educandos de Farmácia na Gestão do SUS: Abordagens e contribuições para ambientes virtuais de aprendizagem. Rev Baiana Saude Publica 2019; 43(1):288-300.. Despite this indication, we should reinforce that the EPS process for training in mental health to work in the CAPS must be concomitantly guided by the RPB principles. Thus, workers should rethink their care practices since they are not only human “resources” or inputs, and care production is not alien to the subjective dimension of work5454 Merhy EE, Franco TB. O reconhecimento de uma produção subjetiva do cuidado. In: Merhy EE, Franco TB, organizadores. Trabalho, produção de cuidado e subjetividade: textos reunidos. São Paulo: Hucitec; 2013. p. 230-249..
The items in the final LM presented in this study show the multiple and complex factors and relationships involving PEPS in the CAPS. Only two rounds were necessary for this study due to the high consensus among experts. The expected loss in the validation study in the first round was 30% to 50% and 20% to 30% in the second round2323 Ferreira L, Ribeiro MS, Oliveira LZ, Szpilman ARM, Esposti CDD, Cruz MM. Validação do modelo lógico de implementação da Política de Educação Permanente em Saúde na Atenção Primária. Trab Educ Saude 2020; 18(2):e0026294.,5555 Wright JTC, Giovinazzo RA. Delphi: uma ferramenta de apoio ao planejamento prospectivo. Cad Pesq Admin 2000; 1(12):54-65.. Thus, the validation of internal consistency was intact, as the loss was only 5% in both rounds.
Final considerations
This evaluability study presented the validation process of the LM referring to the EPS Policy in the CAPS in Barreiras, Bahia, Brazil. The LM design shed light on how the intervention was conducted, understood the PEPS as a valuable tool in mental health training, and agreed on the possible development of the intervention. The collective construction with experts distributed in the four-square model for health education through the Delphi consensus technique was essential. It is an important step in the involvement of potential stakeholders in the evaluation process to strengthen the management of policy education at the local level. Furthermore, the validated LM can be revised and applied in other contexts if they fulfill the necessary adaptations and influences related to the intervention.
Developing the PEPS in the CAPS is essential to avoid bureaucratization and inefficient clinical practices. Operability and fertility criteria were adopted in a formative evaluation to strengthen mental health practices toward comprehensive care. This research aims to guide future investigations in the planning and evaluating mental health training policies and qualify care in different RPB settings.
References
- 1Santos DS, Mishima SM, Merhy EE. Processo de trabalho na Estratégia de Saúde da Família: potencialidades da subjetividade do cuidado para reconfiguração do modelo de atenção. Cien Saude Colet 2018; 23(3):861-870.
- 2Rézio LA, Conciani ME, Queiroz MA. O processo de facilitação de Educação Permanente em Saúde para formação em saúde mental na Atenção Primária a Saúde. Interface (Botucatu) 2020; 24:e200113.
- 3Lancetti A, Amarante PDC. Saúde mental e saúde coletiva. In: Campos GWS, Minayo MCS, Akerman M, Drumond Júnior M, Carvalho YM, organizadores. Tratado de saúde coletiva. São Paulo: Hucitec; 2008. p. 615-634.
- 4Brasil. Ministério da Saúde (MS). Portaria nº 3.088, de 23 de dezembro de 2011. Dispõe sobre a Rede de Atenção Psicossocial para pessoas com sofrimento ou transtorno mental e com necessidades decorrentes do uso de crack, álcool e outras drogas, no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 23 dez.
- 5Rosa MSG, Souza AC, Abrahão AL, Marques D. Inovações na formação em saúde: O programa de educação pelo trabalho - saúde mental. Rev Portuguesa Enferm Saude Mental 2016; 4:39-44.
- 6Rézio LA, Fortuna CM, Borges FA. Pistas para educação permanente em saúde mental na atenção básica guiada pela socioclínica Institucional. Rev Latino AM Enferm 2019; 27:e3204.
- 7Machado MH, Ximenes Neto FRG. Gestão da educação e do trabalho em saúde no SUS: trinta anos de avanços e desafios. Cien Saude Colet 2018; 23(6):1971-1979.
- 8Machado MH, Vieira ALS, Oliveira E. Gestão, trabalho e educação em saúde: perspectivas teórico-metodológicas. In: Baptista TWF, Azevedo CS, Machado CV, organizadores. Políticas, planejamento e gestão em saúde: abordagens e métodos de pesquisa. Rio de Janeiro: Fiocruz; 2015. p. 294-321.
- 9Ceccim RB, Feuerwerker LCM. O quadrilátero da formação para área da saúde: ensino, gestão, atenção e controle social. Physis 2004; 14(1):41-65.
- 10Ceccim RB. Educação permanente em saúde: desafio ambicioso e necessário. Interface (Botucatu) 2005; 9(16):161-177.
- 11Brasil. Ministério da Saúde (MS). Portaria GM/MS nº 1.996, de 20 de agosto de 2007. Diretrizes para a implementação da Política Nacional de Educação Permanente. Diário Oficial da União 2007; 21 ago.
- 12Brasil. Presidência da República. Avaliação de políticas públicas: guia prático de análise ex post, volume 2. Brasília: Casa Civil da Presidência da República; 2018.
- 13Mattos MP, Campos HMN, Gomes DR, Ferreira L, Carvalho RB, Esposti CDD. Educação permanente em saúde nos Centros de Atenção Psicossocial: revisão integrativa de literatura. Saude Debate 2020; 44(127):1271-1293.
- 14Furtado JP, Campos GWS, Oda WY, Onocko-campos T. Planejamento e Avaliação em Saúde: entre antagonismo e colaboração. Cad Saude Publica 2018; 34(7):e00087917.
- 15Wetzel C, Kantorski LP. Avaliação de serviços em saúde mental no contexto da reforma psiquiátrica. Texto Contexto Enferm 2004; 13(4):593-598.
- 16Costa PHA, Colugnati FAB, Ronzani TM. Avaliação de serviços em saúde mental no Brasil. revisão sistemática da literatura. Cien Saude Colet 2015; 20(10):3243-3253.
- 17Oliveira MAF, Cestari TY, Pereira MO, Pinho PH, Gonçalves RMDA, Claro HG. Processos de avaliação de serviços de saúde mental: uma revisão integrativa. Saude Debate 2014; 38(101):368-378.
- 18Trevisan MS, Walser TM. Evaluability assessment: improving evaluation quality and use. New York: SAGE Publications; 2015.
- 19Leviton LC, Khan LK, Rog D, Dawkins N, Cotton D. Evaluability assessment to improve public health polices, programs, and pratices. Annu Rev Public Health 2010; 31:213-233.
- 20Hartz Z. Modelizar as intervenções. In: Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. Avaliação: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 61-74.
- 21Feuerwerker LCM. Educação na saúde - educação por profissionais de saúde - um campo de saber e de práticas sociais em construção. Rev Bras Educ Med 2007; 31(1):3-4.
- 22Almeida PF, Escorel S. Da avaliação em saúde à avaliação em Saúde Mental: gênese, aproximações teóricas e questões atuais. Saude Debate 2001; 25(58):35-47.
- 23Ferreira L, Ribeiro MS, Oliveira LZ, Szpilman ARM, Esposti CDD, Cruz MM. Validação do modelo lógico de implementação da Política de Educação Permanente em Saúde na Atenção Primária. Trab Educ Saude 2020; 18(2):e0026294.
- 24Vitorino SAS, Cruz MM, Barros DC. Validação do modelo lógico teórico da vigilância alimentar e nutricional na atenção primária a saúde. Cad Saude Publica 2017; 33(12):e00014217.
- 25Yin RK. Estudo de Caso: planejamento e métodos. Porto Alegre: Bookman; 2015.
- 26Brasil. Ministério da Saúde (MS). Portaria nº 1.174/GM, de 7 de julho de 2005. Destina incentivo financeiro emergencial para o Programa de Qualificação dos Centros de Atenção Psicossocial - CAPS e dá outras providências. Diário Oficial da União 2005; 8 jul.
- 27Brasil. Ministério da Saúde (MS). Portaria nº 854, de 22 agosto de 2012. Altera a Tabela de Procedimentos, Medicamentos, Órteses, Próteses e Materiais Especiais do Sistema Único de Saúde. Diário Oficial da União 2012; 23 ago.
- 28Brasil. Ministério da Saúde (MS). Nota Técnica nº 11/2019-CGMAD/DAPES/SAS/MS. Esclarecimentos sobre a mudança na Política Nacional de Saúde Mental e nas Diretrizes da Política Nacional sobre Drogas. Diário Oficial da União 2019; 4 fev.
- 29Minayo MCS. Pesquisa social: teoria, método e criatividade. Petrópolis: Vozes; 2016.
- 30Dantas CR, Oda AMGR. Cartografia das pesquisas avaliativas de serviços de saúde mental no Brasil (2004-2013). Physis 2014; 24(4):1127-1179.
- 31Miccas FL, Batista SHSS. Educação permanente em saúde: metassíntese. Rev Saude Publica 2014; 48(1):170-185.
- 32Kantorski LP, Coimbra VCC, Silva ENF, Guedes AC, Cortes JM, Santos F. Avaliação qualitativa de ambiência num Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2059-2066.
- 33Ronchi JP, Avellar LZ. Ambiência do atendimento de crianças e adolescentes em um CAPSi. Psicol Rev 2015; 21(2):379-396.
- 34Inojosa RM. Intersetorialidade e a configuração de um novo paradigma organizacional. RAP 1998; 32(2):35-48.
- 35Santos AM, Santos FRF, Paiva JAC. Processo de regionalização na Bahia, Brasil: desafios para inovação organizacional. Rev Gerencia Politicas Salud 2020; 19. DOI: 10.11144/Jave riana.rgps19.prbb
» https://doi.org/10.11144/Jave riana.rgps19.prbb - 36Santos AM, Assis MMA. Processo de regionalização da saúde na Bahia: aspectos políticos-institucionais e modelagem dos territórios sanitários. G&DR 2017; 13(2):400-422.
- 37Macedo JP, Abreu MM, Fontenele MG, Dimenstein M. A regionalização da saúde mental e os novos desafios da Reforma Psiquiátrica Brasileira. Saude Soc 2017; 26(1):155-170.
- 38Ferraz F, Backes UMS, Matinez FJM, Feuerwerker LCM, Lino MM. Gestão de recursos financeiros da educação permanente em saúde: desafio das comissões de integração ensino-serviço. Cien Saude Colet 2013; 18(6):1683-1693.
- 39Brasil. Ministério da Saúde (MS). Portaria nº 3.194, de 28 de novembro de 2017. Dispõe sobre o Programa para o Fortalecimento das Práticas de Educação Permanente em Saúde no Sistema Único de Saúde - PRO EPS-SUS. Diário Oficial da União 1990; 29 nov.
- 40Gonçalves CB, Pinto ICM, França T, Teixeira CF. A retomada do processo de implementação da Política Nacional de Educação Permanente em Saúde no Brasil. Saude Debate 2019; 43(Esp.):12-23.
- 41Sousa FMS, Severo AKS, Félix-Silva AV, Amorim AKMA. Educação interprofissional e educação permanente em saúde como estratégia para a construção de cuidado integral na Rede de Atenção Psicossocial. Physis 2020; 30(1):e300111.
- 42Mattos MP, Gomes DR, Silva MM, Trindade SMC, Oliveira ERA, Carvalho RB. Prática interprofissional colaborativa em saúde coletiva à luz de processos educacionais inovadores. Rev Baiana Saude Publica 2019; 43(1):271-287.
- 43Gonçalves CB, Pinto ICM, França T, Teixeira CF. A retomada do processo de implementação da Política Nacional de Educação Permanente em Saúde no Brasil. Saude Debate 2019; 43( Esp.):12-23.
- 44Merhy EE, Feuerwerker LCM, Cerqueira P. Da repetição à diferença: construindo sentidos com o outro no mundo do cuidado. In: Franco TB, Ramos VC, organizadores. Semiótica, afecção e cuidado em saúde. São Paulo: Hucitec; 2010; p. 60-75.
- 45Merhy E. Saúde: cartografia do trabalho vivo. São Paulo: Hucitec; 2002.
- 46Chagas MS, Abrahão AL. Produção do cuidado em saúde centrado no trabalho vivo: existência de vida no território da morte. Interface (Botucatu) 2017; 21(63):857-867.
- 47Almeida SA, Merhy EE. Micropolítica do trabalho vivo em saúde mental: composição por uma ética antimanicomial em ato. Psicol Politica 2020; 20(47):65-75.
- 48Deleuze G. Spinoza e a filosofia prática. São Paulo: Escuta; 2002.
- 49França T, Medeiros KR, Belisario SA, Garcia AC, Pinto ICM, Castro JL, Pierantoni CR. Política de Educação Permanente em Saúde no Brasil: a contribuição das Comissões Permanentes de Integração Ensino-Serviço. Cien Saude Colet 2017; 22(6):1817-1828.
- 50Brasil. Ministério da Saúde (MS). Política Nacional de Educação Permanente em Saúde: o que se tem produzido para o seu fortalecimento? Brasília: MS; 2018.
- 51Brasil. Ministério da Educação (MEC). Ministério da Saúde (MS). Portaria Interministerial nº 1.127, de 4 de agosto de 2015. Institui as diretrizes para a celebração dos Contratos Organizativos de Ação Pública Ensino Saúde (COAPES), para o fortalecimento da integração entre ensino, serviços e comunidade no âmbito do Sistema Único de Saúde (SUS). Diário Oficial da União 2015; 5 ago.
- 52Mattos MP, Campos HMN, Queiroz BF, Santos EJ, Cunha RSB, Gomes DR. Tecendo redes de educação construtivista em deontologia farmacêutica: formação e dispositivos ativos na arte de ensinar. Interface (Botucatu) 2020; 24:e190567.
- 53Cunha M, Campos HMN, Mattos MP. A formação construtivista de educandos de Farmácia na Gestão do SUS: Abordagens e contribuições para ambientes virtuais de aprendizagem. Rev Baiana Saude Publica 2019; 43(1):288-300.
- 54Merhy EE, Franco TB. O reconhecimento de uma produção subjetiva do cuidado. In: Merhy EE, Franco TB, organizadores. Trabalho, produção de cuidado e subjetividade: textos reunidos. São Paulo: Hucitec; 2013. p. 230-249.
- 55Wright JTC, Giovinazzo RA. Delphi: uma ferramenta de apoio ao planejamento prospectivo. Cad Pesq Admin 2000; 1(12):54-65.
Publication Dates
- Publication in this collection
31 July 2023 - Date of issue
Aug 2023
History
- Received
22 May 2022 - Accepted
17 Apr 2023 - Published
20 Apr 2023