ABSTRACT

Integrative and Complementary Practices in Health include different health care forms and have been present for more than decades in health services in the Federal District, where the implementation assessment of your policy is not completely defined. To assist in understanding reality, an Evaluability Study was carried out collaboratively, involving discussion with stakeholders, questionnaires and consultation of documents. The policy components were identified, organized and modeled. A simplified-logical-model and one of the operational-logical-models, validated by the participants, are presented in this article. For policy evaluation, the initial aspects of the modeled logical implementation chain were prioritized. It was identified as fundamental the promotion of the actors’ knowledge about the set of resources to the implementation of the Integrative Practices services and about the related intersectoral responsibilities. Formative, qualitative and collaborative assessments are suggested. The modeling carried out in the studies of this policy, shed light on the study of similar ones.

KEYWORDS
Health evaluation; Complementary therapies; Health policy; Public health; Health management

Introduction

Traditional and Complementary Medicine (TCM) originate from various parts of the world and have their use recommended by the World Health Organization (WHO)¹1 World Health Organization. WHO traditional medicine strategy. 2014-2023. Geneva: WHO; 2013. 76 p.. In Brazil, they are called Integrative and Complementary Health Practices (Pics), a term that refers to complex medical systems and other therapeutic resources that contribute to the global promotion of human care, including the expansion of individuals’ co-responsibility for health²2 Brasil. Ministério da Saúde. Política nacional de práticas integrativas e complementares no SUS - PNPIC - SUS. 1. ed. Brasília: Ministério da Saúde; 2006. 92 p.. Since 2001, the Federal District (DF) has defined Integrative Health Practices (PIS) as the set of technologies that address human health in its multidimensionality, promoting its role as protagonist, co-responsibility, emancipation, freedom and ethical attitude, to promote, maintain and recover one’s health³3 Distrito Federal. Política Distrital de Práticas Integrativas em Saúde: PDPIS. Brasília, DF: Fepecs; 2014. [acesso em 2022 nov 6]. Disponível em: https://www.fcm.unicamp.br/fcm/sites/default/files/2016/page/pdpic_distrito_federal.pdf.
https://www.fcm.unicamp.br/fcm/sites/def... .

The National Policy on Integrative and Complementary Practices (PNPIC) for the Unified Health System (SUS) in Brazil was published in 2006, guiding the implementation of these practices in the country and sharing responsibilities with state management, but without defining or standardizing its evaluation2. Partial national monitoring takes place through computerized systems⁴4 Losso LN, Freitas SFT. Avaliação do grau da implantação das práticas integrativas e complementares na Atenção Básica em Santa Catarina, Brasil. Saúde debate. 2017; 41(esp3):171-87., but municipalities are responsible for local monitoring of implementation and results⁵5 Brasil. Ministério da Saúde. Manual de Implantação de Serviços de Práticas Integrativas e Complementares no SUS. Brasília, DF: Ministério da Saúde; 2018. [acesso em 2019 dez 3]. Disponível em: http://189.28.128.100/dab/docs/portaldab/publicacoes/manual_implantacao_servicos_pics.pdf.
http://189.28.128.100/dab/docs/portaldab... .

In 2015, 527,953 individual consultations were registered in the Brazilian Primary Health Care information system in Pics, in 1,362 municipalities⁶6 Benevides IA. Medicina e terapias antroposóficas em 12 anos da Política Nacional de Práticas Integrativas e Complementares: Conceitos, normativas e resultados. Art. Méd. Ampl. 2018; 38(1):30-4.. However, the growth and expansion of Pics in the country does not mean institutionalization and sustainability of the PNPIC, without which there is fragility and instability of supply and difficulty in monitoring and evaluation to contribute to safety and quality⁷7 Barbosa FES, Guimarães MBL, Santos CR, et al. Oferta de Práticas Integrativas e Complementares em Saúde na Estratégia Saúde da Família no Brasil. Cad. Saúde Pública. 2020; 36(1):e00208818..

The multifactorial context of mismatches in the implementation of policies for integrative and complementary practices in health⁷7 Barbosa FES, Guimarães MBL, Santos CR, et al. Oferta de Práticas Integrativas e Complementares em Saúde na Estratégia Saúde da Família no Brasil. Cad. Saúde Pública. 2020; 36(1):e00208818.,⁸8 Contatore OA, Barros NF, Durval MR, et al. Uso, cuidado e política das práticas integrativas e complementares na Atenção Primária à Saúde. Ciênc. saúde coletiva. 2015; 20(10):3263-73.,⁹9 Paiva LF. Práticas Integrativas e Complementares Exercidas pelos Profissionais de Saúde no Sistema Único de Saúde: Revisão Sistemática sem Metanálise. [dissertação]. Florianópolis: UFSC; 2016. 177 p.,¹⁰10 Tesser CD, Sousa IMC, Nascimento MC. Práticas Integrativas e Complementares na Atenção Primária à Saúde brasileira. Saúde debate. 2018; 42(esp1):174-88.,¹¹11 Barros NF, Spadacio C, Costa MV. Trabalho interprofissional e as Práticas Integrativas e Complementares no contexto da Atenção Primária à Saúde: potenciais e desafios. Saúde debate. 2018; 42(esp1):163-73.,¹²12 Ribeiro AA. Dilemas na construção e efetivação da política nacional e distrital de práticas integrativas e complementares em saúde. [dissertação]. Brasília, DF: Universidade de Brasília; 2015. [acesso em 2019 out 24]. Disponível em: http://repositorio.unb.br/handle/10482/18141.
http://repositorio.unb.br/handle/10482/1... , added to the lack of definition of ideal mechanisms for evaluating these practices⁴4 Losso LN, Freitas SFT. Avaliação do grau da implantação das práticas integrativas e complementares na Atenção Básica em Santa Catarina, Brasil. Saúde debate. 2017; 41(esp3):171-87.,¹³13 Sousa IMC, Hortale VA, Bodstein RCA. Medicina Tradicional Complementar e Integrativa: desafios para construir um modelo de avaliação do cuidado. Ciênc. saúde coletiva. 2018; 23(10):3403-12.,¹⁴14 Sousa IMC, Aquino CMF, Bezerra AFB. Custo-efetividade em práticas integrativas e complementares: diferentes paradigmas. J Manag Prim Health Care. 2018; 8(2):343-50.,¹⁵15 Ruela LO, Moura CC, Gradim CVC, et al. Implementação, acesso e uso das práticas integrativas e complementares no Sistema Único de Saúde: revisão da literatura. Ciênc. saúde coletiva. 2019; (24):4239-50.,¹⁶16 Habimorad PHL, Catarucci FM, Bruno VHT, et al. Potencialidades e fragilidades de implantação da Política Nacional de Práticas Integrativas e Complementares. Ciênc. saúde coletiva. 2020; 25(2):395-405., makes studies necessary to better understand their operating logic and seek viable strategies for such evaluations.

The Evaluability Study (EA) has the potential to assist planning, contributing to the feasibility of proposed actions in the field of public health, helping to develop realistic objectives and to provide quick and low-cost feedback on implementation. It does not intend to generate evidence on effectiveness, but to identify the logic of the program, in addition to mismatches in the intervention¹⁷17 Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33..

The EA is of particular value to managers who wish to promote organizational learning, which can serve as an instrument to transfer a program to new leadership, and to anyone planning an evaluation¹⁸18 Thurston WE, Ramaliu A. Evaluability Assessment of Survivors of Torture Program: Lessons Learned. Can J PROGRAM Eval. 2005; 20(2):1-25.. Also, by generating understanding of the mechanisms through which the intervention achieves its results, it may indicate necessary adjustments in objectives, activities or resources, increasing its potential to meet the needs of society¹⁷17 Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33.,¹⁹19 Wholey JS. Exploratory Evaluation. Em: Handbook of Practical Program Evaluation. 3.ed. San Fancisco: Jossey-Bass; 2010. [acesso em 2021 ago 24]. Disponível em: http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf.
http://www.orange.ngo/wp-content/uploads... .

With more than three decades of history since the beginning of the institutionalization of these practices in public health in its territory, the DF can be considered a pioneer in Brazil²⁰20 Lemos MPK, Luiza VL. Contribuições do estudo de avaliabilidade para a gestão de práticas integrativas. In: Gouveia GDA, organizadoras. Práticas Integrativas em Saúde: uma realidade na atenção primária, especializada e hospitalar. São Paulo: Paco e Littera; 2022. p. 119-39. (Série: Saúde em Dia. v. 4).. In the last quadrennium of management, 80% of the Primary Care Managements in the DF offered PIS²¹21 Distrito Federal (Brasil). Relatório Anual de Gestão 2019. Brasília, DF: Secretaria de Estado de Saúde; 2020. [acesso em 2021 ago 9]. Disponível em: https://info.saude.df.gov.br/wp-content/uploads/2021/06/RAG2019_FINAL.pdf.
https://info.saude.df.gov.br/wp-content/... . The District Policy for Integrative Health Practices (PDPIS) – as the object of this study – enabled the translation of a reality that can contribute to the states of the federation.

Due to its strong potential to contribute to the implementation of Pics policies, the EA presented in this article was carried out, with the objective of expressing the PDPIS operating logic and identifying aspects of its implementation for monitoring and evaluation. This publication follows the quality standard suggested by Baratieri et al.²²22 Baratieri T, Nicolotti CA, Natal S, et al. Aplicação do Estudo de Avaliabilidade na área da saúde: uma revisão integrativa. Saúde debate. 2019; 43(120):240-55..

Materials and methods

The PDPIS²³23 Lemos MPK. Avaliabilidade da política de práticas integrativas em saúde do Distrito Federal na perspectiva da implementação na atenção primária. [dissertação]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz; 2021. EA was carried out using a mixed research method, using document consultation, research meetings, a policy strategy spreadsheet and a questionnaire, following the steps described in box 1. Box 1 presents the research strategies and resources used in each evaluability stage, adapted from Wholey¹⁹19 Wholey JS. Exploratory Evaluation. Em: Handbook of Practical Program Evaluation. 3.ed. San Fancisco: Jossey-Bass; 2010. [acesso em 2021 ago 24]. Disponível em: http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf.
http://www.orange.ngo/wp-content/uploads... and Leviton et al.¹⁷17 Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33.. Quali-quantitative data were used for modeling, through frequency analysis and consensus techniques.

Carrying out evaluability with policy stakeholders was a dynamic process, with information being aggregated, validated and adjusted throughout its development¹⁷17 Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33.. EA interactions allowed for adjustments in the logical design and evaluation intentions, based on the repeated use of research resources and the review of information obtained at each stage¹⁷17 Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33.,¹⁹19 Wholey JS. Exploratory Evaluation. Em: Handbook of Practical Program Evaluation. 3.ed. San Fancisco: Jossey-Bass; 2010. [acesso em 2021 ago 24]. Disponível em: http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf.
http://www.orange.ngo/wp-content/uploads... . EA development took place between 2020 and 2021.

In the first stage of the research – focused on involving the interested parties – the context of the study was verified, consulting the available public documents. The DF has Brasília as its only municipality, with around three million inhabitants. The State Department of Health of the Federal District (SES-DF) has central and regionalized management, with no municipal health departments. Each of the seven Health Regions has a Superintendent as the highest authority and it is up to the regional management to act on the implementation of the PIS policy, as well as other health policies. The network has an exclusive PIS basic unit and two Live Pharmacies.

The Management of Integrative Health Practices (GERPIS), central management unit of SES-DF, was responsible for creating and updating the PDPIS. During the study, it was the unit responsible for promoting its implantation and implementation in the SUS-DF, including related educational processes.

The PDPIS, published in 2014 after public consultation, appreciation and approval by the SES-DF Management Collegiate and by the DF Health Council, included: Acupuncture; Art therapy; Self massage; Phytotherapy and Medicinal plants; Hatha Yoga; Homeopathy; Lian Gong in 18 therapies; Anthroposophical Medicine and Therapies; Meditation; music therapy; Reiki; Shantala; Tai Chi Chuan; and Integrative Community Therapy. In 2019, the following were also included in the DF health network: Laya Yoga, Stress Reduction Technique®, Ayurveda³3 Distrito Federal. Política Distrital de Práticas Integrativas em Saúde: PDPIS. Brasília, DF: Fepecs; 2014. [acesso em 2022 nov 6]. Disponível em: https://www.fcm.unicamp.br/fcm/sites/default/files/2016/page/pdpic_distrito_federal.pdf.
https://www.fcm.unicamp.br/fcm/sites/def... ,²⁴24 Brasil. Ministério da Mulher, da Família, e dos Direitos Humanos. Portaria nº 371, de 03 de junho de 2019. Inclui Ayurveda, Laya Yoga e Técnica de Redução de Estresse (TRE® - Tension and Trauma Releasing Exercises) à Política Distrital de Práticas Integrativas em Saúde. Diário Oficial da União. 4 Jul 2019.. In 2022, Auriculotherapy²⁵25 Distrito Federal (Brasil). Secretaria de Estado de Saúde do Distrito Federal. Portaria nº 351, de 06 de maio de 2022. Inclui a Auriculoterapia na Política Distrital de Práticas Integrativas em Saúde. Diário Oficial do Distrito Federal. 6 Maio 2022..

Each PIS had one or more public servants acting as District Technical Reference (RTD), linked to GERPIS, which contributed to the development of clinical management processes, with the qualification of PIS facilitators and with permanent education²⁶26 Distrito Federal (Brasil). Secretaria de Estado de Saúde do Distrito Federal. Portaria nº 1032, de 17 de setembro de 2018. Institui a Referência Técnica Distrital no âmbito da Subsecretaria de Atenção Integral à Saúde. Diário Oficial do Distrito Federal. 17 Set 2018.,²⁷27 Distrito Federal (Brasil). Secretaria de Estado de Saúde Regimento. Decreto nº 39.546, de 19 de dezembro de 2018. Aprova o regimento interno da Secretaria de Estado de Saúde do Distrito Federal. Diário Oficial do Distrito Federal. 19 Dez 2018.. The health units had the figure of the PIS Facilitator, a qualified professional who organizes, supervises, informs, executes and monitors the implementation of the PIS individually or in groups²⁸28 Distrito Federal (Brasil). Secretaria de Estado de São Paulo. Portaria nº 175, de 3 de setembro de 2009. Institucionaliza as Práticas Integrativas de Saúde (PIS). Diário Oficial do Distrito Federal. 3 Set 2009..

In a meeting, the study and concepts used were presented to the GERPIS team. The research questionnaire was applied and validated, three participating Health Regions were randomly selected, information was obtained for selection and contact with other actors involved, and the groups of study participants were intentionally defined, based on the main people involved in the implementation of this policy²⁹29 Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR. 1999 [acesso em 2020 fev 22]; 48(RR-11):1-35. Disponível em: https://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf.
https://www.cdc.gov/mmwr/PDF/RR/RR4811.p... .

Care professionals and central and regional managers of the SES-DF participated in the study, in addition to members of the Health Council of the DF. The selection criteria included professional work with PIS and Managements with the highest number of PIS implemented, prioritizing representativeness of the different PIS and seniority in the SES-DF (box 2).

User representatives on the DF Health Council were the only stakeholders external to SES-DF. Stakeholders exclusively linked to secondary or tertiary care were not consulted, as primary care has historically been emphasized in the implementation of PIS¹1 World Health Organization. WHO traditional medicine strategy. 2014-2023. Geneva: WHO; 2013. 76 p.,²2 Brasil. Ministério da Saúde. Política nacional de práticas integrativas e complementares no SUS - PNPIC - SUS. 1. ed. Brasília: Ministério da Saúde; 2006. 92 p.,³3 Distrito Federal. Política Distrital de Práticas Integrativas em Saúde: PDPIS. Brasília, DF: Fepecs; 2014. [acesso em 2022 nov 6]. Disponível em: https://www.fcm.unicamp.br/fcm/sites/default/files/2016/page/pdpic_distrito_federal.pdf.
https://www.fcm.unicamp.br/fcm/sites/def... .

For the second stage of the research – clarifying the intention of the policy and starting the modelling – the GERPIS group provided documents from the sector. In a meeting, the objectives and guidelines of the PDPIS were identified and aligned, following the views of the participants. The other stakeholders involved answered the survey questionnaire.

Official monitoring reports of the 2016-2019 District Health Plan and regulations, which are publicly accessible, were consulted. Also GERPIS internal documents, from 2020, containing its work processes³⁰30 Lemos MPK, Marques PFP. Sistematização dos processos de trabalho na gestão pública de práticas integrativas em saúde: relato de caso. Res Soc Dev. 2023; 12(1):e13312139692. and records of meetings for the preparation of a new text for the PDPIS.

In view of the third stage – exploring the reality of PDPIS execution –, in addition to the documents and questionnaires, GERPIS professionals expressed their views on each of the 72 PDPIS strategies. A spreadsheet was prepared for this purpose, containing: execution, feasibility, expected results, how to measure results, pertinence in the guideline to which they belong and in relation to the policy objectives, clarity, which operational axis they could be part of, among the proposed axes, activities developed and comments.

In the fourth stage of the research – checking plausibility and adjusting the design –, in light of Leviton¹⁷17 Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33., Thurston and Ramaliu¹⁸18 Thurston WE, Ramaliu A. Evaluability Assessment of Survivors of Torture Program: Lessons Learned. Can J PROGRAM Eval. 2005; 20(2):1-25., Wholey¹⁹19 Wholey JS. Exploratory Evaluation. Em: Handbook of Practical Program Evaluation. 3.ed. San Fancisco: Jossey-Bass; 2010. [acesso em 2021 ago 24]. Disponível em: http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf.
http://www.orange.ngo/wp-content/uploads... and Rossi, Lipsey and Freeman³¹31 Rossi PH, Lipsey MW, Freeman HE. Evaluation: systematic approach. 7. ed. Thousand Oaks: Sage; 2004. 470 p., the elements were rearranged and some original texts of the policy were modified, in order to elaborate the first complete version of the models intervention logic.

This process was based on the detailed material obtained. No information was discarded, so the most frequent ones served as the basis for modeling. The consultation of documents and regulations confirmed and facilitated the organization of contributions from the group of people involved.

The responses obtained in the previous steps were organized in an Excel spreadsheet for frequency analysis. The only open question in the questionnaire ratified information obtained. The proposed operational axes were used for classifying and aligning the strategies, as some generated doubts when reading or were too complex.

Some policy strategies integrated the model guidelines, others remained as strategies, and most were shifted as actions in each strategy. Five actions carried out that were not included in the PDPIS text were included, and twelve strategies were discarded by consensus on their non-implementation.

The Simplified Logical Model (MlogS) explained the relationship between the fundamental elements of the PDPIS, while the Operational Logical Models (MlogAt), one for each policy strategy, explained the execution of activities related to its implementation identified during the study (figures 1 and 2).

The second and third meetings with the GERPIS group allowed for a deeper discussion on the models under construction, which were analyzed and validated by the group, after presenting the results of the questionnaires. There was a consensus suggestion for adjustments to better describe the reality of policy implementation.

Modeling was improved to increase emphasis on three aspects of policy implementation considered by the GERPIS group to be more relevant than represented in the proposed design: educational activities, community participation, and sharing information about PIS.

The fifth and sixth stages of the study – exploring possibilities – were initiated in the last meetings. Priorities and feasibility of monitoring and evaluating the PDPIS were discussed, based on the most frequent responses in the survey questionnaire and the logical models validated by the GERPIS group, as well as on the challenges for implementation and monitoring. All the prepared material was made available so that the participants of this group could register other answers or ideas, closing the stages of the research.

Under registration CAAE 35488720. 3.0000.5240, this research project was approved by the Research Ethics Committees of the Sergio Arouca National School of Public Health and the Health Sciences Teaching and Research Foundation, in the Federal District.

Results

Of the total number of study participants, 38 answered the questionnaire, 25 (65.8%) of whom were female. The complete level of education of the majority was specialization (52.6%), followed by higher education (26.3%), master’s (13.2%) and high school (7.9%). Restricting the analysis only to those who participated in the group of PIS facilitators, 52.9% had a postgraduate degree and 11.8% had no higher education.

The 10 years tenure of 77.1% of the 35 SES participants’ and their diversified professional background help to understand the institutional reality. The three user representatives on the Health Council were participants outside the SES.

GERPIS was made up of administrative technicians, a manager, RTD holders and collaborators of the PIS modalities, a senior civil servant responsible for monitoring and a multiprofessional postgraduate resident. The lead author of this study was part of the team as RTD.

During the stakeholder consultation, there was a lack of clarity in some guidelines and strategies in the policy text, with repetition of ideas. The document referred to the context at the time of its preparation, requiring updates, and presented a structure with little connection between its components, with strategies that were not hierarchical, prioritized, or clearly related to objectives or results.

3 guidelines were outlined in the first modeling with the GERPIS group. Later, one on professional education in PIS was added. The final version of the proposed MlogS of the PDPIS (figure 1) presents four guidelines, which define the lines followed in the operationalization, each one with an objective and two strategies for the construction of action plans. This is the first known modeling of the PDPIS elaborated in 2014. In the dynamics of the model’s operation, the products of the strategies are also feeding and feedback resources of the model and the desired results can exist if there is joint achievement of objectives.

The text of the PDPIS generically predicted the budget guarantee as a strategy for its institutionalization. Funding had been carried out by various sources, but nonspecific and with little definition, which is why MlogS does not make it explicit in the first guideline, translating it as an absent resource in the logic of the policy ’s operation, despite the trend of change in this scenario, which will require updates of the logic-model.

The word strategy came to represent the main sets of actions aimed at certain results, implying a reduction of the 72 in the text to only 08 strategies in the model. While the policy follows its purpose as a major objective, the strategies follow the specific objectives in each guideline. The design of the current functioning of each strategy, with the actions developed, even if partially, irregularly or in an initial phase, and their expected effects was represented in eight MlogAt, exemplified in figure 2.

The text of the policy only mentioned GERPIS for PIS management, care, teaching and research, without making explicit the responsibilities of other stakeholders, which weakens intersectoral actions inside and outside the SES-DF. However, growing movements of decentralization of management focused on PIS, such as standardization, action of supporters at the three levels of care and expansion of a basic unit exclusively for PIS, strengthen its implementation and monitoring.

The partnerships for the management or implementation of the PDPIS were also explicit, when 76% of the respondents in the different groups indicated the existence of, on average, 3.5 partnerships each, in their scope of action, mainly with schools or universities, community groups and other health units.

As for PIS services, they follow the same institutional norms used in the management of other health policies implemented in the DF. However, it did not seem clear to the participants of this study how the PIS were organized in terms of decentralization of management and responsibilities, in order to guarantee intersectoral commitment and the different resources needed to implement the PDPIS.

Regarding monitoring, the District Health Plans (PDS) included PIS offer indicators and the 2019 Annual Management Report indicated 100% achievement of the objective regarding the strengthening of PDPIS actions in the last quadrennium²¹21 Distrito Federal (Brasil). Relatório Anual de Gestão 2019. Brasília, DF: Secretaria de Estado de Saúde; 2020. [acesso em 2021 ago 9]. Disponível em: https://info.saude.df.gov.br/wp-content/uploads/2021/06/RAG2019_FINAL.pdf.
https://info.saude.df.gov.br/wp-content/... . However, in the opinion of the GERPIS team, such indicators were limited to absolute numbers that do not translate or explore enough aspects of reality and the inadequacy of Pics codes in the SUS information systems leads to underreporting of services³²32 Losso LN, Freitas SFT. Avaliação do grau da implantação das práticas integrativas e complementares na Atenção Básica em Santa Catarina, Brasil. Saúde debate. 2017; 41(esp3):171-87..

Despite the limitations, the team highlighted that the indicators played a fundamental role in mapping SES-DF Network services and allowing the planning of new services. By integrating the PDS, they contributed to discussions on priorities in health management and to the expansion of PIS information, from senior management to the community.

Following it has favored the recording of other important data for monitoring, on the facilitators, number of assistance, intersectoral and educational actions, among others, thus existing a partial qualitative monitoring. Current monitoring is consistent with the proposed MlogS guidelines, despite not sufficiently supervising the expected effects in each of them, as explained by the GERPIS group.

In assessing the evaluability, the plausibility of the program is central to determine its potential to be evaluated, since it indicates whether it is feasible to achieve the proposed objectives with the activities foreseen in the continuous causal chain of the logical model¹⁷17 Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33.,³³33 Davies R. Planning Evaluability Assessment- synthesis of the literature with recommendations. 2013. [acesso em 2020 fev 22]. Disponível em: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf.
https://assets.publishing.service.gov.uk... ,³⁴34 Medina MG, Silva GAP, Aquino R. Uso de modelos teóricos na avaliação em Saúde: aspectos conceituais e operacionais. In: Hartz ZMA, Silva LMV. Avaliação em Saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Fiocruz; 2005. p. 41-63..

Stakeholders agreed, for the most part, on the main elements of the policy, which allowed modeling, linking them in a logical way. With the first four objectives of the PDPIS original text, 95% or more of the participants agreed totally or partially; with the fifth, 87%. No participant strongly disagreed. All declared themselves interested in implementing the PDPIS.

Another relevant data from the study was that 73.7% of the participants were at that time or had already been users of any of the PIS available by the SUS in the DF. This result corroborates the monitoring data, demonstrating implementation and access to PIS in the service network. The interest in implementing other PIS within its scope of action was registered by 73.6%, in a similar percentage for the different groups consulted, and the lack of interest may be linked, in the opinion of the GERPIS group, to the current implementation management limitations, such as operationalization of services, monitoring and technical follow-up.

Almost 80% of the participants stated that the community helps with the implementation, however 50% agreed totally or partially that there are professionals in their area or workplace who voluntarily make it difficult. Several participants worked in the same places, so this aspect needs to be further explored to be better understood.

The lack of support from managers was emphasized by participants in the questionnaires and meetings, with implications for the availability of resources and the functioning of services. Part of the participants reported never having been a PIS user or practitioner, which can be associated with a lack of knowledge, which is also a necessary resource for the implementation of the policy and can be a factor that hinders its implementation, increasing the importance of the policy.

The implementation of the PDPIS requires the availability of knowledge for the network, which, despite not being so explicit in the original text, not being uniform among the PIS modalities and not reaching all units, was part of the actions developed over the years, favoring the maintenance of activities with PIS in the network, which was confirmed by all data collection mechanisms.

Of the five activities added to the logical models in relation to the text of the PDPIS, all refer to the dissemination of information and knowledge about PIS. The model becomes plausible as these actions are included.

The response options provided for prioritization of evaluation by the participants were grouped into three groups: precursory aspects of the implementation of PIS services; aspects related to the functioning of these services; and aspects related to the expected effects with the implementation of the PDPIS (table 1).

The first set predominated in the general total of answers, with emphasis on the availability of resources and training or professional regulation for PIS. However, this did not have a uniform response among the consulted groups, as it was not prioritized by groups of central managers.

Only for the GERPIS group did the second set of aspects predominate in the answers, but the operationalization and improvement of PIS services was the third most prioritized option for evaluation by the participants in general, registered by all groups.

Apparently, the central management of PIS perceives itself to be in hold of issues that are precursors to the implementation of services that the rest of the stakeholders involved in this study are not. This reinforces the need for education on the use of PIS as care technologies. At the same time, monitoring does not seem to be enough to provide central management with sufficient information about the operation of network services, in order to improve management.

The knowledge and perception of the functioning of the PIS services, the mechanism of action and the rationale for these practices, seems to be a critical point in the implementation of the PDPIS. Without understanding and acting on this node, the implementation and execution of services becomes difficult, without which their purpose cannot be achieved.

It is noteworthy that the development of the study allowed expanding the vision of managers and others involved with the PIS on the policy and its implementation²⁰20 Lemos MPK, Luiza VL. Contribuições do estudo de avaliabilidade para a gestão de práticas integrativas. In: Gouveia GDA, organizadoras. Práticas Integrativas em Saúde: uma realidade na atenção primária, especializada e hospitalar. São Paulo: Paco e Littera; 2022. p. 119-39. (Série: Saúde em Dia. v. 4)., as well as its results fostering communication and knowledge for all who access them.

The evaluative questions, the last stage of the EA carried out, must translate the desire and need for understanding of the actors, in addition to being able to show directions that add value to the evaluated object, recognizing its results or illuminating its limitations³⁵35 Silva RR, Gasparini M, Alquezar E, et al. Avaliação para Negócios de Impacto Social - Guia prático. São Paulo: Artemisia; 2017. [acesso em 2021 ago 8]. Disponível em: https://move.social/wp-content/uploads/2017/11/2017_Guia-Pr%C3%A1tico_Avalia%C3%A7%C3%A3o-para-Neg%C3%B3cios-de-Impacto-Social_Artemisia_ABF_Move.pdf.
https://move.social/wp-content/uploads/2... . T w o central questions were identified:

• Do the actors involved know and exercise their responsibilities in providing the necessary resources for the implementation of PIS in light of the institutional norms of the SES-DF?

• Do public servants qualified to provide assistance with PIS and their managers feel that they have the intellectual resources needed to perform the services?

Discussion

The centralized responsibility in GERPIS and the little definition of financial resources are aspects identified for improving the implementation of the PDPIS, which can be favored with this study. However, the implementation has been maintained and expanded over time, with a network of actors internal and external to the SES-DF, including users and those who work in a technical and educational way.

The structure of the PDPIS text included almost all the elements described by the Ministry of Health36 as necessary to fulfill its purpose. This study enabled the recognition and modeling of its constituent elements, advancing in relation to the necessary logical organization for monitoring and evaluation.

The PDPIS guidelines identified in this study coincide approximately with the components described for the Recife Municipal Policy for Integrative Practices³⁷37 Cazarin G, Lima SFF, Benevides IA. Avaliabilidade da Política de Práticas Integrativas e Complementares do município de Recife-PE. J Manag Prim Health Care. 2017; 8(2):203-15.. They “express achievement ideals and guide strategic and priority choices”³⁸38 Brasil. Ministério da Saúde. Manual de Planejamento no SUS. Brasília, DF: Ministério da Saúde; 2016. 138 p. (Série: Articulação Interfederativa. v. 4).⁽¹⁰⁴⁾. They indicate lines of action to be followed or paths of action for the elaboration of plans, programs, projects and activities that will make the policy operational, serving as a foundation for the actions of managers³⁶36 Brasil. Ministério da Saúde. Sistema de planejamento do SUS: uma construção coletiva: formulação de políticas específicas de saúde. Brasília, DF: Ministério da Saúde; 2009. (Série: Textos Básicos de Saúde-Cadernos de Planejamento. v. 7). [acesso em 2021 set 28]. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/sistema_planejamento_sus_v7.pdf.
https://bvsms.saude.gov.br/bvs/publicaco... .

The guideline on guaranteeing access to homeopathic, anthroposophic, medicinal plants and herbal medicines, in the original text, despite the growth and appreciation of the two units of Live Pharmacy implemented, was not feasible, due to the unavailability of resources, technologies and structure, aspects mentioned by Mendes et al.³⁹39 Mendes LV, Campos MR, Chaves GC, et al. Disponibilidade de medicamentos nas unidades básicas de saúde e fatores relacionados: abordagem transversal. Saúde debate. 2014 [acesso em 2014 dez 30]; 38(esp):109-123. Disponível em: https://doi.org/10.5935/0103-1104.2014S009.
https://doi.org/10.5935/0103-1104.2014S0... .

To explore aspects of evaluation, monitoring or improvement of the policy, it is necessary to consider its innovative proposal in the current national context of health services and its implementation limitations, with clear weaknesses in monitoring. Even so, the offer and monitoring of these practices in the DF stand out, considering that the data available in Brazil on Pics is scarce and even the offer is still timid, despite the positive effects for users and services that use them¹⁵15 Ruela LO, Moura CC, Gradim CVC, et al. Implementação, acesso e uso das práticas integrativas e complementares no Sistema Único de Saúde: revisão da literatura. Ciênc. saúde coletiva. 2019; (24):4239-50..

The logical design constructed and presented here can support other planning instruments, which can be part of or derived from the text of the policy, favoring the existence of parameters, criteria and indicators for planning, monitoring and evaluating its execution, in addition to specifying quantitative and qualitative goals, that may or may not be included in official government plans.

Considering that a policy is a general planning instrument, eliminating the need to quantify objectives, targets and resources³⁶36 Brasil. Ministério da Saúde. Sistema de planejamento do SUS: uma construção coletiva: formulação de políticas específicas de saúde. Brasília, DF: Ministério da Saúde; 2009. (Série: Textos Básicos de Saúde-Cadernos de Planejamento. v. 7). [acesso em 2021 set 28]. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/sistema_planejamento_sus_v7.pdf.
https://bvsms.saude.gov.br/bvs/publicaco... , the use of more targeted and detailed management instruments than the policy itself would allow for easy readjustment and faithful representation of the implementation reality over the years, without the need to adjust the core of the policy, its purpose and its guidelines.

The set of modeled information allows a first look at the implementation reality, but requires periodic updates to represent reality. Likewise, the modeling of educational actions with managers and professionals may be able to assist in decision-making for future adjustments, with the potential to qualify the response of the health system⁴⁰40 Guerra S, Albuquerque AC, Marques P, et al. Construção participativa da modelização das ações educacionais da estratégia de Planificação da Atenção à Saúde: subsídios para avaliação da efetividade. Cad. Saúde Pública. 2022; 38(3):e00115021..

The modeled intervention is comprehensive and complex, provoking questions and reflections that require flexibility and adjustments to the particularities of each context involved⁴¹41 Cardoso GCP, Oliveira EA, Casanova AO, et al. Participação dos atores na avaliação do Projeto QualiSUS-Rede: reflexões sobre experiência de abordagem colaborativa. Saúde debate. 2019; 43(120):54-68.,⁴²42 Champagne F, Brouselle A, Hartz Z, et al. A avaliação no Campo da Saúde: conceitos e métodos. Em: Brouselle A, Champagne F, Contandriopoulos AP, et al., organizadores. Avaliação - conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 217-38.. In this sense, the generated models were able to logically represent the main components for the operationalization of the policy under study, but could not bring about many advances in relational aspects that interfere with its implementation.

The human aspects need to be explored, as the interprofessional relationships themselves can affect and be positively affected by the implementation of Pics in health units, for a complex of reasons, including the hierarchy of powers in the traditional training of professionals in contrast to the greater horizontality in the dissemination and use of knowledge of Pics¹¹11 Barros NF, Spadacio C, Costa MV. Trabalho interprofissional e as Práticas Integrativas e Complementares no contexto da Atenção Primária à Saúde: potenciais e desafios. Saúde debate. 2018; 42(esp1):163-73..

Despite the specific Pics policies in the country, the implementation often occurs independently by the interested professionals themselves¹⁰10 Tesser CD, Sousa IMC, Nascimento MC. Práticas Integrativas e Complementares na Atenção Primária à Saúde brasileira. Saúde debate. 2018; 42(esp1):174-88.,⁴³43 Aguiar J, Kanan LA, Masiero AV. Práticas Integrativas e Complementares na atenção básica em saúde: estudo bibliométrico da produção brasileira. Saúde debate. 2020; (43):1205-18., lacking resources for its realization. For this reason, studying Pics management can be a tendency to support the growth and quality of the offer⁴³43 Aguiar J, Kanan LA, Masiero AV. Práticas Integrativas e Complementares na atenção básica em saúde: estudo bibliométrico da produção brasileira. Saúde debate. 2020; (43):1205-18., since the practice implemented, financed and developed directly by professionals, without management support, would not characterize a State policy, generating instability in the offer⁷7 Barbosa FES, Guimarães MBL, Santos CR, et al. Oferta de Práticas Integrativas e Complementares em Saúde na Estratégia Saúde da Família no Brasil. Cad. Saúde Pública. 2020; 36(1):e00208818..

Even with a good level of education – 65.8% of participants reporting a specialization or master’s degree – specific training for Pics is one of the limitations to the expansion process. There is little national regulation, insertion of the subject in teaching is incipient and experiences in permanent education in Pics are local⁹9 Paiva LF. Práticas Integrativas e Complementares Exercidas pelos Profissionais de Saúde no Sistema Único de Saúde: Revisão Sistemática sem Metanálise. [dissertação]. Florianópolis: UFSC; 2016. 177 p.,¹⁰10 Tesser CD, Sousa IMC, Nascimento MC. Práticas Integrativas e Complementares na Atenção Primária à Saúde brasileira. Saúde debate. 2018; 42(esp1):174-88.. Access to inputs is also fundamental for the quality of services and the resolvability of health care¹²12 Ribeiro AA. Dilemas na construção e efetivação da política nacional e distrital de práticas integrativas e complementares em saúde. [dissertação]. Brasília, DF: Universidade de Brasília; 2015. [acesso em 2019 out 24]. Disponível em: http://repositorio.unb.br/handle/10482/18141.
http://repositorio.unb.br/handle/10482/1... ,¹⁴14 Sousa IMC, Aquino CMF, Bezerra AFB. Custo-efetividade em práticas integrativas e complementares: diferentes paradigmas. J Manag Prim Health Care. 2018; 8(2):343-50..

It is known that there is a political and technical lack of preparation of health professionals to work with Pics in the SUS, being necessary, for its consolidation in the network, educational processes that train professionals in line with the SUS guidelines and with the principles of Collective Health¹⁵15 Ruela LO, Moura CC, Gradim CVC, et al. Implementação, acesso e uso das práticas integrativas e complementares no Sistema Único de Saúde: revisão da literatura. Ciênc. saúde coletiva. 2019; (24):4239-50.,⁴⁴44 Azevedo E, Pelicioni MCF. Práticas integrativas e complementares de desafios para a educação. Trab Educ e Saúde. 2011; 9(3):361-78.. The training of professionals for Pics in the SUS should receive special attention, so that there is responsibility and understanding of its potentialities and limits, avoiding trivialization or misuse⁴³43 Aguiar J, Kanan LA, Masiero AV. Práticas Integrativas e Complementares na atenção básica em saúde: estudo bibliométrico da produção brasileira. Saúde debate. 2020; (43):1205-18..

This is not restricted to PIS. Other studies on health promotion identified the need to strengthen formal education and expand articulated intersectoral actions⁴⁵45 Prado NMBL, Medina MG, Aquino R. Intervenção intersetorial para promoção da saúde em sistemas locais: estudo de avaliabilidade. Saúde debate. 2017; (41):214-27., in addition to the desire to expand material and immaterial resources to consolidate processes⁴⁶46 Dias MSA, Oliveira IP, Silva LMS, et al. Política Nacional de Promoção da Saúde: um estudo de avaliabilidade em uma região de saúde no Brasil. Ciênc. saúde coletiva. 2018; 23(1):103-14..

Thus, with the purpose of answering questions and gaps in knowledge about the intervention, its operation and results, the importance of formative evaluations is highlighted, seeking to improve the program in the course of its execution and with the involvement of the actors⁴²42 Champagne F, Brouselle A, Hartz Z, et al. A avaliação no Campo da Saúde: conceitos e métodos. Em: Brouselle A, Champagne F, Contandriopoulos AP, et al., organizadores. Avaliação - conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 217-38.,⁴⁷47 Vieira-da-Silva LM. Avaliação de políticas e programas de saúde. Rio de Janeiro: Fiocruz; 2014.,⁴⁸48 Brasil. Casa Civil da Presidência da República; Instituto de Pesquisa Econômica Aplicada. Avaliação de Políticas Públicas: guia prático de análise ex ante. Brasília, DF: Ipea; 2018. v. 1. 192 p. [acesso em ago 25]. Disponível em: https://www.ipea.gov.br/portal/images/stories/PDFs/livros/livros/180319_avaliacao_de_politicas_publicas.pdf.
https://www.ipea.gov.br/portal/images/st... .

It is suggested to focus on the set of resources necessary for the operation of PIS services in the health network, so that managers and other actors also appropriate intellectual resources on PIS, obtaining knowledge about the functioning of these services in the network, in order to complement their health care management framework.

Final considerations

Carrying out the study made it possible to clarify and align the elements of the PDPIS in a coherent way, in the current context, with the consensus of the interested parties involved. This makes it possible to explore aspects to be evaluated, monitored or improved.

The participation of those interested in the construction and validation of the modeling carried out facilitates the understanding and execution of elementary aspects of the logical chain of implementation of the policy throughout the network, which can have repercussions on improving its operation in the short, medium and long term.

In addition, the generated logical models can be adjusted whenever there are changes in the scenario, as well as be used for planning, executing, monitoring and improving the implementation of the PDPIS, in a network, to achieve its purpose.

The modeling of the district policy makes it possible to advance towards studies of each PIS and its operationalization in the Health Regions of the DF. Other levels of health care and the context outside the SES – limitations of this study – could also be more easily explored.

It is suggested to encourage the set of resources needed to implement the PDPIS together with conventional health services, with emphasis on knowledge about PIS and related sectoral and intersectoral responsibilities.

The elaboration of action plans derived from the policy can be a good opportunity to agree on these responsibilities, favoring decentralized actions to meet local needs. It is hoped that this article can contribute to the development of PIS in the country.

Acknowledgements

To the State Department of Health of the Federal District and the Oswaldo Cruz Foundation (FIOCRUZ), that made the source research of this article possible, resulting from the Professional Master’s Degree in Public Health, from the National School of Public Health Sergio Arouca/FIOCRUZ.

To the team of the Management of Integrative Health Practices of the Federal District and to the other participants, who collaborated with the study and allowed a look at the reality of the implementation of the PDPIS.

To the Deputy Directorate for Teaching and Innovation at the Sergio Arouca National School of Public Health, who contributed to the publication of the article.

References

Publication Dates

History