Abstract

Assessment in the mental health area is a mechanism able to generate information that positively helps decision-making. Therefore, it is necessary to appropriate on the existing discussions, reasoning the challenges and possibilities linked to knowledge production within this scientific filed. A systematic review of publications about the Brazilian scientific production on mental health service assessment was performed, identifying and discussing methods, assessment perspectives and results. The search for articles was done in IBECS, Lilacs and Scielo databases, considering the publication of Federal Law 10.216. Thirty-five articles were selected based on the used terms and on the inclusion and exclusion criteria. Scientific production in this field is concentrated in the South and Southwest regions and holds different scopes and participants. Such wide range of possibilities is adopted as a way to help improving services and decision-making processes in mental health care. Advances in humanized, participative and community care are highlighted, but requiring more investments, professional qualification and organizational improvements. It is postulated greater integration among research, with evaluations going beyond structural aspects and the comparison with hospitalocentric models.

Key words
Mental health; Mental health services; Health evaluation; Health services evaluation; Review

Introduction

The current Brazilian mental health policies result from the mobilizations and claims of users, their family members and health professionals who aimed to achieve changes in the exclusion and imprisonment of people with mental disorders. Such demanding and popular participation processes were strengthened in the 1980s and originated the Brazilian Psychiatric Reform (RP). This reform grounded State policies such as Law n. 10.216 from April, 2001, representing great advance in the care given to mentally disordered people, heading towards community, aiming to guarantee the human rights and to enhance citizenship¹1. Brasil. Lei n.° 10.216, de 6 de abril de 2001. Dispõe sobre a proteção e os direitos das pessoas portadores de transtornos mentais e redireciona o modelo assistencial em saúde mental. Diário Oficial da União 2001; 9 abr.,²2. Hirdes A. A reforma psiquiátrica no Brasil: uma (re) visão. Cien Saude Colet 2009; 14(1):297-305..

The mental health assistance model is redirected, with the creation of substitutive services to the psychiatric hospital, i.e., a network of care devices to serve the population with mental disorders in an opened and communitarian approach and in their own territories. With the RP's extension process, the Psychosocial Care Network (RAPS) emerges, integrating the Brazilian Unified Health System (SUS) in 2011³3. Brasil. Portaria n° 3.088, de 23 de dezembro de 2011. Institui a Rede de Atenção Psicossocial para pessoas com sofrimento ou transtorno mental e com necessidades decorrentes do uso de crack, álcool e outras drogas, no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 26 dez.. RAPS is the integrated set of health care services to people with mental disorders and/or drug users. The network is composed of six care levels and their mechanisms, such as the Psychosocial Care Center (CAPS), that can be of three types (I, II and III) depending on their structure and working hours, specialized in people with general mental disorders, children and adolescents (CAPSi) or drug users (CAPSad); Basic Health Units (UBS); teams of the Family Health Strategy (ESF), street clinics; therapeutic residences, among others³3. Brasil. Portaria n° 3.088, de 23 de dezembro de 2011. Institui a Rede de Atenção Psicossocial para pessoas com sofrimento ou transtorno mental e com necessidades decorrentes do uso de crack, álcool e outras drogas, no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 26 dez.. As a dimension of RP's expansion in Brazil in the last few years, according to the SUS data, by the end of 2014 there were 2209 CAPS units of all types and specialties divided among 1413 Brazilian cities, whereas in 2008 the records showed 1326 CAPS units in 947 cities⁴4. Sala de Apoio à gestão Estratégica [acessado 2015 fev 19]. Disponível em: http://189.28.128.178/sage/.
http://189.28.128.178/sage/... .

Therefore, assessment and evaluation acquire a key function in the suppression of models based on psychiatric hospitals and for social participation over the theme⁵5. Almeida PF, Escorel S. Da avaliação em saúde à avaliação em Saúde Mental: gênese, aproximações teóricas e questões atuais. Saúde Debate 2001; 25(58):35-47.. Since the mental health assistance model was reformulated and expanded, the assessment processes got the political function of working as instruments to enhance the potential of practices applied to replace the hospitalocentric model⁶6. Wetzel C, Kantorski LP. Avaliação de serviços em saúde mental no contexto da reforma psiquiátrica. Texto Contexto Enferm. 2004; 13(4):593-598..

Although there are many concepts about health and mental health assessment, mental health is herein understood from a collective health perspective, as a reflexive process about an object, generating substantial information for its understanding and improvement⁷7. Contandriopoulos AP. Avaliando a institucionalização da avaliação. Cien Saude Colet 2006; 11(3):705-711.. The authors corroborate with Almeida and Escorel⁵5. Almeida PF, Escorel S. Da avaliação em saúde à avaliação em Saúde Mental: gênese, aproximações teóricas e questões atuais. Saúde Debate 2001; 25(58):35-47., who highlight that the mental health assessment must allow feedbacks to enhance the quality of the provided assistance and to reverse or minimize barriers.

Thus, mental health assessment emerges as a process able to influence decision making by generating information that leads to more accurate judgments⁵5. Almeida PF, Escorel S. Da avaliação em saúde à avaliação em Saúde Mental: gênese, aproximações teóricas e questões atuais. Saúde Debate 2001; 25(58):35-47.,⁷7. Contandriopoulos AP. Avaliando a institucionalização da avaliação. Cien Saude Colet 2006; 11(3):705-711.. Due to the complexity of the theme and the care reformulation, assessment strategies found in the recent model are necessary monitor its implementation and functionality⁸8. Lima FC, Schneider DR. Avaliação dos centros de atenção psicossocial: uma revisão integrativa da literatura nacional. Revista Caminhos 2013; 4(6):39-64.. The services, their logics and practices may be improved by deep reflections which are set in order to find ways of achieving effectiveness and efficiency as well as of giving users’ more quality of life.

However, as it was emphasized by Medina et al.⁹9. Medina MG, Silva GAP, Aquino R, Hartz ZMA. Uso de modelos teóricos na avaliação em saúde: aspectos conceituais e operacionais. In: Hartz ZAM, Silva LMV, organizadores. Avaliação em Saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Editora Fiocruz; 2005. p. 41-63., the evaluator visions/conceptions and his/ her insertions in the area guide the delimitation between object and objectives. To minimize these factors, the viewpoint from the different actors participating in these services (users, families, professionals etc.) must be taking into account aiming to guide the policies made in this field as well as the RAPS’ organization itself³3. Brasil. Portaria n° 3.088, de 23 de dezembro de 2011. Institui a Rede de Atenção Psicossocial para pessoas com sofrimento ou transtorno mental e com necessidades decorrentes do uso de crack, álcool e outras drogas, no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 26 dez..

On the other hand, considering the participation of different actors, the programs often present distinct logics, not always reaching consensus about the program's aims and outcomes. The difficulty in getting to the consensus, as well as the different positions and world visions, displays epistemological and methodological challenges that can impact the validity of the assessment⁹9. Medina MG, Silva GAP, Aquino R, Hartz ZMA. Uso de modelos teóricos na avaliação em saúde: aspectos conceituais e operacionais. In: Hartz ZAM, Silva LMV, organizadores. Avaliação em Saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Editora Fiocruz; 2005. p. 41-63..

Hence, it is necessary to appropriate about the discussions that permeates mental health assessment and to reason on the challenges and possibilities of knowledge construction within this field. Guided by this, the current systematic review aims to analyze the Brazilian mental health services’ assessment production, as well as to identify and discuss the assessment perspectives, within the services and their results. It takes as basis the RAPS, evaluating the care services that are compose it. As a cut period for the articles’ search, the Federal Law n. 10.216 from April 2001 was defined, for being a milestone that has changed the mental health assistance model in Brazil¹1. Brasil. Lei n.° 10.216, de 6 de abril de 2001. Dispõe sobre a proteção e os direitos das pessoas portadores de transtornos mentais e redireciona o modelo assistencial em saúde mental. Diário Oficial da União 2001; 9 abr..

Methodology

The search was performed in the Virtual Health Library databases: Lilacs, IBECS and SciELO. The search terms were defined according to the Health Sciences’ Descriptors (DECS), which is an indexation dictionary developed by Bireme. The search was done from January to February 2013.

In the search process, a boolean operator "and" was applied in the association of the following terms: Health Services Research; Health Services Evaluation; Health Evaluation; Program Evaluation; Health Care Quality, Access, and Evaluation; Evaluation Studies; Health Research Evaluation; Evaluation; with Community Mental Health Services; Mental Health; and Mental Health Services. Subsequently, the same operator was used, associating these terms translated into Portuguese, according to DECS.

One thousand two hundred thirty five (1235) articles related to the associated terms were found after the aforementioned descriptors were used. Next, their abstracts were read in order to select those of interest, according to the inclusion and exclusion criteria.

The inclusion criteria were: 1) empirical research assessing mental health services in Brazil; 2) research done after the federal law 10.2016 and until the search moment (February 2013); 3) assessments involving users, professionals, family members or other actors that participate in these services’ daily routines; and 4) studies on this field written in Portuguese, Spanish and English. The articles presenting at least one of the following features were excluded: 1) theoretical studies, experience reports, studies focused on evaluating psychometric properties of instruments, thesis, dissertations and/or researches about mental health services outside Brazil; 2) studies carried out before Federal Law 10.216; 3) assessments involving actors who are not part of these services’ dynamics; and 4) studies written in different languages rather than Portuguese, English and Spanish.

After reading the abstracts, applying the inclusion and the exclusion criteria and discarding the duplicated abstracts, only 65 articles remained. The remaining articles were then read and 35 were rejected, since they did not meet the aforementioned inclusion criteria. After advisory from specialists, two more articles were included. The references of the 32 remaining articles were read and three more articles were incorporated, resulting in 35 articles composing the final sample. The search process is shown in Figure 1.

The selected articles were tabulated and the following items were discerned: authors, year and journal of publication, theme description, assessment type, theoretical-methodological references, methods and results. Finally, it was done a descriptive analysis of the sample and a qualitative analysis of the study's results, together with a critical discussion of the material. The whole search procedure took place through peer review reaching a consensus; two researchers collected and read the articles, which were evaluated according to the inclusion and exclusion criteria. The EndNote web program was used as support tool and as a material organization mechanism.

Results

Bibliometric indicators

Studies meeting the herein set inclusion criteria were published in eight out of the eleven years that have followed the implementation of Federal Law 10.216. Articles related to the theme were missed only in the years of 2002, 2003 and 2005. An average of four articles were annually published (SD = 3.16). The years of 2009 and 2011 stood out with 9 and 10 articles published, respectively. Most of the articles were published in public and collective health journals; the others were published in nursing, psychiatry and psychology periodicals. Of the 35 sudies in the review, 8 (22.9%) resulted from the same multicentric research taken in CAPS of the South region. It is worth highlighting the participation of research groups from the Federal universities of Pelotas (UFPEL), São João del-Rei (UFSJ), Rio Grande do Sul (UFRGS) and the State University of Campinas (UNICAMP).

Regions and Services Studied

Almost all studies were conducted in the South and Southwest regions, with 20 of them assessing services in Southwest and 15 in South. Besides these, one article analyzed services in the North region and another one evaluated CAPS and Psychiatric Hospitals in national sphere, encompassing Northeast e Midwest regions also. Since this study involved all the Brazilian regions, it was counted as four studies, so the number of articles is counted as 38 and not the 35 mentioned above.

CAPSs were assessed in 24 studies (68.6%). UBSs were evaluated in five articles (14.3%). CAPSad, CAPSi, mental health outpatient facilities, psychiatric hospitals and therapeutic residences were the target in three assessments (8.6%) each. Two research projects assessed regional mental health reference centers in a city in the State of Minas Gerais (5.7%), one study evaluated an income generation program, and another one checked on psychiatric services in a general hospital. Eight studies (22.8%) have assessed more than one type of service making the sum of the frequency exceed 100%. Out of these eight, two studies directly approached the network concept; one of them assessed the CAPS network in the city of Campinas and the other evaluated the substitutive services network in São Paulo, with both studies before RAPS’ publication.

Methods, Instruments and Participants

Most of the studies had a qualitative approach (15 studies – 42.9%). The studies with quantitative approach represented 40.0% of the sample (14 studies). Six articles (17.1%) used a mixed approach, with quantitative and qualitative methods. Case studies and participative studies prevailed in the qualitative works. Transversal designs (descriptive and correlative) were common in the quantitative works. Nineteen assessments (54.3%) used more than one way to collect data. Interviews were the most applied collection strategies, in 17 assessments (48.6%). Questionnaires and scales were used in 16 assessments (45.7%). The Brazilian version of the Scale to Evaluate Patients’ Satisfaction with the Mental Health Services (SATIS – BR) was one of the main data collection instruments in 12 assessments (34.3%) and the Scale to Evaluate the Work Impact on the Mental Health Services (IMPACTO – BR) was used in 14.3% of the selected studies. The other data collection strategies were: medical records (three studies), focus groups (four studies) and field observation (seven).

Mental health service users were part of the target population in 22 assessments (62.9%); professionals working in these services were the target in 23 (65.7%); users’ family members composed the sample in 19 studies (54.3%); and managers of the services and of the mental health networks took part in three assessments (8.6%). Twenty-one studies (60.0%) evaluated more than one group of participants, thus getting repeated in the frequency counting. University's researchers (professors, graduation and post-graduation students) conducted the studies with some co-authorship from professionals of the services.

Theoretical and Methodological References

The most used theoretical-methodological reference was the Fourth Generation Evaluation by Guba and Lincoln¹⁰10. Guba EG, Lincoln YS. Fourth generation evaluation. Newbury Park: Sage Publications; 1989., grounding nine studies (25.7%). Gadamer's Hermeneutics was used in four studies (11.4%); the evaluation method by Donabedian, the summative research by Selltiz et al.¹¹11. Selltiz C, Wrightsman L, Cook S. Métodos de pesquisa nas relações sociais. São Paulo: EPU; 1987. and Contandriopoulos et al.¹²12. Contandriopoulos AP, Champagne F, Potvin L, Denis JL, Bouyle P. Saber preparar uma pesquisa. São Paulo: Hucitec Abrasco; 1994. were applied to three studies, each (6.2%). Other theoretical-methodological references such as the social representation theory, the collective subject discourse and production management theories have given bases to one study each. Thirteen quantitative evaluations (37.1%) did not explicit their theoretical references, which compromised their classification.

Assessment Type Classification

The typologies suggested by Novaes¹³13. Novaes HMD. Avaliação de programas, serviços e tecnologias em saúde. Rev Saude Publica 2000; 34(5):547559.and Donabedian¹⁴14. Donabedian A. The quality of care: How can it be assessed? JAMA 1988; 260(12):1743-48. were used to classify the assessments. Such references were applied since they are an effort to identify and systematize the focuses and core criteria in health evaluation processes. Thus, they can provide better understanding of the approaches and assessment functions in mental health.

Novaes¹³13. Novaes HMD. Avaliação de programas, serviços e tecnologias em saúde. Rev Saude Publica 2000; 34(5):547559. identifies three health assessment types: evaluation research, evaluation for management and evaluation for decision-making. As for the review studies, 19 studies were classified as evaluations for management (54.3%), and their main features were: the attempt to improve the service; natural context; presence of evaluators internal to the service; emphasis on the quantitative methods; and indicators able to be quantified and replicated¹³13. Novaes HMD. Avaliação de programas, serviços e tecnologias em saúde. Rev Saude Publica 2000; 34(5):547559.. The other 16 studies (45.7%) were categorized as evaluation for decision-making, and they were based on the following criteria: in-depth characterization and understanding of the service; aiming to impact the decision-making process; decisive position of the internal evaluator; prevailing use of qualitative methods; and apprehension in natural context¹³13. Novaes HMD. Avaliação de programas, serviços e tecnologias em saúde. Rev Saude Publica 2000; 34(5):547559.. While the first focus on formulating indicators and criteria propositions to the well functioning of services, the second aims to reach a more contextualized understanding in order to help its actors’ decision-making processes¹³13. Novaes HMD. Avaliação de programas, serviços e tecnologias em saúde. Rev Saude Publica 2000; 34(5):547559..

Donabedian¹⁴14. Donabedian A. The quality of care: How can it be assessed? JAMA 1988; 260(12):1743-48. suggests evaluating the quality of health services according to three categories: structure, process and outcomes. Twenty studies (57.1%) assessed aspects related to the services’ structure and attributes (material, human and organizational structure resources); 25 studies (71.4%) assessed the process category, which regards the actions taken towards giving and receiving care (users and professional's activities); and 29 (82.9%) studies encompassed the outcomes dimension, involving the care effects on health condition of users, family members and professionals. It is worth highlighting that 22 studies (62.9%) associated more than one assessment category, with 16 (45.7%) assessing structure, processes and outcomes.

Qualitative Analysis of the Articles’ Results

In order to allow substantial advances around the empirical work and given the insufficiency of merely percentage data to approach 35 articles, the results of the evaluations were qualitatively analyzed. The following analytical categories, detailed along this section of results, were previously raised and improved by reading the studies: effectiveness and efficacy of treatments; users, family members and professionals’ satisfaction level; resources (human, structural, financial etc.); working and care giving procedures; and management.

The assessments point towards advances in the care given to mental disorder patients based on the Brazilian RP. The recent policies in this field, the substitutive services and RAPS represent these progresses and guarantee access to a humanized, participative and communitarian treatment¹⁵15. Camatta MW, Nasi C, Adamoli AN, Kantorski L P, Schneider JF. Avaliação de um centro de atenção psicossocial: o olhar da família. Cien Saude Colet 2011; 16(11):4405-4414.,¹⁶16. Duarte MLC, Kantorski LP. Avaliação da atenção prestada aos familiares em um centro de atenção psicossocial. Rev. bras. enferm. 2011; 64(1):47-52.. Users and family members present high satisfaction levels with substitutive services, mainly CAPS, ESF and the therapeutic residences¹⁷17. Bandeira M, Silva MA, Camilo CA, Felício CM. Satisfação de familiares de pacientes psiquiátricos com os serviços de saúde mental e seus fatores associados. J. bras. psiquiatr. 2011; 60(4):284-293.–²³23. Silva MA, Bandeira M, Scalon JD, Quaglia MAC. Satisfação dos pacientes com os serviços de saúde mental: a percepção de mudanças como preditora. J. bras. psiquiatr. 2012; 61(2):64-71.. Such satisfaction appears to be linked to integral care, hospitality and humanizing attitudes, ruptures with social isolation, establishing bonds, improvement in clinical conditions, quality of life and assistance in dealing with mental disorders¹⁶16. Duarte MLC, Kantorski LP. Avaliação da atenção prestada aos familiares em um centro de atenção psicossocial. Rev. bras. enferm. 2011; 64(1):47-52.–¹⁹19. Coimbra VCC, Kantorski LP, Oliveira MM, Pereira DB, Nunes CK, Eslabão AD. Avaliação da satisfação dos usuários com o cuidado da saúde mental na Estratégia Saúde da Família. Rev. esc. enferm. USP 2011; 45(5):1150-1156.,²²22. Kantorski LP, Jardim VR, Wetzel C, Olschowsky A, Schneider JF, Heck RM, Bielemann VL, Schwartz E, Coimbra VCC, Quevedo ALA, Saraiva SS. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saude Publica 2009; 43(Supl. 1):29-35.,²⁴24. Costa CS, Bandeira M, Cavalcanti RLA, Scalon JD. A percepção de pacientes e familiares sobre os resultados do tratamento em serviços de saúde mental. Cad Saude Publica 2011; 27(5):995-1007.–²⁶26. Surjus LTLS, Campos RO. A avaliação dos usuários sobre os Centros de Atenção Psicossocial (CAPS) de Campinas, SP. Rev. latinoam. psicopatol. fundam. 2011; 14(1):122-133.. However, due to the comparison with the traditional hospitalocentric models, it is difficult to critically evaluate these recent strategies, in which the sense of treatment itself gets its meaning from the contact with the substitutive services¹⁷17. Bandeira M, Silva MA, Camilo CA, Felício CM. Satisfação de familiares de pacientes psiquiátricos com os serviços de saúde mental e seus fatores associados. J. bras. psiquiatr. 2011; 60(4):284-293.,²⁶26. Surjus LTLS, Campos RO. A avaliação dos usuários sobre os Centros de Atenção Psicossocial (CAPS) de Campinas, SP. Rev. latinoam. psicopatol. fundam. 2011; 14(1):122-133..

In a study conducted in CAPS III of Campinas, Campos et al.²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22. found high efficacy in the continence of users and their families at the time of crisis and psychosocial rehabilitation. Tomasi et al.²⁸28. Tomasi E, Facchini LA, Piccini RX, Thumé E, Silva RA, Gonçalves H, Silva SM. Efetividade dos centros de atenção psicossocial no cuidado a portadores de sofrimento psíquico em cidade de porte médio do Sul do Brasil: uma análise estratificada. Cad Saude Publica 2010; 26(4):807-815. found significant reduction in crisis occurrence, in medication use and in the number of psychiatric hospitalizations among CAPS’ long term and intensive care users. Among the non-intensive care users, medication use also diminished and their participation in therapeutic workshops and groups have increased. Users have reported improvement in humor, personal issues, interest for life, self-confidence, quality of sleep, emotional stability and the capacity to handle difficult situations in a study by Silva et al.²³23. Silva MA, Bandeira M, Scalon JD, Quaglia MAC. Satisfação dos pacientes com os serviços de saúde mental: a percepção de mudanças como preditora. J. bras. psiquiatr. 2012; 61(2):64-71.. Jaegger et al.²¹21. Jaegger RC, Guitton A P, Lyrio JM, Santos MM, Freitas RCO, Gonçalves SR, Lima LA, Legay LF. A experiência de morar fora: avaliação da satisfação de usuários em um serviço de saúde mental. Cad. Saúde Colet. 2004; 12(1):27-39.by assessing the satisfaction degree of therapeutic residences’ users found these devices’ importance to the communitarian care and insertion. Such studies raise indications about the effectiveness and efficacy of substitutive strategies.

Regarding the professionals’ satisfaction, the studies showed intermediate and high satisfaction levels with the services, with the main satisfaction being the factors associated to interdisciplinary teamwork¹⁸18. Camilo CA, Bandeira M, Leal RMAC, Scalon JD. Avaliação da satisfação e sobrecarga em um serviço de saúde mental. Cad. saúde colet. 2012; 20(1):82-92.,²⁵25. Kantorski LP, Coimbra VCC, Silva ENF, Guedes AC, Cortes JM, Santos F. Avaliação qualitativa de ambiência num Centro de Atenção Psicossocial Cien Saude Colet 2011; 16(4):2059-2066.,²⁹29. Rebouças D, Abelha L, Legay LF, Lovisi GM. O trabalho em saúde mental: um estudo de satisfação e impacto. Cad Saude Publica 2008; 24(3):624-632.–³³33. Santos AM, Cardoso DAJ, Vieira DPB, Araújo FC, Farias HS, Mota S P, Monteiro JCMS, Bahia SHA. Análise dos níveis de satisfação de trabalhadores de saúde mental de um Hospital público de referência psiquiátrica em Belém (PA). Rev. baiana saúde pública 2011; 35(4):813-825.. High satisfaction scores associated to work were observed in substitutive services, due to innovative and differentiated projects²⁵25. Kantorski LP, Coimbra VCC, Silva ENF, Guedes AC, Cortes JM, Santos F. Avaliação qualitativa de ambiência num Centro de Atenção Psicossocial Cien Saude Colet 2011; 16(4):2059-2066.,³¹31. Rebouças D, Legay LF, Abelha L. Satisfação com o trabalho e impacto causado nos profissionais de serviço de saúde mental. Rev Saude Publica2007; 41(2):244250.–³³33. Santos AM, Cardoso DAJ, Vieira DPB, Araújo FC, Farias HS, Mota S P, Monteiro JCMS, Bahia SHA. Análise dos níveis de satisfação de trabalhadores de saúde mental de um Hospital público de referência psiquiátrica em Belém (PA). Rev. baiana saúde pública 2011; 35(4):813-825..

Besides these positive outcomes, most of the studies point towards dissatisfaction and work overload because of large demand, lack of human and infrastructure resources, insufficient professional formation, among others. These factors limit the autonomy of action, hindering the organization of services and making professionals assume various responsibilities and feel unmotivated¹⁵15. Camatta MW, Nasi C, Adamoli AN, Kantorski L P, Schneider JF. Avaliação de um centro de atenção psicossocial: o olhar da família. Cien Saude Colet 2011; 16(11):4405-4414.,¹⁸18. Camilo CA, Bandeira M, Leal RMAC, Scalon JD. Avaliação da satisfação e sobrecarga em um serviço de saúde mental. Cad. saúde colet. 2012; 20(1):82-92.,¹⁹19. Coimbra VCC, Kantorski LP, Oliveira MM, Pereira DB, Nunes CK, Eslabão AD. Avaliação da satisfação dos usuários com o cuidado da saúde mental na Estratégia Saúde da Família. Rev. esc. enferm. USP 2011; 45(5):1150-1156.,²²22. Kantorski LP, Jardim VR, Wetzel C, Olschowsky A, Schneider JF, Heck RM, Bielemann VL, Schwartz E, Coimbra VCC, Quevedo ALA, Saraiva SS. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saude Publica 2009; 43(Supl. 1):29-35.,²³23. Silva MA, Bandeira M, Scalon JD, Quaglia MAC. Satisfação dos pacientes com os serviços de saúde mental: a percepção de mudanças como preditora. J. bras. psiquiatr. 2012; 61(2):64-71.,³¹31. Rebouças D, Legay LF, Abelha L. Satisfação com o trabalho e impacto causado nos profissionais de serviço de saúde mental. Rev Saude Publica2007; 41(2):244250.–⁴⁰40. Wetzel C, Kantorski LP, Olschowsky A, Schneider JF, Camatta MW. Dimensões do objeto de trabalho em um Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2133-2143..

Moreover, it is questioned the insufficient number of CAPS, especially the CAPSad and CAPSi⁴¹41. Zappitelli MC, Gonçalves EC, Mosca I. Panorama da saúde mental no Estado de São Paulo: leitos psiquiátricos e assistência extra-hospitalar. Rev. adm. saúde 2006; 8(31):71-78.. There is need for greater focus on issues related to mental health and/or global aspects of health such as health promotion, drug use, sexuality etc.²⁴24. Costa CS, Bandeira M, Cavalcanti RLA, Scalon JD. A percepção de pacientes e familiares sobre os resultados do tratamento em serviços de saúde mental. Cad Saude Publica 2011; 27(5):995-1007.,²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22.,⁴²42. Campos RTO, Campos GWS, Ferrer AL, Corrêa CAS, Madureira PR, Gama CAP, Dantas DV, Nascimento R. Avaliação de estratégias inovadoras na organização da Atenção Primária à Saúde. Rev Saude Publica 2012; 46(1):43-50.,⁴³43. Oliveira E, Oliveira MAF, Claro HG, Paglione HB. Práticas assistenciais no centro de atenção psicossocial de álcool, tabaco e outras drogas. Rev. ter. ocup. 2010; 21(3):247-254._. In a study that evaluated the approach to sexually transmitted diseases (STDs) in CAPS and psychiatric hospitals in Brazil, the CAPS presented better results than psychiatric hospitals in prevention and care of STDs and therapeutic resources, especially in social reintegration. Besides, there were few services with sex education programs and CAPS had greater scarcity of human and material resources than psychiatric hospitals³⁴34. Melo APS, Acúrcio FA, Cherchiglia ML, Veloso CCG, Guimarães MDC. Avaliação de serviços de saúde mental: assistência e prevenção às doenças sexualmente transmissíveis no contexto do Projeto PESSOAS. Rev. méd. Minas Gerais2007; 17(1/2, Supl.4):S240-S248..

With regard to work and care processes, the organization in an interdisciplinary team is placed as an indispensable factor for comprehensive, warm and distinctive care. The procedures and approaches are discussed and arranged for technical or reference teams, varying according to the needs and being prepared by means of singular therapeutic projects²⁶26. Surjus LTLS, Campos RO. A avaliação dos usuários sobre os Centros de Atenção Psicossocial (CAPS) de Campinas, SP. Rev. latinoam. psicopatol. fundam. 2011; 14(1):122-133.,²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22.,⁴³43. Oliveira E, Oliveira MAF, Claro HG, Paglione HB. Práticas assistenciais no centro de atenção psicossocial de álcool, tabaco e outras drogas. Rev. ter. ocup. 2010; 21(3):247-254.–⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100..

However, such interdisciplinary dynamics, in which the reference teams organize the work procedures, face some barriers as: the invisibility of some users who deal with activities and therapeutic projects that do not meet their needs⁴⁰40. Wetzel C, Kantorski LP, Olschowsky A, Schneider JF, Camatta MW. Dimensões do objeto de trabalho em um Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2133-2143.; limited view about the technicians in CAPS; strict role limitations and their influence in the work⁴⁶46. Pinho LB, Kantorski LP, Wetzel C, Schwartz E, Lange C, Zilmer JGV. Avaliação qualitativa do processo de trabalho em um centro de atenção psicossocial no Brasil. Rev. Panam. Salud Públ. 2011; 30(4):354-360.; work overload and excess of responsibility for the cases⁴³43. Oliveira E, Oliveira MAF, Claro HG, Paglione HB. Práticas assistenciais no centro de atenção psicossocial de álcool, tabaco e outras drogas. Rev. ter. ocup. 2010; 21(3):247-254.; isolated action of some professionals, especially psychiatrists⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100.; and lack of knowledge, by CAPS, about their own care models³⁷37. Olschowsky A, Glanzner CH, Mielke FB, Kantorski L P, Wetzel C. Avaliação de um Centro de Atenção Psicossocial: a realidade em Foz do Iguaçu. Rev. esc. enferm. USP 2009; 43(4):781-787..

In the same time some studies found many therapeutic modalities and projects that have extrapolated the institutions’ walls²⁶26. Surjus LTLS, Campos RO. A avaliação dos usuários sobre os Centros de Atenção Psicossocial (CAPS) de Campinas, SP. Rev. latinoam. psicopatol. fundam. 2011; 14(1):122-133.,³⁴34. Melo APS, Acúrcio FA, Cherchiglia ML, Veloso CCG, Guimarães MDC. Avaliação de serviços de saúde mental: assistência e prevenção às doenças sexualmente transmissíveis no contexto do Projeto PESSOAS. Rev. méd. Minas Gerais2007; 17(1/2, Supl.4):S240-S248.,³⁶36. Nascimento AF, Galvanese ATC. Avaliação da estrutura dos centros de atenção psicossocial do município de São Paulo, SP. Rev Saude Publica2009; 43(Supl. 1):8-15.,⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100., other studies have pointed towards services and actions not covering users’ features and needs¹⁵15. Camatta MW, Nasi C, Adamoli AN, Kantorski L P, Schneider JF. Avaliação de um centro de atenção psicossocial: o olhar da família. Cien Saude Colet 2011; 16(11):4405-4414.,²²22. Kantorski LP, Jardim VR, Wetzel C, Olschowsky A, Schneider JF, Heck RM, Bielemann VL, Schwartz E, Coimbra VCC, Quevedo ALA, Saraiva SS. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saude Publica 2009; 43(Supl. 1):29-35.,²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22.,³⁶36. Nascimento AF, Galvanese ATC. Avaliação da estrutura dos centros de atenção psicossocial do município de São Paulo, SP. Rev Saude Publica2009; 43(Supl. 1):8-15.,³⁷37. Olschowsky A, Glanzner CH, Mielke FB, Kantorski L P, Wetzel C. Avaliação de um Centro de Atenção Psicossocial: a realidade em Foz do Iguaçu. Rev. esc. enferm. USP 2009; 43(4):781-787.,⁴⁷47. Mielke FB, Olschowsky A, Pinho LB, Wetzel C, Kantorski LP. Avaliação qualitativa da relação de atores sociais com a loucura em um serviço substitutivo de saúde mental. Rev. bras. enferm. 2012; 65(3):501-507.. Thus, the following changes are highlighted: more diversification and increase in the offer of activities and assistance in CAPS²²22. Kantorski LP, Jardim VR, Wetzel C, Olschowsky A, Schneider JF, Heck RM, Bielemann VL, Schwartz E, Coimbra VCC, Quevedo ALA, Saraiva SS. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saude Publica 2009; 43(Supl. 1):29-35.; the extension of the service to a 24-7 assistence²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22.,³⁵35. Mielke FB, Kantorski LP, Jardim VMR, Olschowsky A. Avaliação de um serviço substitutivo em saúde mental. Cogitare Enferm 2009; 14(1):52-58.; and the reversion of intramural assistance in some CAPS, with greater integration into the daily life of communities and use of their resources¹⁵15. Camatta MW, Nasi C, Adamoli AN, Kantorski L P, Schneider JF. Avaliação de um centro de atenção psicossocial: o olhar da família. Cien Saude Colet 2011; 16(11):4405-4414.,³⁷37. Olschowsky A, Glanzner CH, Mielke FB, Kantorski L P, Wetzel C. Avaliação de um Centro de Atenção Psicossocial: a realidade em Foz do Iguaçu. Rev. esc. enferm. USP 2009; 43(4):781-787.,⁴⁷47. Mielke FB, Olschowsky A, Pinho LB, Wetzel C, Kantorski LP. Avaliação qualitativa da relação de atores sociais com a loucura em um serviço substitutivo de saúde mental. Rev. bras. enferm. 2012; 65(3):501-507..

A contradictory scenario was also observed in inserting families into treatments. In some cases, the care given to the family is seen as a positive point by the substitutive strategies¹⁹19. Coimbra VCC, Kantorski LP, Oliveira MM, Pereira DB, Nunes CK, Eslabão AD. Avaliação da satisfação dos usuários com o cuidado da saúde mental na Estratégia Saúde da Família. Rev. esc. enferm. USP 2011; 45(5):1150-1156., but in other situations it is difficult to empower the family in the treatment and in the routine of all services through shared care¹⁵15. Camatta MW, Nasi C, Adamoli AN, Kantorski L P, Schneider JF. Avaliação de um centro de atenção psicossocial: o olhar da família. Cien Saude Colet 2011; 16(11):4405-4414.,¹⁶16. Duarte MLC, Kantorski LP. Avaliação da atenção prestada aos familiares em um centro de atenção psicossocial. Rev. bras. enferm. 2011; 64(1):47-52.,¹⁸18. Camilo CA, Bandeira M, Leal RMAC, Scalon JD. Avaliação da satisfação e sobrecarga em um serviço de saúde mental. Cad. saúde colet. 2012; 20(1):82-92.,³⁵35. Mielke FB, Kantorski LP, Jardim VMR, Olschowsky A. Avaliação de um serviço substitutivo em saúde mental. Cogitare Enferm 2009; 14(1):52-58.,³⁷37. Olschowsky A, Glanzner CH, Mielke FB, Kantorski L P, Wetzel C. Avaliação de um Centro de Atenção Psicossocial: a realidade em Foz do Iguaçu. Rev. esc. enferm. USP 2009; 43(4):781-787.,⁴⁰40. Wetzel C, Kantorski LP, Olschowsky A, Schneider JF, Camatta MW. Dimensões do objeto de trabalho em um Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2133-2143.,⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100.. Home visits emerge as important mental health care strategy¹⁹19. Coimbra VCC, Kantorski LP, Oliveira MM, Pereira DB, Nunes CK, Eslabão AD. Avaliação da satisfação dos usuários com o cuidado da saúde mental na Estratégia Saúde da Família. Rev. esc. enferm. USP 2011; 45(5):1150-1156., however, with apparent under-use by some teams¹⁵15. Camatta MW, Nasi C, Adamoli AN, Kantorski L P, Schneider JF. Avaliação de um centro de atenção psicossocial: o olhar da família. Cien Saude Colet 2011; 16(11):4405-4414.,⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100.. This strategy got a paradoxical feature in a study involving therapeutic residences’ users, becoming a safety factor for some, but a privacy invasion for others²¹21. Jaegger RC, Guitton A P, Lyrio JM, Santos MM, Freitas RCO, Gonçalves SR, Lima LA, Legay LF. A experiência de morar fora: avaliação da satisfação de usuários em um serviço de saúde mental. Cad. Saúde Colet. 2004; 12(1):27-39..

As for a network perspective, the integration among services emerges³⁶36. Nascimento AF, Galvanese ATC. Avaliação da estrutura dos centros de atenção psicossocial do município de São Paulo, SP. Rev Saude Publica2009; 43(Supl. 1):8-15. as an aim to be achieved⁴⁴44. Miranda L, Campos RTO. Análise das equipes de referência em saúde mental: uma perspectiva de gestão da clínica. Cad Saude Publica 2010; 26(6):1153-1162.. Professionals recognize the importance of developing care networks outside CAPS, but its support is hard due to the lack of community resources, work overload etc.⁴³43. Oliveira E, Oliveira MAF, Claro HG, Paglione HB. Práticas assistenciais no centro de atenção psicossocial de álcool, tabaco e outras drogas. Rev. ter. ocup. 2010; 21(3):247-254.. The need to organize the mental health assistance in the UBSs is highlighted, mainly through the ESF teams, in a way that reinforces the work of primary health care services⁴²42. Campos RTO, Campos GWS, Ferrer AL, Corrêa CAS, Madureira PR, Gama CAP, Dantas DV, Nascimento R. Avaliação de estratégias inovadoras na organização da Atenção Primária à Saúde. Rev Saude Publica 2012; 46(1):43-50..

Campos et al.⁴²42. Campos RTO, Campos GWS, Ferrer AL, Corrêa CAS, Madureira PR, Gama CAP, Dantas DV, Nascimento R. Avaliação de estratégias inovadoras na organização da Atenção Primária à Saúde. Rev Saude Publica 2012; 46(1):43-50. compared UBS’ performance according to the implementation of primary health care and mental health arrangements such as the matrix support, therapeutic projects, clinical case discussions, among others. The authors have identified better results in UBSs that have implemented a bigger number of strategies able to articulate primary health care and mental health in the following aspects: integration of community health agents in the teams; the perception of improvement in the assistance given by professionals; and the facility to referral and care⁴²42. Campos RTO, Campos GWS, Ferrer AL, Corrêa CAS, Madureira PR, Gama CAP, Dantas DV, Nascimento R. Avaliação de estratégias inovadoras na organização da Atenção Primária à Saúde. Rev Saude Publica 2012; 46(1):43-50.. Another study identified that ESF implementation increased the efficacy in mental health consultation appointments⁴⁸48. Koga M, Furegato ARF, Santos JLF. Opiniões da equipe e usuários sobre a atenção à saúde mental num programa de saúde da família. Rev. Latino-Am. Enfermagem 2006; 14(2):163-169.. Ribeiro et al.⁴⁹49. Ribeiro MS, Alves MJM, Silva PM, Vieira EMM. Comparação da assistência em saúde mental em unidades básicas de saúde com ou sem equipe do Programa de Saúde da Família. Rev. psiquiatr. Rio Gd. Sul 2009; 31(1):40-50. compared the mental health assistance profile of UBS with and without the ESF team, and found that those with ESF team presented better data record patterns and more participation of clinicians in referrals.

However, the integration between mental health and primary health care also presents the following barriers: lack of services and professionals in primary health care⁴¹41. Zappitelli MC, Gonçalves EC, Mosca I. Panorama da saúde mental no Estado de São Paulo: leitos psiquiátricos e assistência extra-hospitalar. Rev. adm. saúde 2006; 8(31):71-78.; difficulty in implementing matrix support⁴²42. Campos RTO, Campos GWS, Ferrer AL, Corrêa CAS, Madureira PR, Gama CAP, Dantas DV, Nascimento R. Avaliação de estratégias inovadoras na organização da Atenção Primária à Saúde. Rev Saude Publica 2012; 46(1):43-50.,⁴⁹49. Ribeiro MS, Alves MJM, Silva PM, Vieira EMM. Comparação da assistência em saúde mental em unidades básicas de saúde com ou sem equipe do Programa de Saúde da Família. Rev. psiquiatr. Rio Gd. Sul 2009; 31(1):40-50.; lack of professional training, with short understanding about the care propositions and miss information to users⁴⁸48. Koga M, Furegato ARF, Santos JLF. Opiniões da equipe e usuários sobre a atenção à saúde mental num programa de saúde da família. Rev. Latino-Am. Enfermagem 2006; 14(2):163-169.; referrals as a way to transfer responsibilities⁴⁰40. Wetzel C, Kantorski LP, Olschowsky A, Schneider JF, Camatta MW. Dimensões do objeto de trabalho em um Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2133-2143.; obstacles in referral and counter-referral influencing the dialogue between services³⁴34. Melo APS, Acúrcio FA, Cherchiglia ML, Veloso CCG, Guimarães MDC. Avaliação de serviços de saúde mental: assistência e prevenção às doenças sexualmente transmissíveis no contexto do Projeto PESSOAS. Rev. méd. Minas Gerais2007; 17(1/2, Supl.4):S240-S248.,⁴²42. Campos RTO, Campos GWS, Ferrer AL, Corrêa CAS, Madureira PR, Gama CAP, Dantas DV, Nascimento R. Avaliação de estratégias inovadoras na organização da Atenção Primária à Saúde. Rev Saude Publica 2012; 46(1):43-50.,⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100.,⁴⁹49. Ribeiro MS, Alves MJM, Silva PM, Vieira EMM. Comparação da assistência em saúde mental em unidades básicas de saúde com ou sem equipe do Programa de Saúde da Família. Rev. psiquiatr. Rio Gd. Sul 2009; 31(1):40-50.; and lack of policies going beyond health sector²⁶26. Surjus LTLS, Campos RO. A avaliação dos usuários sobre os Centros de Atenção Psicossocial (CAPS) de Campinas, SP. Rev. latinoam. psicopatol. fundam. 2011; 14(1):122-133..

As for the municipal management role of subsidizing professionals’ actions and services’ functioning, the following problems are pinpointed: investment in hospitalocentric models⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100.; lack of communication with professionals²²22. Kantorski LP, Jardim VR, Wetzel C, Olschowsky A, Schneider JF, Heck RM, Bielemann VL, Schwartz E, Coimbra VCC, Quevedo ALA, Saraiva SS. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saude Publica 2009; 43(Supl. 1):29-35.; untrained manpower and lack of training resources²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22.; lack of investments in CAPS’ infrastructure⁴⁰40. Wetzel C, Kantorski LP, Olschowsky A, Schneider JF, Camatta MW. Dimensões do objeto de trabalho em um Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2133-2143.; and the need of political projects that invests in structure and mental health network flow⁴⁰40. Wetzel C, Kantorski LP, Olschowsky A, Schneider JF, Camatta MW. Dimensões do objeto de trabalho em um Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2133-2143..

The different stigmas given to individuals with mental disorder are evident⁴⁵45. Schneider JF, Camatta MW, Nasi C, Adamoli AN, Kantorski LP. Avaliação de um centro de atenção psicossocial brasileiro. Cienc. Enferm. 2009; 15(3):91-100.. Such stigmatization results from society, but also from professionals⁴⁷47. Mielke FB, Olschowsky A, Pinho LB, Wetzel C, Kantorski LP. Avaliação qualitativa da relação de atores sociais com a loucura em um serviço substitutivo de saúde mental. Rev. bras. enferm. 2012; 65(3):501-507.. Thus, the substitutive services and RAPS work towards social inclusion, demystifying traditional concepts about the psychic suffering⁴⁶46. Pinho LB, Kantorski LP, Wetzel C, Schwartz E, Lange C, Zilmer JGV. Avaliação qualitativa do processo de trabalho em um centro de atenção psicossocial no Brasil. Rev. Panam. Salud Públ. 2011; 30(4):354-360.,⁴⁷47. Mielke FB, Olschowsky A, Pinho LB, Wetzel C, Kantorski LP. Avaliação qualitativa da relação de atores sociais com a loucura em um serviço substitutivo de saúde mental. Rev. bras. enferm. 2012; 65(3):501-507..

Discussion

Assessment is a complex activity and it must not be performed in technicist and instrumental ways. It should take into account services such as reflexive systems within a wide range of connections and interdependences that express conficting relations in the contexts they are inserted in⁵⁰50. Magalhães R, Bodstein R. Avaliação de iniciativas e programas intersetoriais em saúde: desafios e aprendizados. Cien Saude Colet 2009; 14(3):861-868.. This multiple systems rationality is taken to mental health area, since an isolated assessment application may delegitimize public services such as CAPS, whose obstacles encountered for their effectiveness also reflect the barriers of other services and scenarios⁵¹51. Campos RTO, Furtado JP Entre a saúde coletiva e a saúde mental: um instrumental metodológico para avaliação da rede de Centros de Atenção Psicossocial (CAPS) do Sistema Único de Saúde. Cad Saude Publica 2005; 22(5):1053-1062..

In this sense, from the concepts of Novaes¹³13. Novaes HMD. Avaliação de programas, serviços e tecnologias em saúde. Rev Saude Publica 2000; 34(5):547559., it is important to note the prevalence of assessments that aim to impact the decision-making processes or contribute to the improvement of interventions and mental health services. This opposes to the evaluations that aim only to produce knowledge recognized by the academy, from an objectivity and neutrality supposedly reached by the distance between evaluators and people in the service¹³13. Novaes HMD. Avaliação de programas, serviços e tecnologias em saúde. Rev Saude Publica 2000; 34(5):547559..

It is therefore necessary to understand the importance of creating conditions for critical judgments, implementing strategies that enhance exchanges, learning and the opening of new horizons for intervention, understanding the evaluation beyond a rational guideline of choices⁷7. Contandriopoulos AP. Avaliando a institucionalização da avaliação. Cien Saude Colet 2006; 11(3):705-711.. As it was clarified by Contandriopoulos et al.⁵²52. Contandriopoulos AP, Champagne F, Denis JL, Pineault R. A avaliação na área da saúde: conceitos e métodos. In: Hartz ZAM, organizadora. Avaliação em saúde: dos modelos conceituais à prática na análise da implantação de programas. Rio de Janeiro: Editora Fiocruz; 2000. p. 29-47., "as assessment ultimately aims to help decision-making, it is worth questioning about the influence that the information provided by the evaluator may have on the decisions".

It is known that the actors who compose the mental health system present distinct information needs and, mostly, they are not able to get to consensus about the assessment methods because the results do not reach everybody's expectations. A challenge for the evaluation and assessment is to incorporate the views of this plurality of actors and their positions, allowing relevant information to contextualized decisions⁷7. Contandriopoulos AP. Avaliando a institucionalização da avaliação. Cien Saude Colet 2006; 11(3):705-711..

Above all, it has to consider the participation of users involved in the construction of policies and context of services⁶6. Wetzel C, Kantorski LP. Avaliação de serviços em saúde mental no contexto da reforma psiquiátrica. Texto Contexto Enferm. 2004; 13(4):593-598.. Such factor should override potential disabilities or unavailability of services to hear the users and consider their opinions, providing spaces of empowerment over their own living conditions¹⁹19. Coimbra VCC, Kantorski LP, Oliveira MM, Pereira DB, Nunes CK, Eslabão AD. Avaliação da satisfação dos usuários com o cuidado da saúde mental na Estratégia Saúde da Família. Rev. esc. enferm. USP 2011; 45(5):1150-1156.,²³23. Silva MA, Bandeira M, Scalon JD, Quaglia MAC. Satisfação dos pacientes com os serviços de saúde mental: a percepção de mudanças como preditora. J. bras. psiquiatr. 2012; 61(2):64-71..

Analyzing the need of producing consistent information for those involved in the services and the complexity of mental health, it is imperative to consider the combination of objective indicators and subjective phenomena, combining different techniques and methods that capture reality on the move⁸8. Lima FC, Schneider DR. Avaliação dos centros de atenção psicossocial: uma revisão integrativa da literatura nacional. Revista Caminhos 2013; 4(6):39-64., allow processes of collective transformation and are able to produce consistent information for those involved⁷7. Contandriopoulos AP. Avaliando a institucionalização da avaliação. Cien Saude Colet 2006; 11(3):705-711.. The combination of qualitative and quantitative methods, with actors from various interest groups, is a way for Brazil's evidence-based policies on mental health⁵³53. Campos GWS, Onocko-Campos RT, Del Barrio LR. Políticas e práticas em saúde mental: as evidências em questão. Cien Saude Colet 2013; 18(10):2797-2805.. The need for more studies with mixed methods is highlighted, but understanding that the process of interpretation and reflection on the data is essentially qualitative

In the qualitative approach studies, triangulation was found bringing consistency and being used in designs, collection and analysis of data. In the evaluations with quantitative methods, adequacy of instruments initiatives through adaptations and validations increased the validity and reliability of data, enabling comparison⁵⁴54. Silva NS, Melo JM, Esperidião E. Avaliação dos serviços de assistência em saúde mental brasileiros: revisão integrativa da literatura. Rev. Min. Enferm. 2012; 16(2):280-288..

Due to the intrinsic relationship between structural factors, work processes and results, it is emphasized that, from the Donabedian¹⁴14. Donabedian A. The quality of care: How can it be assessed? JAMA 1988; 260(12):1743-48. classification, almost half of the studies evaluated the quality of care encompassing these three integrated dimensions. This can help to a more comprehensive and integrated understanding of the reality of services.

Thus, it is important to think about possibilities of integrating the evaluative production, consolidating paths for services and developing an evidence-based mental health system⁸8. Lima FC, Schneider DR. Avaliação dos centros de atenção psicossocial: uma revisão integrativa da literatura nacional. Revista Caminhos 2013; 4(6):39-64.,⁵⁵55. Mari JJ, Bressan RA, Almeida-Filho N, Gerolin J, Sharan P, Saxena S. Mental health research in Brazil: policies, infrastructure, financing and human resources. Rev Saude Publica 2006; 40(1):161-169.. The goal is not to build "a single thought and dominant" and restrict the creativity of researchers, but encompass some theoretical coherence between knowledge⁵5. Almeida PF, Escorel S. Da avaliação em saúde à avaliação em Saúde Mental: gênese, aproximações teóricas e questões atuais. Saúde Debate 2001; 25(58):35-47.. Therefore, it is important to incorporate indicators from previous studies and replicate models and consistent evaluative techniques, so that you can compare results and reflect on services and policies⁸8. Lima FC, Schneider DR. Avaliação dos centros de atenção psicossocial: uma revisão integrativa da literatura nacional. Revista Caminhos 2013; 4(6):39-64.. Production in the area appears to grow in recent years, with the possible explanations: 1) higher interest on the subject by academy, boosted by the pioneering work of research groups in the area, such as UFPEL, UFSJ and UNICAMP8; and 2) increased investments by funding agencies, allowing "a modest but growing presence in mental health research in the international scenario"⁵⁵55. Mari JJ, Bressan RA, Almeida-Filho N, Gerolin J, Sharan P, Saxena S. Mental health research in Brazil: policies, infrastructure, financing and human resources. Rev Saude Publica 2006; 40(1):161-169..

However, this growth trend should be problematized, due to concentration of studies in the South and Southeast. These findings make us question the reasons of this non-appropriation over the theme in North, Northeast and Midwest. It is therefore necessary to consider local, regional and national differences between services, providing an expanded overview of the Brazilian reality⁵⁴54. Silva NS, Melo JM, Esperidião E. Avaliação dos serviços de assistência em saúde mental brasileiros: revisão integrativa da literatura. Rev. Min. Enferm. 2012; 16(2):280-288.. The development of research on mental health services in these areas can be promoted through groups, collaborative networks or targeted funding.

An important aspect concerns the account in the assessment of the particularities that certain conditions within mental health have, such as drug use, care for children and adolescents, among others. It is believed that the low number of evaluations on services such as CAPSad and CAPSi is a reflection of insufficiencies of these devices (there are only 309 and 201 CAP-Si CAPSad in Brazil⁴4. Sala de Apoio à gestão Estratégica [acessado 2015 fev 19]. Disponível em: http://189.28.128.178/sage/.
http://189.28.128.178/sage/... ), or obstacles in the shared care. Such aspects and services cannot be seen as mere adjacencies resulting from the transposition of reified models that crystallize practices and disregard particularities. These are factors that must be fully encompassed in order to equally and democratically adjust the actions taken to fulfill the populations’ needs, according to a broad health perspective⁴⁴44. Miranda L, Campos RTO. Análise das equipes de referência em saúde mental: uma perspectiva de gestão da clínica. Cad Saude Publica 2010; 26(6):1153-1162..

As for the analyses done over the assessment results, prevail better working and assistance conditions with: reversal of the insufficiency of substitutive services; advances in financing and provision of human and structural resources; better work guarantees; strengthening of network perspective; and changes in the training and professional qualification¹⁵15. Camatta MW, Nasi C, Adamoli AN, Kantorski L P, Schneider JF. Avaliação de um centro de atenção psicossocial: o olhar da família. Cien Saude Colet 2011; 16(11):4405-4414.,¹⁸18. Camilo CA, Bandeira M, Leal RMAC, Scalon JD. Avaliação da satisfação e sobrecarga em um serviço de saúde mental. Cad. saúde colet. 2012; 20(1):82-92.,¹⁹19. Coimbra VCC, Kantorski LP, Oliveira MM, Pereira DB, Nunes CK, Eslabão AD. Avaliação da satisfação dos usuários com o cuidado da saúde mental na Estratégia Saúde da Família. Rev. esc. enferm. USP 2011; 45(5):1150-1156.,²²22. Kantorski LP, Jardim VR, Wetzel C, Olschowsky A, Schneider JF, Heck RM, Bielemann VL, Schwartz E, Coimbra VCC, Quevedo ALA, Saraiva SS. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saude Publica 2009; 43(Supl. 1):29-35.,²³23. Silva MA, Bandeira M, Scalon JD, Quaglia MAC. Satisfação dos pacientes com os serviços de saúde mental: a percepção de mudanças como preditora. J. bras. psiquiatr. 2012; 61(2):64-71.,²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22.,³⁴34. Melo APS, Acúrcio FA, Cherchiglia ML, Veloso CCG, Guimarães MDC. Avaliação de serviços de saúde mental: assistência e prevenção às doenças sexualmente transmissíveis no contexto do Projeto PESSOAS. Rev. méd. Minas Gerais2007; 17(1/2, Supl.4):S240-S248.–⁴⁰40. Wetzel C, Kantorski LP, Olschowsky A, Schneider JF, Camatta MW. Dimensões do objeto de trabalho em um Centro de Atenção Psicossocial. Cien Saude Colet 2011; 16(4):2133-2143..

The matrix support emerges as a strategy to reverse the referral logic by defining flows, qualifying teams and operationalizing the shared care. Based on the contact with reality, such arrangement allows the construction of reflexive spaces about practices and knowledge in different fields⁵⁶56. Bonfim IG, Bastos ENE, Góis CWL, Tófoli LF. Apoio matricial em saúde mental na atenção primária à saúde: uma análise da produção científica e documental. Interface (Botucatu) 2013; 17(45):287-300.. It should be enhanced and institutionalized through structured systems, where specialized teams such as CAPS, entails continuous actions of supervision and continuing education to the rest of the professionals and network services⁴⁹49. Ribeiro MS, Alves MJM, Silva PM, Vieira EMM. Comparação da assistência em saúde mental em unidades básicas de saúde com ou sem equipe do Programa de Saúde da Família. Rev. psiquiatr. Rio Gd. Sul 2009; 31(1):40-50.. Thus it is possible to reorient the work, promoting the integration of services, with indiscriminate referrals giving way to co-responsibility⁴²42. Campos RTO, Campos GWS, Ferrer AL, Corrêa CAS, Madureira PR, Gama CAP, Dantas DV, Nascimento R. Avaliação de estratégias inovadoras na organização da Atenção Primária à Saúde. Rev Saude Publica 2012; 46(1):43-50.,⁵⁶56. Bonfim IG, Bastos ENE, Góis CWL, Tófoli LF. Apoio matricial em saúde mental na atenção primária à saúde: uma análise da produção científica e documental. Interface (Botucatu) 2013; 17(45):287-300.,⁵⁷57. Figueiredo MD, Campos RTO. Saúde Mental na atenção básica à saúde de Campinas, SP: uma rede ou um emaranhado? Cien Saude Colet 2009; 14(1):129-138..

Such findings are corroborated by Gregório et al.⁵⁸58. Gregório G, Tomlinson M, Gerolin J, Kieling C, Moreira HC, Razzouk D, Mari JJ. Setting priorities for mental health research in Brazil. Rev. Bras. Psiquiatr. 2012; 34(4):434-439. who reviewed the mental health research agenda in Brazil and found the following priorities: interventions in the primary care level and the integration between primary and mental health; assessments of theses services’ policies; analysis about the cost and effectiveness of antipsychotics; development of interventions able to decrease drug use; identification of obstacles to treatment; and the training and supervision of non-specialized professionals.

Thereby, it is questioned the participation of management (federal, state and municipal levels) in the formulation and implementation of policies and projects that allow the integrated and broad work from RP³⁷37. Olschowsky A, Glanzner CH, Mielke FB, Kantorski L P, Wetzel C. Avaliação de um Centro de Atenção Psicossocial: a realidade em Foz do Iguaçu. Rev. esc. enferm. USP 2009; 43(4):781-787.. The need for democratic management as well as the need to enhance the training given to the managers must be considered, since they are responsible for the management of the RAPS, besides their responsibility for the work provided by the interdisciplinary teams²⁷27. Campos RTO, Furtado JP, Passos E, Ferrer AL, Miranda L, Gama CAP. Avaliação da rede de centros de atenção psicossocial: entre a saúde coletiva e a saúde mental. Rev Saude Publica 2009; 43(Supl. 1):16-22..

Based on the findings it is possible to notice the beginning of a second moment (or phase) in the assessment and evaluation of mental health services in Brazil. The substitutive strategies and the deinstitutionalization process, despite their barriers, appear to be more human, effective and efficient than the hospitalocentric models. Therefore, the assessment and evaluation processes must overpass the mere comparison between substitutive services and psychiatric hospitals. This does not mean that such comparisons are unnecessary and that they do not need to be reinforced through assessments, due to the complexity of the field and political factors. However, it is suggested to go beyond these comparisons by applying critical exercises – such as those found in some reviewed articles – that problematize the barriers of policies and substitute care models, ‘breaking crystallized’ practices and knowledge. In this way, it is believed that the services can be enhanced, contributing to the consolidation of Brazilian R P, which is a continuous process.

Finally, the current study presented the following limitations: disregard of articles indexed in other databases or published later; the used terminology, since there was no standardization in articles’ keywords; and the exclusion of dissertations, thesis, reports and a completely gray literature difficult to systematize. Despite these limitations, it is believed that it was possible to demonstrate and review the panorama of the evaluation of mental health services in Brazil.

Conclusions

The current review, just as it happens with well-succeeded assessments, instead of closing the discussions about mental health services in Brazil, questions the problem and opens possibilities to new inquiries. Mental health assessment does not end in itself, being a questioning, dynamic and continuous exercise that holds different actors, methods and competences⁵²52. Contandriopoulos AP, Champagne F, Denis JL, Pineault R. A avaliação na área da saúde: conceitos e métodos. In: Hartz ZAM, organizadora. Avaliação em saúde: dos modelos conceituais à prática na análise da implantação de programas. Rio de Janeiro: Editora Fiocruz; 2000. p. 29-47..

Different concepts about mental health assessment were observed and they covered many methods and participants, fact that reflects the complexity of this field. It is postulated a greater integration among research, aiming at gathering and deepening the knowledge. However, this does not mean imposing paradigms and methods and restricting researchers’ creativity.

It is essential to expand the substitutive devices and RAPS by investing in infrastructure, qualification of human resources and organizational improvement. However, just the structural components are not enough to surpass the challenges set by the complexity of the theme. There is the necessity that assessments of mental health services go beyond these aspects and the comparison with traditional hospitalocentric models, by questioning the difficulties of implementing the new models and policies and reinforcing their potentialities.

Acknowledgments

To Daniela Mota and Leonardo Martins for the discussions and valuable contributions. To the reviewers for the suggestions that have improved the quality of the article. To CAPES, FAPEMIG and CNPQ.

References

Publication Dates

History