Abstract

In the cross-cultural adaptation of instruments, the qualitative component of adaptation is generally poorly reported, sometimes being superficially addressed. In this study we aimed to describe the qualitative component of the cross-cultural adaptation process and to demonstrate validity evidence based on test content of the Recovery Self-Assessment-RSA-R Family/Brazil. We conducted a qualitative study that included the steps of preparation, translation, back-translation, expert’s assessment, workshop with a researcher from Yale University, and two pilot studies involving family members of patients attended at mental health services. Among the results, we found considerable validity evidence based on test content with a percentage of agreement above 80%. Pilot studies contributed to accentuating this evidence, assisting in the cultural adequacy of the statements and in the operational equivalence of the instrument. The adaptation process of the RSA-R Family/Brazil proved to be complex. From this experience, we concluded that presenting validity evidence based on test content is important to ensure the applicability tools to the target culture. The instrument will still be evaluated as for psychometric characteristics through statistical techniques.

Key words:
Recovery; Mental Health Services; Family; Validation studies; Qualitative analysis

Introduction

According to the Standards for Educational and Psychological Testing, established by the American Educational Research Association, validity evidence based on test content, also generally known as content validity, is one of the five sources of validity evidence of an instrument. Test content refers to themes, words and format of the items, administration, and test scores¹1 American Educational Research Association. American Psychological Association. National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.^,22 Sireci S, Faulkner-Bond M. Validity evidence based on test content. Psicothema 2014]; 26(1):100-107..

Validity evidence based on test content, in turn, refers to the level at which the content is congruous with the purposes of a test¹1 American Educational Research Association. American Psychological Association. National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014., i.e., “it evaluates the degree to which each element of an assessment instrument is relevant and representative for a specific construct with a particular evaluation purpose”³3 Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet 2011; 16(7):3061-3068.(p.3063, free translation).

Overall, cross-cultural adaptation studies are focused on providing statistical evidences for demonstrating the validity and reliability of an instrument. Therefore, they lack emphasis on validity evidence based on test content and on the qualitative process of adapting a questionnaire to a new culture. However, this process must be equally important to statistical evidence, considering that a good adaptation cannot disregard differences arising from language, cultural context, and lifestyle, since results of this process will be reflected on the statistical results³3 Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet 2011; 16(7):3061-3068..

In this study we address the cross-cultural adaptation of an instrument for assessing the recovery-oriented practices of mental health services, with special emphasis on the qualitative process involved in adapting it to the Brazilian context. Within this context, recovery, in the field of mental health, is understood “as a deeply personal, unique process of changing one’s attitudes, values, feelings, goals, skills, and/or roles. It is a way of living a satisfying, hopeful, and contributing life even with limitations caused by illness”⁴4 Anthony WA. Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosoc Rehabil J 1993; 16(4):11-23.(p.525).

We chose the Recovery Self-Assessment RSA-R Family instrument to involve family members of users of mental health services in their care, since, after the psychiatric reform in Brazil, the family became the most responsible people for providing care in recovery. Thus, providing tools that include them would allow establishing a partnership between relatives and services, enabling them to participate in decisions, and sharing the time and responsibility dedicate to care with them⁵5 Rosa LCS. Transtorno mental e o cuidado na familia. 2ª ed. São Paulo: Cortez; 2008..

RSA is one of the most used scales to assess the recovery-oriented practices of mental health services⁶6 Williams J, Leamy M, Bird V, Harding C, Larsen J, Le Boutillier C, Oades L, Slade M. Measures of the recovery orientation of mental health services : systematic review. Soc Psychiatry Psychiatr Epidemiol 2012; 47(11):1827-1835.. Such use favors the reflection on strengths and on limitations of the services within this scope⁷7 O'Connell M, Tondora J, Croog G, Evans A, Davidson L. From rhetoric to rutine: assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatr Rehabil J 2005; 28(4):378-386.. This scale has good psychometric properties and is commonly used in evaluations of mental health services in other countries⁸8 Barbic SP, Kidd SA, Davidson L, Connell MJO. Validation of the brief version of the recovery self-assessment (rsa-b) using rasch measurement theory. Psychiatr Rehabil J 2015; 38(4):349-358.

9 Bola J, Chan THC, Chen EH, Ng R. Cross-validating chinese language mental health recovery measures in Hong Kong. Res Soc Work Pract 2016; 26(6):630-640.

10 Burgess P, Pirkis J, Coombs T, Rosen A. Assessing the value of existing recovery measures for routine use in Australian mental health services. Aust N Z J Psychiatry 2011; 45(4):267-280.

11 Kidd SA, George L, O'Connell M, Sylvestre J, Kirkpatrick H, Browne G, Thabane L. Fidelity and recovery-orientation in assertive community treatment. Community Ment Health J 2010; 46(4):342-350.

12 Leamy M, Clarke E, Boutillier C Le, Bird V, Choudhury R, Macpherson R, Pesola F, Sabas K, Williams P, Slade M. Recovery practice in community mental health teams: national survey. Br J Psychiatry 2016; 209(4):340-346.

13 Petrakis M, Brophy L, Lewis J, Stylianou M, Scott M, Cocks N, Buckley L, Halloran K. Consumer measures and research co-production: a pilot study evaluating the recovery orientation of a mental health program collaboration. Asia Pac J Soc Work Dev 2014; 24(1-2):94-108.

14 Rosenberg D, Svedberg P, Schon U-K. Establishing a recovery orientation in mental health services: evaluating the recovery self-assessment (rsa) in a Swedish context. Psychiatr Rehabil J 2015; 38(4):328-335.

15 Simpson A, Hannigan B, Coffey M, Barlow S, Cohen R, Jones A, Všetečková, J, Faulkner A, Thornton A, Cartwright M. Recovery-focused care planning and coordination in England and Wales: a cross-national mixed methods comparative case study. BCM Psychiatry 2016; 16:147.

16 Waldemar AK, Esbensen BA, Korbek L, Petersen L, Arnfred S. Recovery orientation in mental health inpatient settings: Inpatient experiences? Int J Ment Health Nurs 2018; 27(3):1177-1187.

17 Ye S, Pan J, Wong DFK, Bola JR. Cross-validation of mental health recovery measures in a Hong Kong chinese sample. Res Soc Work Pract 2013; 23(3):1177-1187.^-1818 Zuaboni G, Kieser LD, Glavanovits K, Utschakowski J, Behrens J. Recovery Self Assessment: Translation and cultural adaption of a recovery oriented assessment instrument. Pflege 2015; 28(4):233-243..

In our study we aimed to describe the qualitative component of the cross-cultural adaptation process of the RSA-R Family/Brazil and to demonstrate validity evidence based on test content of the instrument. We present the main results of the instrument adaptation process by analyzing how the qualitative component was treated. Our study becomes relevant for reviewing methodological processes in cross-cultural adaptation research, based on the experience reported in each step, thus showing the considerable bonding potential of the target population in this type of research.

In addition, investing in the adaptation of instruments for assessing mental health services, within the recovery-oriented focus, shall bring gains in the field of public health. Among these gains we can mention that the instrument allows knowing the degree to which mental health services involve users and family members in the treatment; likewise, there is the gain of enabling the implementation of services that promote the autonomy of people with mental disorders, since authors of studies conducted so far¹⁹19 Thomas EC, Despeaux KE, Drapalski AL, Bennett M. Person-oriented recovery of individuals with serious mental illnesses: a review and meta-analysis of longitudinal findings. Psychiatr Serv 2018; 69(3):259-267.

20 Zipursky RB, Reilly TJ, Murray RM. The myth of schizophrenia as a progressive brain disease. Schizophr Bull 2012; 39(6):1363-1372.^-2121 Leonhardt BL, Huling K, Hamm JA, Roe D, Hasson-Ohayon I, McLeod HJ, Lysaker PH. Recovery and serious mental illness: a review of current clinical and research paradigms and future directions. Expert Rev Neurother 2017; 17(11):1117-1130. point to reducing the chronicity of the disease and the disabilities arising from it, thus improving health conditions and increasing the quality of life of people with mental disorders and their family members.

Methods

This is a qualitative study that involved family members of persons in recovery attended at Psychosocial Support Centers (Centros de Atenção Psicossocial - CAPS) and Community Centers (Centros de Convivência - CECOs) located in Campinas, state of São Paulo, Brazil, from June 2016 to December 2017. Family members of users attended at these services for a period longer than three months, aged over 18 years, who were able to communicate in Brazilian Portuguese, without any cognitive impairment, and who agreed to participate were eligible for the study. In this convenience sampling, individuals were invited to participate in the study as voluntaries and did not receive financial incentives. The study was approved by the Research Ethics Committee of the University of Campinas.

Assessment Instrument

RSA-R was designed by Yale University, from the United States of America, and contains 32 items for each stakeholders (person in recovery, family member, provider, and CEO and directors). However, in the versions concerning family members and CEO and directors, the instrument contains appendixes with 8 and 4 additional items, respectively, for assessing 6 domains of recovery⁷7 O'Connell M, Tondora J, Croog G, Evans A, Davidson L. From rhetoric to rutine: assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatr Rehabil J 2005; 28(4):378-386.^,2222 O'Connell M, Tondora J, Kidd, Stayner, Hawkins, Davidson L. RSA-R Family Member/Significant Other Version [Internet]. Yale University; 2007 [accesado 2017 fev 14]. Disponivel em: http://medicine.yale.edu/psychiatry/prch/tools/RSA_Family_Advocate_tcm800-204209_tcm800-284-32.pdf.
http://medicine.yale.edu/psychiatry/prch... .

The RSA-R Family version consists of 32 items and 8 appendixes, with five response options to be chosen: 1 (strongly disagree) to 5 (strongly agree), and two additional options, D/K (do not know) and N/A (not applicable). It is divided into six factors: life goals, involvement, diversity of treatment options, choices, individually-tailored services, inviting, and the family only appendix⁷7 O'Connell M, Tondora J, Croog G, Evans A, Davidson L. From rhetoric to rutine: assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatr Rehabil J 2005; 28(4):378-386.^,2222 O'Connell M, Tondora J, Kidd, Stayner, Hawkins, Davidson L. RSA-R Family Member/Significant Other Version [Internet]. Yale University; 2007 [accesado 2017 fev 14]. Disponivel em: http://medicine.yale.edu/psychiatry/prch/tools/RSA_Family_Advocate_tcm800-204209_tcm800-284-32.pdf.
http://medicine.yale.edu/psychiatry/prch... .

Procedure

The cross-cultural adaptation of the RSA-R Family scale for the Brazilian context was carried out mainly following the Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes²³23 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P, ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (pro) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005; 8(2):94-104. and other guidelines proposed in the literature²2 Sireci S, Faulkner-Bond M. Validity evidence based on test content. Psicothema 2014]; 26(1):100-107.^,33 Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet 2011; 16(7):3061-3068.. It involved a total of seven steps, as described next.

Preparation

Authorization was obtained from the main author of the scale, in such a way it could be translated and adjusted to Brazilian Portuguese. A literature review was also conducted on the concept of “recovery” and its equivalence with the notion of psychosocial rehabilitation, generally used in Brazil²⁴24 Onocko Campos RT, Costa M, Pereira MB, Ricci EC, Silva TEG, Erazo CLJ, Reis G, Davidson L. Recovery, citizenship, and psychosocial rehabilitation: A dialog between Brazilian and American mental health care approaches. Am J Psychiatr Rehabil 2017; 20(3):311-326..

Ideally, the process of cross-cultural adaptation of an instrument should achieve the maximum equivalence between the original instrument and its adapted version. According to Gorenstein et al.²⁵25 Gorenstein C, Wang Y, Hungerbühler I. Fundamentos de mensuração em saúde mental. In: Gorenstein C, Wang Y, Hungerbühler I, editores. Instrumentos de avaliação em saúde mental. Porto Alegre: Artmed; 2016. p. 1-52., “equivalence implies that the effectively observed differences between samples from different cultures result from cultural differences, which are not caused by the form or the evaluation of the constructs of interest”²⁵25 Gorenstein C, Wang Y, Hungerbühler I. Fundamentos de mensuração em saúde mental. In: Gorenstein C, Wang Y, Hungerbühler I, editores. Instrumentos de avaliação em saúde mental. Porto Alegre: Artmed; 2016. p. 1-52.(p.13, free translation). Equivalence is divided into several categories. There is no consensus in the literature on the categories and their denominations; nevertheless, overall, equivalences associated with conceptual definitions, instrument translation, application, and psychometric properties are included²⁵25 Gorenstein C, Wang Y, Hungerbühler I. Fundamentos de mensuração em saúde mental. In: Gorenstein C, Wang Y, Hungerbühler I, editores. Instrumentos de avaliação em saúde mental. Porto Alegre: Artmed; 2016. p. 1-52..

Translation and Back-translation

Initially, a bilingual translator, whose mother tongue was Brazilian Portuguese and who was aware of the research objectives, translated the scale from English to Portuguese. Then, a second bilingual translator, whose mother tongue was English and who did not know the objectives of the research, did the back translation, translating the instrument back to English. From these two versions, the research team compared the back-translation with the original instrument, in order to identify inconsistencies and make corrections.

Experts’ Evaluation

The version resulting from the previous steps was evaluated by five bilingual experts in the field of mental health, whose mother tongue was Brazilian Portuguese. There were four psychologists and one occupational therapist, who worked in mental health services and were aware of the recovery construct. Such professionals were familiar with the target population and the construct (Chart 1)²⁶26 Pereira MB. Avaliação de serviços de saúde mental: validação da versão para trabalhadores de instrumento de avaliação de Recovery [tese]. Campinas: Universidade Estadual de Campinas; 2019.. They aimed at verifying the validity evidence based on test content by assessing conceptual, semantic, idiomatic, and experiential equivalence, concerning the title, instructions, response options, and items. The agreement percentage in these aspects was above 80%. After agreement analysis and, considering the experts’ suggestions, the research team made a qualitative analysis and adapted the instrument.

Pilot I-Focus Group Assessment

A first pilot study was carried out with the participation of nine family members, based on the Focus Group (FG) technique, divided into three meetings. In the first two meetings, the objectives of the research were explained, the Informed Consent Form was filled in, and the items present in the instrument were discussed. In the third meeting, the already modified instrument was applied to the same participants, and aspects regarding the layout of the instrument (title, instructions, response options) were discussed, in addition to the items, in such a way to guarantee conceptual, semantic, and operational equivalences of the instrument²⁷27 Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000; 25(24):3186-3191..

Workshop with a researcher from Yale University

The “Workshop: Recovery-Oriented Mental Health Systems of care: assessment, indicators and meaning” was held with the participation of Professor PhD. Maria O’Connell, from Yale University, United States, the main author of the original instrument. In this space were discussed items identified in the FG and deemed as challenging, which were previously back-translated, in order to guarantee the equivalence between the original instrument and the target version.

Pilot II-Interviews

The pre-final questionnaire was applied to 10 family members, using the Cognitive Debriefing Interviews technique. This technique implied an in-depth interview, during which the participants were able to explain the questions out loud and comment on any difficulties in understanding²³23 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P, ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (pro) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005; 8(2):94-104.^,2828 Eremenco SL, Cella D, Arnold BJ. A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Eval Health Prof 2005; 28(2):212-232.. An education professional (graduated in Pedagogy), who works with Popular Education in the city of Campinas, participated in the discussion about the instrument to analyze the material and propose changes.

A harmonization step was carried out after each step, from which, according to the findings and the consensus of the research team, the instrument was adjusted seeking to maintain similarities to the original content, without disregarding the singularity to the cultural context of the Brazilian Portuguese language²³23 Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P, ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (pro) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005; 8(2):94-104.. These steps are illustrated in Figure 1.

Data Analysis

The qualitative analysis of these steps followed the critical hermeneutic approach, according to which several perceptions are sought to be articulated in an intelligible way, obtaining a more complex and reliable result of the object under study at the end of the analysis, thus establishing a dialogical relationship between participants and researchers, assuming that any analysis will only be performed within this articulation²⁹29 Onocko-Campos R. O planejamento no labirinto: uma viagem hermeneutica. São Paulo: Hucitec Editora; 2003..

Results

Validation involves careful attention to possible distortions of meanings resulting from the inadequate representation of the construct and measurement aspects, such as the test format and administration conditions or language level, which may limit or qualify the interpretation of test scores in material terms¹1 American Educational Research Association. American Psychological Association. National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.. The contribution of experts and family members in this study was paramount to identify these aspects.

As a result of the experts’ evaluation step, we obtained an agreement percentage above 80%, both for the layout of the instrument and for the items. Thus, for the RSA-R Family/Brazil, the minimum agreement percentage between raters was 87.5% (items 26 and 38), and a maximum agreement percentage of 100% (items 7, 8, 10, 11, 12, 19, 20, 28, 35, and 36). Regarding the layout, there was an agreement of 98.75% regarding the instrument title; of 96.25% regarding the instructions; and of 98.75% regarding the scale response options (Table 1).

Nineteen family members also participated in this study, 9 in the Pilot I and 10 in the Pilot II. We present the sociodemographic data in Table 2. Most participants were women, with a mean age of 53.22 years (±18.1) in Pilot I, and of 57.60 (±18.4) in Pilot II. Regarding the bond established with the user, most of the participants were parents, or spouses, of people diagnosed with schizophrenia or depression and who had been in treatment for more than 12 years in mental health services.

We analyzed and adopted the observations and suggestions of experts and family members throughout the process of cross-cultural adaptation, which resulted in five modifications to the RSA-R Family/Brazil scale (Chart 2).

From the analysis of Pilot studies I and II, we identified limitations in the operational equivalence, specifically concerning the way the questionnaire was applied to the target population, especially with participants with low level of education. The first modification involved applying the scale in the form of an interview, and not as a self-administered questionnaire, as in the original scale. We also deemed necessary to create a document with general guidelines containing instructions and response options, which the interviewer should read and hand in to the participants to facilitate their answers. The interviewer should also reciprocate the reading of the statements and response options, noting down each of the responses given by the participants.

Throughout the cross-cultural adaptation process, we observed difficulties in adapting the sentence “my loved one” which, in the original version, was used to refer to the person in recovery. Initially, we chose to translate it, into Portuguese, as “meu familiar” (“my relative”). However, in the pilot studies, we verified some difficulties at the time of answering the questions, mainly on the part of participants whose level of education was low. Thus, taking advantage of the fact that the scale would no longer be self-administered, the second modification consisted in indicating, in the sentences, that the interviewer could say the user’s name instead of “my relative.” This modification was made in items 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 34, 35, and 36. This made the sentences more specific and easier to understand, preventing them from having more than one interpretation.

The third modification refers to the division of the scale into two parts, considering that 32 items were related to the family member’s perception of the assistance provided to the person in recovery, and the remaining 8 items were formulated in the first person singular, investigating their own experience as a relative, which generated misunderstandings at the time of responding the questions. Therefore, we chose to insert in the first part of the scale the 8 items regarding the family member’s own experience and, in the second part, the remaining 32 items related to the participant’s perception of the user’s experience.

The fourth modification was made based on the experts’ suggestion and improved, later, with the popular education professional in the pilot study step. According to the experts, denominating all the response options would facilitate the understanding of the scale. In the original scale, only the response options “1=strongly disagree” and “5=strongly agree” were denominated. In the Brazilian version of the scale, all the response options were designated, “1=Strongly disagree,” “2=Disagree,” “3=Undecided,” “4=Agree,” and “5=Strongly agree”. We also added graphic elements to facilitate the understanding.

Sentences in the original scale were long and had examples to improve the understanding. However, in both pilot studies, this became a problem, because, overall, participants were only focused on understanding some examples and not the sentence as a whole. In this sense, as a fifth modification, we deemed necessary to shorten and reformulate the writing of the sentences in a simpler language, making them clearer and facilitating their understanding on the part of participants. We also excluded the examples of items 2, 9, 16, 19, and 22.

Furthermore, some items have become challenging throughout the cross-cultural adaptation process (items 13, 21, 25, 29, 30, and 32) due to cultural differences and, in some cases, the lack of certain practices in mental health services in Brazil. For example, item 21, which refers to the practice of “peers support,” was divided into two items that question, in general, the existence of these and other similar strategies in mental health institutions.

Finally, the use of the “recovery” construct - to which the scale refers - was challenging throughout the cross-cultural adaptation, since according to the experts’ evaluation, the literal translation into Portuguese as “recuperação” could evoke a simplistic interpretation of the term. Initially, we considered to keep the term in English, an option that has been discussed in other scientific articles³⁰30 Baccari IOP, Onocko-Campos R, Stefanello S. Recovery: revisão sistemática de um conceito. Cien Saude Colet 2015; 20(1):125-136.^,3131 Vasconcelos EM. As abordagens Anglo-Saxônicas de empoderamento e recovery (recuperação, restabelecimento) em saúde mental II: uma avaliação crítica para uma apropriação criteriosa no cenário brasileiro. Cad Bras Saude Ment 2017; 9(21):48-65.. Nevertheless, in the first pilot study, the need for finding an appropriate term in Brazilian Portuguese remained evident. We discussed this aspect with the research team and the researcher responsible from the U.S. for the aforementioned workshop. We reached the consensus on keeping the word “recuperação” as a literal translation, but inserted a brief explanation of the concept in the general guidelines that interviewers must read before filling in the questionnaire with the participants.

Discussion

Providing instruments for the assessment of mental health services in Brazil, which are not centered only on the managerial perspective, is an imperative task. The RSA-R Family/Brazil allows including a long-forgotten stakeholder in the country’s evaluative research. This insertion can contribute to improving the quality of mental health services by incorporating the collected information into the treatment³²32 Dantas CDR, Oda AMGR. Cartografia das pesquisas avaliativas de serviços de saúde mental no Brasil (2004-2013). Physis Rev Saude Colet 2014; 24(4):1127-1179..

The validity evidence based on test content was supported by the fact that, fundamentally, the instrument was evaluated and discussed by experts on the mental health field, who analyzed it and considered conceptual, semantic, idiomatic, and experiential equivalences, which resulted in substantial agreement between the items and the construct to be measured.

The equivalence between the original instrument and its version adapted to Brazil was maintained by including family members as a stakeholder in both pilot studies. There was a special emphasis on using techniques that allowed face-to-face meetings with participants, involving them not only as spectators who filled in the questionnaires, but also encouraging them so they could talk about the strengths, weaknesses, and applicability of the instrument to the Brazilian context.

We made this choice aiming at maintaining the procedures adopted in the creation of the original RSA, in which there was involvement of stakeholders (person in recovery, family member, provider, and CEO and directors) who assessed the scale in terms of content and understanding⁷7 O'Connell M, Tondora J, Croog G, Evans A, Davidson L. From rhetoric to rutine: assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatr Rehabil J 2005; 28(4):378-386.. This strategy was repeated in most of the cross-cultural adaptations of RSA performed in other countries, especially in the versions concerning providers and users¹⁰10 Burgess P, Pirkis J, Coombs T, Rosen A. Assessing the value of existing recovery measures for routine use in Australian mental health services. Aust N Z J Psychiatry 2011; 45(4):267-280.^,1515 Simpson A, Hannigan B, Coffey M, Barlow S, Cohen R, Jones A, Všetečková, J, Faulkner A, Thornton A, Cartwright M. Recovery-focused care planning and coordination in England and Wales: a cross-national mixed methods comparative case study. BCM Psychiatry 2016; 16:147.^,3333 Mcloughlin KA, Wick A Du, Collazzi CM, Puntil C. Recovery-oriented practices of psychiatric-mental health nursing staff in an acute hospital setting. J Am Psychiatr Nurses Assoc 2013; 19(3):152-159.^,3434 Lodge AC, Kuhn W, Earley J, Manser SS. Initial development of the recovery-oriented services assessment: a collaboration with peer-provider consultants. Psychiatr Rehabil J 2018; 41(2):92-102.. In the process of cross-cultural adaptation of the RSA Family carried out in Canada and China, no involvement of family members in the process was identified, only the participation in answering the questionnaire⁹9 Bola J, Chan THC, Chen EH, Ng R. Cross-validating chinese language mental health recovery measures in Hong Kong. Res Soc Work Pract 2016; 26(6):630-640.^,1111 Kidd SA, George L, O'Connell M, Sylvestre J, Kirkpatrick H, Browne G, Thabane L. Fidelity and recovery-orientation in assertive community treatment. Community Ment Health J 2010; 46(4):342-350..

In Brazil, as pointed out by Vasconcelos³¹31 Vasconcelos EM. As abordagens Anglo-Saxônicas de empoderamento e recovery (recuperação, restabelecimento) em saúde mental II: uma avaliação crítica para uma apropriação criteriosa no cenário brasileiro. Cad Bras Saude Ment 2017; 9(21):48-65., there are socioeconomic and cultural differences (low education, difficulty in accessing citizen’s rights and exercising citizenship) that challenge the dialogue about recovery and its implementation. This statement was evidenced in the cross-cultural adaptation of the RSA-R Family/Brazil in four factors.

The first one was the participants’ difficulty in filling in the instrument without guidance, in such a way we made the Brazilian version of the RSA-R Family/Brazil to be applied through an interview. In specific studies on the family member’s version, we found no evidence of the use of face-to-face strategies to fill in the questionnaire⁹9 Bola J, Chan THC, Chen EH, Ng R. Cross-validating chinese language mental health recovery measures in Hong Kong. Res Soc Work Pract 2016; 26(6):630-640.^,1111 Kidd SA, George L, O'Connell M, Sylvestre J, Kirkpatrick H, Browne G, Thabane L. Fidelity and recovery-orientation in assertive community treatment. Community Ment Health J 2010; 46(4):342-350.. In the aforementioned Chinese study, the questionnaire was filled in by mail, which resulted in great loss of data⁹9 Bola J, Chan THC, Chen EH, Ng R. Cross-validating chinese language mental health recovery measures in Hong Kong. Res Soc Work Pract 2016; 26(6):630-640..

However, we observed other strategies adopted to support participants in completing the questionnaire - specifically in the RSA Person in Recovery version - such as the research team reading the questions to illiterate users in China¹⁷17 Ye S, Pan J, Wong DFK, Bola JR. Cross-validation of mental health recovery measures in a Hong Kong chinese sample. Res Soc Work Pract 2013; 23(3):1177-1187.. In addition, we observed the possibility of answering the questions through face-to-face or telephone interviews with researchers in the United Kingdom¹²12 Leamy M, Clarke E, Boutillier C Le, Bird V, Choudhury R, Macpherson R, Pesola F, Sabas K, Williams P, Slade M. Recovery practice in community mental health teams: national survey. Br J Psychiatry 2016; 209(4):340-346.^,1515 Simpson A, Hannigan B, Coffey M, Barlow S, Cohen R, Jones A, Všetečková, J, Faulkner A, Thornton A, Cartwright M. Recovery-focused care planning and coordination in England and Wales: a cross-national mixed methods comparative case study. BCM Psychiatry 2016; 16:147.. Also, we verified the hiring of providers from the peer support program, for the recruitment and provision of support to users, who helped users to fill in the questionnaire in Sweden and Canada¹¹11 Kidd SA, George L, O'Connell M, Sylvestre J, Kirkpatrick H, Browne G, Thabane L. Fidelity and recovery-orientation in assertive community treatment. Community Ment Health J 2010; 46(4):342-350.^,1414 Rosenberg D, Svedberg P, Schon U-K. Establishing a recovery orientation in mental health services: evaluating the recovery self-assessment (rsa) in a Swedish context. Psychiatr Rehabil J 2015; 38(4):328-335.. These strategies resulted in positive outcomes, mainly in Sweden, since users reported they felt safe with the peer support work and appreciated the received support¹⁴14 Rosenberg D, Svedberg P, Schon U-K. Establishing a recovery orientation in mental health services: evaluating the recovery self-assessment (rsa) in a Swedish context. Psychiatr Rehabil J 2015; 38(4):328-335..

In the original RSA-R questionnaire, item 21 refers to the existence of peer support programs in mental health services. However, such programs have not yet been implemented in Brazil and, therefore, throughout the process of cross-cultural adaptation, this was a challenging topic. After the evaluation of experts and family members, we decided to keep it, considering the challenge of implementing practices that directly include persons in recovery and their families. This discussion reinforces the importance of using instruments internationally developed, which not only evaluate the currently available practices and strategies, but also those that mobilize the creation of new recovery-oriented strategies, knowledge, and projects in mental health services and that enable an international dialogue.

According to our investigation, we can state that applying the RSA-R Family/Brazil in the form of an interview is a valid strategy. Furthermore, this strategy has already been employed in other study whose authors adapted another mental health scale for users and family members in Brazil³⁵35 Bandeira M, Calzavara MGP, Costa CS, Cesari L. Avaliação de serviços de saúde mental: adaptação transcultural de uma medida da percepção dos usuários sobre os resultados do tratamento. J Bras Psiquiatr 2009; 58(2):107-114.^,3636 Bandeira M, Felicio CM, Cesari L. Validation of the perception of change scale - family version (emp-f) as a treatment outcome measure in mental health services. Rev Bras Psiquiatr 2010; 32(3):283-287.. Nevertheless, this does not prevent researchers from thinking about new strategies in the future - such as the implementation of peer support programs, both for users and family members -, since there are several studies whose authors argue that the implementation of peer programs (and their support in research) generates positive results for users and family members, who usually feel less embarrassed when responding to a peer rather than to a professional³⁴34 Lodge AC, Kuhn W, Earley J, Manser SS. Initial development of the recovery-oriented services assessment: a collaboration with peer-provider consultants. Psychiatr Rehabil J 2018; 41(2):92-102..

The second factor that made the adaptation of the RSA-R Family to the Brazilian context challenging was the number of items and the language used in the original version, which requires great literacy skills. For the Brazilian version, we needed to reduce the extent of the items, removing examples and adapting the items to a simpler language, with the assistance of a popular educator. This factor was acknowledged in a literature review⁶6 Williams J, Leamy M, Bird V, Harding C, Larsen J, Le Boutillier C, Oades L, Slade M. Measures of the recovery orientation of mental health services : systematic review. Soc Psychiatry Psychiatr Epidemiol 2012; 47(11):1827-1835. and in previous studies carried out in Canada and in the United States¹¹11 Kidd SA, George L, O'Connell M, Sylvestre J, Kirkpatrick H, Browne G, Thabane L. Fidelity and recovery-orientation in assertive community treatment. Community Ment Health J 2010; 46(4):342-350.^,3434 Lodge AC, Kuhn W, Earley J, Manser SS. Initial development of the recovery-oriented services assessment: a collaboration with peer-provider consultants. Psychiatr Rehabil J 2018; 41(2):92-102.. Authors of such studies recommended that, in future attempts to develop new items for the RSA, one must consider the formulation of simple and clear questions concerning part of the construct, which may not only improve the structure of the scale, but also facilitate the interpretation of future outcomes on the part of participants, researchers, and clinicians⁸8 Barbic SP, Kidd SA, Davidson L, Connell MJO. Validation of the brief version of the recovery self-assessment (rsa-b) using rasch measurement theory. Psychiatr Rehabil J 2015; 38(4):349-358.. In this sense, the Brazilian version of the RSA-R Family would be the precursor in enabling the scale in a more accessible language, broadening comparison possibilities with other countries with similar social standards.

RSA-R, in all its versions, has a five-point Likert-type scale for measuring agreement. Adapting this scale of responses was the third challenging factor in the Brazilian version of the RSA-R Family. To do so, we needed to write simpler words in each number of the scale and insert graphic elements. Although we did not verify similar experiences in other countries, in Sweden, for instance, the Likert scale was repeated as the header on each new page of the questionnaire¹⁴14 Rosenberg D, Svedberg P, Schon U-K. Establishing a recovery orientation in mental health services: evaluating the recovery self-assessment (rsa) in a Swedish context. Psychiatr Rehabil J 2015; 38(4):328-335.. In a study conducted in the United States, the authors created an instrument based on the RSA, with the alteration to a five-point Likert-type scale for measuring frequency³⁴34 Lodge AC, Kuhn W, Earley J, Manser SS. Initial development of the recovery-oriented services assessment: a collaboration with peer-provider consultants. Psychiatr Rehabil J 2018; 41(2):92-102.. We believe that changes made in the Brazilian version of the RSA-R Family will help family members in filling in the questionnaire and in its applicability, which will allow future studies to provide psychometric evidence of variability and distribution between responses. Nevertheless, we do not disregard the possibility of adapting, in the Brazilian version of the RSA-R Family, the frequency scale for future questionnaires or to reduce the number of response categories. According to the study on the brief version of the RSA, the reduced number of categories may be more favorable for mental health users, as well as for the general population, and encourages further research for testing the RSA with fewer categories of responses to improve the valid recovery-oriented measurement potential in people with mental illness⁸8 Barbic SP, Kidd SA, Davidson L, Connell MJO. Validation of the brief version of the recovery self-assessment (rsa-b) using rasch measurement theory. Psychiatr Rehabil J 2015; 38(4):349-358..

The adaptation of the recovery concept was the last challenging factor; it was inserted in most items as “recuperação” after consensus in the FG with family members. As reported in the literature, it was difficult to find an equivalent term in Brazilian Portuguese. Overall, the term “recuperação” (Portuguese term for “recovery”) is mistaken for “cura” (Portuguese term for “cure”). As well as researchers from other countries (China, Germany, and Sweden) who have adapted this scale, we believe that recovery, as a paradigm of mental health care, may broaden the spectrum of intervention and stimulate the creation of activities and practices that enable us to overcome the biomedical model, enhancing the autonomy of users and their family members in activities, such as job creation, educational and cultural access, and the implementation of strategies such as peer support programs⁹9 Bola J, Chan THC, Chen EH, Ng R. Cross-validating chinese language mental health recovery measures in Hong Kong. Res Soc Work Pract 2016; 26(6):630-640.^,1010 Burgess P, Pirkis J, Coombs T, Rosen A. Assessing the value of existing recovery measures for routine use in Australian mental health services. Aust N Z J Psychiatry 2011; 45(4):267-280.^,1414 Rosenberg D, Svedberg P, Schon U-K. Establishing a recovery orientation in mental health services: evaluating the recovery self-assessment (rsa) in a Swedish context. Psychiatr Rehabil J 2015; 38(4):328-335.^,1818 Zuaboni G, Kieser LD, Glavanovits K, Utschakowski J, Behrens J. Recovery Self Assessment: Translation and cultural adaption of a recovery oriented assessment instrument. Pflege 2015; 28(4):233-243..

Although in this study we sought to adapt the RSA-R Family instrument to the Brazilian context, we brought to light elements of discussion about the cultural differences in Brazil and the applicability possibilities of the construct, when compared with other countries. In addition, we sought to maintain, in this version, the highest level of equivalence with the original questionnaire, seeking to respect, at all steps, the culture to which we were adapting it. Therefore, it is worth stating that, in order to demonstrate evidence of the reliability and validity of this scale in the present context, future psychometric studies shall be necessary.

Acknowledgments

The authors thank Espaço da Escrita - Pró-Reitoria de Pesquisa (UNICAMP) - for the language services provided.

References

Publication Dates

History