ABSTRACT
OBJECTIVE:
To adapt the Leisure Attitude Measurement to the Brazilian culture and to evaluate the face and content validity of the Brazilian version for older population.
METHODS:
Methodological study of cross-cultural adaptation in five stages: initial translation; synthesis of translations; back translation; evaluation by a committee of experts using a face and content validity assessment instrument; pre-test with 36 elderly, selected by convenience, with the application of a pre-test evaluation instrument. Data were analyzed descriptively and internal consistency measured by Cronbach's alpha coefficient.
RESULTS:
Evidenced face and content validity of the adapted version, as well as its equivalence with the original version. In the pretest, the elderly were 71.5 years old on average, 66.7% were women, 47.2% had a stable union, 66.7% lived with family members, 47.2% had 12 or more years of education and 58.3% received two minimum wages or more. The instrument revealed good internal consistency with a coefficient of 0.95 for the total global instrument and 0.88, 0.92 and 0.88 for the cognitive, affective and behavioral domains, respectively.
CONCLUSIONS:
The instrument's adaptation to the Brazilian culture was successful and allows to assess the attitude of the elderly in relation to leisure in a reliable manner, even though the results are a preliminary version, to be concluded after the psychometric analysis. The instrument could be incorporated in various health fields in Brazil and will allow the production of standardized data, comparison between cultures and strategies to promote positive attitudes towards leisure.
DESCRIPTORS:
Aged; Leisure Activities; Attitude to Health; Validation Study
INTRODUCTION
Older adults constitute a rapidly and substantially growing segment of the population worldwide, with estimates that draw attention by pointing to a total of 1.2 billion people aged 60 and over in 2025 11. Afshar PF, Foroughan M, Vedadhir A, Tabatabaie MG. Psychometric properties of the Persian version of Social Adaptation Self-evaluation Scale in community-dwelling older adults. Clin Interv Aging. 2017;12:579-84. https://doi.org/10.2147/CIA.S129407
https://doi.org/10.2147/CIA.S129407... . If, on the one hand, these data show an opportunity for advancement in terms of quality and life expectancy, on the other hand, they create uncertainties regarding the social, financial and health maintenance of this age group 22. Gvozd R, Rossaneis MA, Pissinati PSC, Guirardello EB, Haddad MCFL. Cultural adaptation of the Retirement Resources Inventory for Brazilian culture. Rev Saude Publica. 2019;53:60. https://doi.org/10.11606/s1518-8787.2019053000863
https://doi.org/10.11606/s1518-8787.2019... . Because this phase is usually associated with the exit of the individual from the labor market, it is marked by physical, psychological, social and mental changes, which may lead to the loss of personal identity and interfere with quality of life 33. Liberatti VM, Martins JT, Ribeiro RP, Scholze AL, Galdino MJQ, Trevisan GS. Qualidade de vida na concepção de docentes de enfermagem aposentadas por uma universidade pública. Cienc Cuid Saude. 2016;15(4):655-61. https://doi.org/10.4025/cienccuidsaude.v15i4.29968
https://doi.org/10.4025/cienccuidsaude.v... ,44. Lee C, Payne LL, Berdychevsky L. The roles of leisure attitudes and self-efficacy on attitudes toward retirement among retirees: a sense of coherence theory approach. Leis Sci. 2020;42(2):158-69. https://doi.org/10.1080/01490400.2018.1448025
https://doi.org/10.1080/01490400.2018.14... .
It is critical for the successful aging to keep an active mind and body 55. Lambrini K, Konstantinos K, Christos I, Areti T. Leisure activity in the third age. J Healthc Commun. 2018;3:12. https://doi.org/10.4172/2472-1654.100123
https://doi.org/10.4172/2472-1654.100123... , whether by investing in engaging in occupational and social activities or autonomy and well-being 22. Gvozd R, Rossaneis MA, Pissinati PSC, Guirardello EB, Haddad MCFL. Cultural adaptation of the Retirement Resources Inventory for Brazilian culture. Rev Saude Publica. 2019;53:60. https://doi.org/10.11606/s1518-8787.2019053000863
https://doi.org/10.11606/s1518-8787.2019... . In this sense, the literature deals with the benefits that leisure represents as a strategy to cope with the challenges, losses and disabilities in old age 66. Ferrari RFR, Baldissera VDA, Lange C, Carreira L, Silva ES. Attitude toward leisure among the community-dwelling elderly: an interface for health promotion. Texto Contexto Enferm. 2016;25(4):e1280015. https://doi.org/10.1590/0104-07072016001280015
https://doi.org/10.1590/0104-07072016001... ,77. Kao IC, Chang LC. Long-term effects of leisure education on leisure needs and stress in older adults. Educ Gerontol. 2017;43(7):356-64. https://doi.org/10.1080/03601277.2017.1299447
https://doi.org/10.1080/03601277.2017.12... . Here, leisure is understood as a set of experiences that can develop in many cultural contents and which expresses itself in the time available beyond professional obligations, in a free and authentic way, and the individual can actually enjoy idleness 88. Puke N, Marcellino NC. Possibilidades de interface entre lazer e fenomenologia. Movimento. 2014;20:307-27. https://doi.org/10.22456/1982-8918.40321
https://doi.org/10.22456/1982-8918.40321... .
In the health field, involvement in leisure activities promotes physical health, positive social interactions, psychological well-being and quality of life 55. Lambrini K, Konstantinos K, Christos I, Areti T. Leisure activity in the third age. J Healthc Commun. 2018;3:12. https://doi.org/10.4172/2472-1654.100123
https://doi.org/10.4172/2472-1654.100123... ,77. Kao IC, Chang LC. Long-term effects of leisure education on leisure needs and stress in older adults. Educ Gerontol. 2017;43(7):356-64. https://doi.org/10.1080/03601277.2017.1299447
https://doi.org/10.1080/03601277.2017.12... ,99. Nimrod G, Shrira A. The paradox of leisure in later life. J Gerontol B Psychol Sci Soc Sci. 2016;71(1):106-11. https://doi.org/10.1093/geronb/gbu143
https://doi.org/10.1093/geronb/gbu143... . Thus, leisure can be considered as a conditioner and determinant of the health-disease process, especially regarding mental health 1010. Belo P, Navarro-Pardo E, Pocinho R, Carrana P, Margarido C. Relationship between mental health and the education level in elderly people: mediation of leisure attitude. Front Psychol. 2020;11:573. https://doi.org/10.3389/fpsyg.2020.00573
https://doi.org/10.3389/fpsyg.2020.00573... . In addition, involvement in leisure activities is a predictor of health self-perception and functional capacity in the older adults, and some leisure activities are related to health-promoting behavior 1111. Hicks SA, Siedlecki KL. Leisure activity engagement and positive affect partially mediate the relationship between positive views on aging and physical health. J Gerontol B Psychol Sci Soc Sci. 2017;72(2):259-67. https://doi.org/10.1093/geronb/gbw049
https://doi.org/10.1093/geronb/gbw049... .
People's practices during leisure says more about their attitude than in any other context 44. Lee C, Payne LL, Berdychevsky L. The roles of leisure attitudes and self-efficacy on attitudes toward retirement among retirees: a sense of coherence theory approach. Leis Sci. 2020;42(2):158-69. https://doi.org/10.1080/01490400.2018.1448025
https://doi.org/10.1080/01490400.2018.14... ,1010. Belo P, Navarro-Pardo E, Pocinho R, Carrana P, Margarido C. Relationship between mental health and the education level in elderly people: mediation of leisure attitude. Front Psychol. 2020;11:573. https://doi.org/10.3389/fpsyg.2020.00573
https://doi.org/10.3389/fpsyg.2020.00573... ,1212. Adams EV, Taniguchi ST, Hite SJ, Ward PJ, Mugimu CB, Nsubuga YK. Leisure defined by perceiving choice: perspectives of Ugandan women. World Leis J. 2018;60(4):265-80. https://doi.org/10.1080/16078055.2018.1517106
https://doi.org/10.1080/16078055.2018.15... . Therefore, it is relevant to know the attitude of the older people in relation to leisure in order to allow a better understanding of this type of social practice and stimulate more positive attitudes and engagement in leisure activities 66. Ferrari RFR, Baldissera VDA, Lange C, Carreira L, Silva ES. Attitude toward leisure among the community-dwelling elderly: an interface for health promotion. Texto Contexto Enferm. 2016;25(4):e1280015. https://doi.org/10.1590/0104-07072016001280015
https://doi.org/10.1590/0104-07072016001... ,1313. Castro VC, Carreira L. Leisure activities and attitude of institutionalized elderly people: a basis for nursing practice. Rev Lat Am Enfermagem. 2015;23(2):307-14. https://doi.org/10.1590/0104-1169.3650.2556
https://doi.org/10.1590/0104-1169.3650.2... .
Based on the concepts of social psychology, attitude is a system of knowledge and beliefs, provided with feelings, for or against a particular social object, that generates the intention of attitude on the knowledge and feelings related to that object. Thereby, it is a multidimensional construct that aggregates cognitive, affective and behavioral components, developed from individual characteristics, learning processes, environmental influences and previous experiences 1414. Munusturlar S. Examination of the leisure attitudes of individuals living in Turkey from an east to west perspective. Balt J Sport Health Sci. 2017;2(105):47-57. https://doi.org/10.33607/bjshs.v2i105.25
https://doi.org/10.33607/bjshs.v2i105.25... . This way, the person's attitude can change over time, while acquiring new experiences and assimilating new knowledge, as happens during the aging process 1313. Castro VC, Carreira L. Leisure activities and attitude of institutionalized elderly people: a basis for nursing practice. Rev Lat Am Enfermagem. 2015;23(2):307-14. https://doi.org/10.1590/0104-1169.3650.2556
https://doi.org/10.1590/0104-1169.3650.2... .
To assess the attitude of the older people in relation to leisure, it is essential to use reliable instruments whose results allow the elaboration of health strategies to be appropriate to reality. Leisure Attitude Measurement (LAM) is one of these instruments and aims to measure cognitive, affective and behavioral leisure-related attitude 1515. Ragheb MG, Beard JG. Measuring leisure attitude. J Leis Res. 1982;14(2):155-67. https://doi.org/10.1080/00222216.1982.11969512
https://doi.org/10.1080/00222216.1982.11... . It has been adapted and validated for other countries such as China 1616. Deng J, Walker GJ, Swinnerton G. Leisure attitudes: a comparison between Chinese in Canada and Anglo-Canadians. Leisure (Waterloo, Ont). 2005;29(2):239-73. https://doi.org/10.1080/14927713.2005.9651331
https://doi.org/10.1080/14927713.2005.96... ,1717. Walker GJ, Deng J, Chapman R. Leisure Attitudes: a follow-up study comparing Canadians, Chinese in Canada, and Mainland Chinese. World Leis J. 2007;49(4):207-15. https://doi.org/10.1080/04419057.2007.9674513
https://doi.org/10.1080/04419057.2007.96... , Malásia 1818. Chiu LK, Kayat K. Psychological determinants of leisure time physical activity participation among public university students in Malaysia. AJTLHE. 2010;2(2):35-45. , Portugal 1919. Freire T, Fonte C. Escala de atitudes face ao lazer em adolescentes e jovens adultos. Paideia (Ribeirão Preto). 2007;17(36):79-87. https://doi.org/10.1590/S0103-863X2007000100008
https://doi.org/10.1590/S0103-863X200700... ,2020. Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
https://doi.org/10.1080/14927713.2013.77... and Turkey 2121. Akgül BM, Gürbüz B. Bo Zaman Tutum Ölçe i: geçerlik ve güvenirlik çali masi. Gazi Beden E itimi ve Spor Bilimleri Dergisi. 2011 [cited 2020 May 7];16(1):37-43. Available from: https://dergipark.org.tr/tr/pub/gbesbd/issue/28050/305073
https://dergipark.org.tr/tr/pub/gbesbd/i... .
Leisure is not experienced in the same way by all individuals; after all, there is no single appropriate approach to explain the social structures and values related leisure in other countries and cultures 1212. Adams EV, Taniguchi ST, Hite SJ, Ward PJ, Mugimu CB, Nsubuga YK. Leisure defined by perceiving choice: perspectives of Ugandan women. World Leis J. 2018;60(4):265-80. https://doi.org/10.1080/16078055.2018.1517106
https://doi.org/10.1080/16078055.2018.15... . Given this, cross-cultural adaptation emerges as an alternative to the development of a new instrument because it requires less resources and favors the comparison between different contexts and populations 2222. Lino CRM, Brüggemann OM, Souza ML, Barbosa SFF, Santos EKA. The cross-cultural adaptation of research instruments, conducted by nurses in Brazil: an integrative review. Texto Contexto Enferm. 2017;26(4):e1730017. https://doi.org/10.1590/0104-07072017001730017
https://doi.org/10.1590/0104-07072017001... .
Considering that cross-cultural adaptation procedures have increasing emphasis as an instrument for the development of public health practice and science 2222. Lino CRM, Brüggemann OM, Souza ML, Barbosa SFF, Santos EKA. The cross-cultural adaptation of research instruments, conducted by nurses in Brazil: an integrative review. Texto Contexto Enferm. 2017;26(4):e1730017. https://doi.org/10.1590/0104-07072017001730017
https://doi.org/10.1590/0104-07072017001... ,2323. Machado RS, Fernandes ADBF, Oliveira ALCB, Soares LS, Gouveia MTO, Silva GRF. Cross-cultural adaptation methods of instruments in the nursing area. Rev Gaucha Enferm. 2018;39:e2017-0164. https://doi.org/10.1590/1983-1447.2018.2017-0164
https://doi.org/10.1590/1983-1447.2018.2... , the aim of this study was to adapt the Leisure Attitude Measurement to the Brazilian culture and assess the face and content validity of the Brazilian version for older adults.
METHODS
This is a methodological study of cross-cultural adaptation. Authorization was obtained from one of the authors of the instrument, as well as from the copyright holder, through e-mail, for cross-cultural adaptation. The translation and adaptation process of the instrument was outlined in five steps ( Figure ): 1) Initial translation into the target language; 2) Synthesis of translations; 3) Back-translation to the original language; 4) Evaluation by a committee of experts; 5) Pre-test 2424. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Recommendations for the cross-cultural adaptation of the DASH & QuickDASH Outcome Measures. Institute for Work & Health; 2007 [cited 2019 Jan 30]. Available from: http://dash.iwh.on.ca/sites/dash/files/downloads/cross_cultural_adaptation_2007.pdf
http://dash.iwh.on.ca/sites/dash/files/d... .
The original version of LAM consists of 36 items, divided into three domains related to attitude topics, with 12 items each. A five-level Likert scale is used as a response system (“1” = extremely negative attitude, “5” = extremely positive attitude, “3” = midpoint) and scores are obtained by arithmetically adding the responses to the items. For the separate domains, the midpoint is 36 and for the global instrument is 108. Values above this point reveal positive attitudes and, below, negative attitudes towards leisure 2020. Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
https://doi.org/10.1080/14927713.2013.77... .
Two translations (T1 and T2) of the LAM into Portuguese were performed by two independent translators, both Brazilians and mastery of American culture and English language, and only one of them was informed about the objectives of the instrument and the study. The two translated versions (T1 and T2) were created into a synthesis version (T12), based on consensus between the translators, through a single virtual meeting lasting approximately forty minutes. On that occasion, the discussion about the differences, changes and comments regarding the translations was mediated by the researcher. In the third stage, T12 was back-translated (RT1 and RT2) by two American translators, with mastery of the Portuguese language and Brazilian culture, in which both were unaware of the objectives of the study. The Brazilian and American translators were selected by a company specializing in certified translations. The three stages took place between August 2017 and February 2018.
For the committee of experts, were invited four bilingual (Portuguese/English) health professionals and an English teacher, who was trained and specialized in the health area, working as an English teacher in regular and language schools. Of the four healthcare professionals, one is a PhD in Physical Education, a professor with extensive experience in assessing the psychometric properties of measuring instruments; three are nurses, two PhDs and one PhD student in Nursing, all professors, one with experience in evaluating the psychometric properties of measuring instruments and the health of the older population, and the other with experience in quantitative research methodologies and experiences in older adults health. After accepting to participate in the study, the experts received all the material by email, which included the two translations, the synthesis version and the two back-translations, as well as an instrument developed by the researcher to perform face and content validation of LAM. The committee review process took place between March and May 2018.
To assess semantic, idiomatic, conceptual and cultural equivalence between the original and the adapted instrument, the specialists received guidelines to fill out the instrument for evaluating the face and content validity. According to the adopted protocol 1919. Freire T, Fonte C. Escala de atitudes face ao lazer em adolescentes e jovens adultos. Paideia (Ribeirão Preto). 2007;17(36):79-87. https://doi.org/10.1590/S0103-863X2007000100008
https://doi.org/10.1590/S0103-863X200700... , in semantic equivalence, the meaning of words was assessed through vocabulary and grammar; in idiomatic equivalence, the colloquial expressions used were analyzed; in conceptual equivalence, it was about the importance of the items and aspects that were addressed; and, in cultural equivalence, it was observed whether the items captured and measured the daily life experiences of the target audience. Each specialist analyzed one item at a time, compared to its original version, and indicated whether it was equivalent, partially equivalent or not equivalent between the two versions, using different fields for semantic, idiomatic, conceptual and cultural aspects. In cases of partial equivalence or non-equivalence between items, experts were instructed to suggest adjustments.
The Content Validity Index (CVI) of each item was also assessed by applying a four-point likert scale, in which experts scored “1” if not relevant, “2” if item needed major revision to be relevant, “3” if item needed minor revision to be relevant or “4” if relevant item. The score was calculated by the sum of agreement of the items marked by “three” or “four” by the experts, divided by the total number of answers, being considered appropriate those items with agreement rate higher than 80% 2525. Cunha CM, Almeida Neto OPA, Stackfleth R. Principais métodos de avaliação psicométrica da validade de instrumentos de medida. Rev Aten Saude. 2016;14(47):75-83. https://doi.org/10.13037/ras.vol14n47.3391
https://doi.org/10.13037/ras.vol14n47.33... .
The pre-test version of Brazilian LAM was based on the expert's suggestions and, one more time, sent to them for final opinion. Once approved, a pre-test was carried out with 36 seniors who attended the University of the Third Age (UnATI), a project from the State University of Maringá. Data collection took place in two weekdays, when there were classes, in June 2018, and the sample was selected by convenience. After reading and answering to the pre-test version of the Brazilian LAM, the older people filled out a pre-test evaluation instrument, developed by the researcher, in order to assess the clarity and understanding of the items. This instrument had six questions, a space for suggestions and variables for participant's sociodemographic characterization, namely: age, gender, educational level, income and family arrangement.
Data were described using simple frequency tables. The internal consistency of the instrument for the 36 items analyzed was measured using Cronbach's alpha coefficient. Cronbach's alpha with values above 0.7 was considered ideal. Data were analyzed using the Statistical Analysis Software Program (SAS, version 9.4) from a spreadsheet created in the Excel program.
Ethical approval for the study was granted by the Human Research Ethics Committee of the State University of Maringa (COPEP/UEM) (protocol 2,194,308/2017).
RESULTS
Of the 36 items translated in the first stage, 18 (50%) showed some discrepancy in T1 and T2, the main ones being related to similar words or phrases in Brazilian Portuguese (eg “quite” and “considerable”; “some” and “way”) and use of first or third person pronouns (“make us” and “make the person”). The other half of the items were identical in the translations, with six belonging to the cognitive domain, seven to the affective domain and five to the behavioral domain.
Inthe synthesis of the translations, discrepancies with the translators were resolved, opting for the expressions or words that provided the item with greater clarity. It was also considered, regarding the older people community in the Brazilian context, to discuss the exchange of some words from items that were identical in T1 and T2 for more popular terms in Brazilian Portuguese, without modifying the meaning expressed in the item, for better the understanding of respondents, namely: “self-development” exchanged to “personal development”; “waste” exchanged to “spend”; “had time available” exchanged “had time to spend”. In the back-translated versions, 27 items (75%) were identical, and the words that differed between RT1 and RT2 in the remaining nine items were considered synonymous, showing that the T12 version corresponded to the original version.
In the fourth stage, the experts evaluated the equivalence of the items, the statement and the measurement scale of the Brazilian version of LAM (LAM-BV), as well as the relevance of each item, calculated using the CVI. The CVI was excellent (1.0) for 31 items and acceptable (0.8) for five items, requiring adjustments. The agreementpercentage of the experts was 86.2%.
A total of 69 suggestions were made by the five experts, which considered 16 (44.5%) items equivalent to the original version in all aspects. Another 20 (55.5%) items were partially equivalent, and of these, eight received this rating from at least two experts and two items were concomitantly assessed as non-equivalent, in at least one aspect by one of the experts. Table 1 pointed out these 20 items and their respective CVI, totaling 23 partial equivalences in semantic, 11 in idiomatic, nine in conceptual and ten in cultural, and two non-equivalences in idiomatic and one in semantic, conceptual and cultural aspects.
Partial equivalences and non-equivalences between items of the original and adapted versions of LAM, in the semantic, idiomatic, conceptual and cultural aspects, according to the experts and their respective CVI. Maringá, Paraná, Brazil, 2018.
The main suggestions for the partially equivalent or non-equivalent items were to remove the word “time” from the expression “leisure time” in items 3 and 16; add the word “individual” or “person” in items 4, 6 and 7; exchange “development” by the word “improvement” in item 8; use “get involved” instead of “participate” in items 13, 19 and 34, considering the legitimate translation of the original version and the deeper meaning of “get involved” in the Brazilian context; to replace the verb“spend”by using“I dedicate” in items 22 and 35, since “spending” may have a negative meaning; to exchange the expression “I do not like to be in a hurry” to “I like to enjoy every moment” in item 19; replace “environment” with “place” in item 30, as it is the most used expression in the Brazilian context. Box shows the original and final versions after the cross-cultural adaptation process.
Translation and cross-cultural adaptation of Leisure Attitude Measurement to Brazilian Portuguese.
In the items considered equivalent, the experts also made suggestions, such as modifying the expression “make good use” to “a smart way to use” in item 1, as it helps understanding and is authentic to the original version; replacing the expression “time flies” with “time goes by fast” in item 13, avoiding the use of figures of speechin an academic instrument; remove the word “absorb” in item 24, keeping only “receive” for better understanding; replace “I do” with “ I perform” in items 25, 31 and 32; change the expression “when I am busy” to “when I have little available time” in item 34, since this interpretation of “busy” in the Brazilian context refers to the availability of time.
In the pre-test with 36 older people from UnATI, all instruments were self-applied and the average time to complete them was approximately eight minutes, with a minimum of 4 and a maximum of 20 minutes. Regarding the sociodemographic characteristics of the participants, 24 (66.7%) were women; aged between 60 and 81 years old, with an average of 71.5 years (SD=5.17); 17 (47.2%) were married or in a domestic partnership; 24 (66.7%) lived with family members; 17 (47.2%) had 12 years of education or more and 13 (36%) had 4 to 11 years of education; and 21 (58.3%) had an income of two minimum wages or more (minimum wage = R$ 954.00).
Regarding the older adults' evaluation about the instrument, all respondents considered that the directions and the answer options were easy to understand, 33 (91.7%) reported understanding of all items, 35 (97.2%) considered all items important and six older people suggested improvements for the instrument. Three participants reported difficulty in understanding items 16, 27, 30, 32, 33 and 35, but when approaching them individually, items 30, 32, 33 and 35 were correctly interpreted by these elderlies and assessed themas appropriate. Item 16 was adapted, replacing the phrase “I can be myself” with “I feel comfortable with myself”, as well as item 27, in which the sentence “as my income allows” was placed in the beginning of the sentence.
In addition, one participant requested examples of leisure in the directions, which was not possible to meet, since the definition of leisure is subjective for each individual; and one respondent asked if it was wrong to have chosen the same answer for all items, which led to the addition of the phrase “There are no right or wrong answers” in the statement.
When evaluating the 36 items, the LAM-BV showed an adequate internal consistency, with a Cronbach's alpha coefficient (α) of 0.95. The individual analysis of the components resulted in α=0.88, 0.92 and 0.88 for the cognitive, affective and behavioral domains, respectively. Table 2 shows the correlation between items and the internal consistency of each domain.
Correlation between items and α by domain of LAM if the item is excluded, according to a pretest with the older population. Maringá, Paraná, Brazil, 2018 (n = 36).
It was observed that item 1 had the lowest (r = 0.35) correlation between items in the cognitive domain, and item 27 the lowest (r = 0.34) correlation between items in the behavioral domain. However, when excluding these items, the gain in reliability is little, as the α coefficient for each of the domains goes from 0.88 to 0.89. In the affective domain, however, no item showed a correlation between items less than 0.5. Therefore, in both situations, it was decided to keep all items of the instrument.
DISCUSSION
This research introduces to the Brazilian context the possibility of evaluating the older population's attitude towards leisure, using a reliable instrument, even though the results deal with a preliminary version. Evidence on attitude and leisure can contribute to the analysis of the socio-cognitive processes associated with involvement in leisure, which allows the development of interventions to promote positive attitudes in this sense 2020. Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
https://doi.org/10.1080/14927713.2013.77... .
LAM-BV showed satisfactory equivalences and adequate CVI compared to the original version. The content validation of the instrument in the aspects presented is valued as it associates qualitative strategies, by assessing equivalence and quantitative, as well as by calculating the CVI 2222. Lino CRM, Brüggemann OM, Souza ML, Barbosa SFF, Santos EKA. The cross-cultural adaptation of research instruments, conducted by nurses in Brazil: an integrative review. Texto Contexto Enferm. 2017;26(4):e1730017. https://doi.org/10.1590/0104-07072017001730017
https://doi.org/10.1590/0104-07072017001... .
The LAM-BV translation and cross-cultural adaptation process were successful after the necessary grammatical and cultural adjustments. The two Chinese versions of LAM, the first reduced, with 24 items 1616. Deng J, Walker GJ, Swinnerton G. Leisure attitudes: a comparison between Chinese in Canada and Anglo-Canadians. Leisure (Waterloo, Ont). 2005;29(2):239-73. https://doi.org/10.1080/14927713.2005.9651331
https://doi.org/10.1080/14927713.2005.96... and the second following the same standards as the original version 1717. Walker GJ, Deng J, Chapman R. Leisure Attitudes: a follow-up study comparing Canadians, Chinese in Canada, and Mainland Chinese. World Leis J. 2007;49(4):207-15. https://doi.org/10.1080/04419057.2007.9674513
https://doi.org/10.1080/04419057.2007.96... , were translated separately by two bilingual Chinese scholars, who later discussed the adequacy of the terms of the two versions. These were reviewed by a third party, a Chinese scholar and, finally, the three scholars jointly developed the final version of the instrument. In the Turkish version 2121. Akgül BM, Gürbüz B. Bo Zaman Tutum Ölçe i: geçerlik ve güvenirlik çali masi. Gazi Beden E itimi ve Spor Bilimleri Dergisi. 2011 [cited 2020 May 7];16(1):37-43. Available from: https://dergipark.org.tr/tr/pub/gbesbd/issue/28050/305073
https://dergipark.org.tr/tr/pub/gbesbd/i... , LAM kept the same structure as the original version and was translated by two bilingual researchers in the field of physical education and sports and back-translated into English by a specialist.
The adaptation process is broad and does not consist of a simple translation, therefore, the expert committee must be made up of professionals with enough training, qualification and experience to make judgments and make the right decisions, considering the responsibility to create the pre-final version of the instrument 2222. Lino CRM, Brüggemann OM, Souza ML, Barbosa SFF, Santos EKA. The cross-cultural adaptation of research instruments, conducted by nurses in Brazil: an integrative review. Texto Contexto Enferm. 2017;26(4):e1730017. https://doi.org/10.1590/0104-07072017001730017
https://doi.org/10.1590/0104-07072017001... ,2525. Cunha CM, Almeida Neto OPA, Stackfleth R. Principais métodos de avaliação psicométrica da validade de instrumentos de medida. Rev Aten Saude. 2016;14(47):75-83. https://doi.org/10.13037/ras.vol14n47.3391
https://doi.org/10.13037/ras.vol14n47.33... . The original version of LAM was evaluated by 31 specialists in the field of leisure and social psychology, and it was considered acceptable 1515. Ragheb MG, Beard JG. Measuring leisure attitude. J Leis Res. 1982;14(2):155-67. https://doi.org/10.1080/00222216.1982.11969512
https://doi.org/10.1080/00222216.1982.11... ,2020. Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
https://doi.org/10.1080/14927713.2013.77... .
The conception of the final version is achieved through the pre-test, when it is possible to assess the clarity of the adapted instrument, identify and review unsuitable terms that may cause confusion 2626. Ortiz-Gutiérrez S, Cruz-Avelar A. Translation and cross-cultural adaptation of health assessment tools. Actas Dermosifiliogr. 2018;109(3):202-6. https://doi.org/10.1016/j.adengl.2018.02.003
https://doi.org/10.1016/j.adengl.2018.02... . In the validation study of the LAM version for use by Malaysian university students, the authors referred to a pre-test with 105 students to obtain adequate reliability, however, they pointed out that the deeper meanings of certain items may not have been found accurately in the target language 1818. Chiu LK, Kayat K. Psychological determinants of leisure time physical activity participation among public university students in Malaysia. AJTLHE. 2010;2(2):35-45. .
Despite the current use of LAM in different contexts, most scientific publications in this area are aimed at adolescents and university students. A study carried out with Portuguese people aged 14 to 19 years showed that the attitude related to leisure is a significant predictor of satisfaction in leisure 2727. Freire F, Teixeira A. The influence of leisure attitudes and leisure satisfaction on adolescents' positive functioning: the role of emotion regulation. Front Psychol. 2018;9:1349. https://doi.org/10.3389/fpsyg.2018.01349
https://doi.org/10.3389/fpsyg.2018.01349... . The results of three other recent studies carried out with undergraduate students in South Africa 2828. Muzindutsi PF, Viljoen D. Socio-demographic factors influencing leisure attitude among undergraduate students at a South African university. Int J Soc Sci Human Stud. 2016 [cited 2020 May 07];8(1):103-15. Available from: https://www.semanticscholar.org/paper/SOCIO-DEMOGRAPHIC-FACTORS-INFLUENCING-LEISURE-AMONG-Muzindutsi-Viljoen/46a2fbc10864cdf516a720cd08cb0a64cee809ba
https://www.semanticscholar.org/paper/SO... , Turkey 2929. Murathan T, Türker A, Murathan F. Investigation of university students attitudes toward leisure time activities (Ardahan University sample). Turkish J Sport Exerc. 2019;21(2):162-9. https://doi.org/10.15314/tsed.473228
https://doi.org/10.15314/tsed.473228... and China 3030. Choi S, Yoo Y. Leisure attitude and satisfaction with leisure and life: proposing leisure prioritization and justification. World Leis J. 2016;59(2):140-55. https://doi.org/10.1080/16078055.2016.1216886
https://doi.org/10.1080/16078055.2016.12... showed, respectively, that the attitude related to leisure is influenced by sociodemographic characteristics; engineering and physical education students have a more positive attitude to leisure than students from other undergraduate courses; and the attitude related to leisure reflects two new concepts, namely, prioritization and justification of leisure.
As for the older population, LAM was applied in the south of Brazil, in the version adapted to European Portuguese, with residents of long-term institutions 1313. Castro VC, Carreira L. Leisure activities and attitude of institutionalized elderly people: a basis for nursing practice. Rev Lat Am Enfermagem. 2015;23(2):307-14. https://doi.org/10.1590/0104-1169.3650.2556
https://doi.org/10.1590/0104-1169.3650.2... and older adults in the community 66. Ferrari RFR, Baldissera VDA, Lange C, Carreira L, Silva ES. Attitude toward leisure among the community-dwelling elderly: an interface for health promotion. Texto Contexto Enferm. 2016;25(4):e1280015. https://doi.org/10.1590/0104-07072016001280015
https://doi.org/10.1590/0104-07072016001... . In both studies, a predominantly positive attitude towards leisure was obtained, with the behavioral domain showing the lowest average 1313. Castro VC, Carreira L. Leisure activities and attitude of institutionalized elderly people: a basis for nursing practice. Rev Lat Am Enfermagem. 2015;23(2):307-14. https://doi.org/10.1590/0104-1169.3650.2556
https://doi.org/10.1590/0104-1169.3650.2... ,66. Ferrari RFR, Baldissera VDA, Lange C, Carreira L, Silva ES. Attitude toward leisure among the community-dwelling elderly: an interface for health promotion. Texto Contexto Enferm. 2016;25(4):e1280015. https://doi.org/10.1590/0104-07072016001280015
https://doi.org/10.1590/0104-07072016001... , results similar to the validation studies of LAM in Portugal 1919. Freire T, Fonte C. Escala de atitudes face ao lazer em adolescentes e jovens adultos. Paideia (Ribeirão Preto). 2007;17(36):79-87. https://doi.org/10.1590/S0103-863X2007000100008
https://doi.org/10.1590/S0103-863X200700... ,2020. Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
https://doi.org/10.1080/14927713.2013.77... . The LAM also was applied in a survey conducted with retired Americans, showing that the attitude related to leisure is a resource that influences the way the person deals with retirement 44. Lee C, Payne LL, Berdychevsky L. The roles of leisure attitudes and self-efficacy on attitudes toward retirement among retirees: a sense of coherence theory approach. Leis Sci. 2020;42(2):158-69. https://doi.org/10.1080/01490400.2018.1448025
https://doi.org/10.1080/01490400.2018.14... . A Portuguese study 1010. Belo P, Navarro-Pardo E, Pocinho R, Carrana P, Margarido C. Relationship between mental health and the education level in elderly people: mediation of leisure attitude. Front Psychol. 2020;11:573. https://doi.org/10.3389/fpsyg.2020.00573
https://doi.org/10.3389/fpsyg.2020.00573... that involved the use of LAM revealed that older people with higher levels of education and with a more positive attitude related to leisure had better psychological well-being.
The internal consistency of the LAM-BV was adequate and the findings show the reliability of the original instrument, which has a Cronbach's alpha coefficient of 0.94 for the global instrument and 0.91, 0.93 and 0.89 for the cognitive, affective and behavioral domains, respectively 1515. Ragheb MG, Beard JG. Measuring leisure attitude. J Leis Res. 1982;14(2):155-67. https://doi.org/10.1080/00222216.1982.11969512
https://doi.org/10.1080/00222216.1982.11... . In the other versions adapted for other countries, Cronbach's alpha coefficient ranged from 0.88 to 0.97 for the global instrument 1919. Freire T, Fonte C. Escala de atitudes face ao lazer em adolescentes e jovens adultos. Paideia (Ribeirão Preto). 2007;17(36):79-87. https://doi.org/10.1590/S0103-863X2007000100008
https://doi.org/10.1590/S0103-863X200700... ,2020. Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
https://doi.org/10.1080/14927713.2013.77... ,2121. Akgül BM, Gürbüz B. Bo Zaman Tutum Ölçe i: geçerlik ve güvenirlik çali masi. Gazi Beden E itimi ve Spor Bilimleri Dergisi. 2011 [cited 2020 May 7];16(1):37-43. Available from: https://dergipark.org.tr/tr/pub/gbesbd/issue/28050/305073
https://dergipark.org.tr/tr/pub/gbesbd/i... ; from 0.81 to 0.94 for the cognitive domain; from 0.77 to 0.92 for the affective domain; and from 0.76 to 0.91 for the behavioral domain 1616. Deng J, Walker GJ, Swinnerton G. Leisure attitudes: a comparison between Chinese in Canada and Anglo-Canadians. Leisure (Waterloo, Ont). 2005;29(2):239-73. https://doi.org/10.1080/14927713.2005.9651331
https://doi.org/10.1080/14927713.2005.96... ,1717. Walker GJ, Deng J, Chapman R. Leisure Attitudes: a follow-up study comparing Canadians, Chinese in Canada, and Mainland Chinese. World Leis J. 2007;49(4):207-15. https://doi.org/10.1080/04419057.2007.9674513
https://doi.org/10.1080/04419057.2007.96... ,1919. Freire T, Fonte C. Escala de atitudes face ao lazer em adolescentes e jovens adultos. Paideia (Ribeirão Preto). 2007;17(36):79-87. https://doi.org/10.1590/S0103-863X2007000100008
https://doi.org/10.1590/S0103-863X200700... ,2020. Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
https://doi.org/10.1080/14927713.2013.77... .
In different applications, a lower Cronbach's alpha coefficient was observed in the behavioral subscale. The authors of the original version 1515. Ragheb MG, Beard JG. Measuring leisure attitude. J Leis Res. 1982;14(2):155-67. https://doi.org/10.1080/00222216.1982.11969512
https://doi.org/10.1080/00222216.1982.11... already attributed this situation to the greater heterogeneity in the content of the items in this domain. Cronbach's alpha coefficient indicates the homogeneity of a psychometric instrument and is one of the most used statistical tests in cross-cultural adaptation research conducted by nursing in Brazil 2222. Lino CRM, Brüggemann OM, Souza ML, Barbosa SFF, Santos EKA. The cross-cultural adaptation of research instruments, conducted by nurses in Brazil: an integrative review. Texto Contexto Enferm. 2017;26(4):e1730017. https://doi.org/10.1590/0104-07072017001730017
https://doi.org/10.1590/0104-07072017001... . Their scores range from 0 to 1, but values above 0.7 1414. Munusturlar S. Examination of the leisure attitudes of individuals living in Turkey from an east to west perspective. Balt J Sport Health Sci. 2017;2(105):47-57. https://doi.org/10.33607/bjshs.v2i105.25
https://doi.org/10.33607/bjshs.v2i105.25... are considered acceptable.
Some limitations can be pointed out in this study, such as the reduced number of specialists participating in the committee, which may have overestimated the results; and carrying out the pre-test with a specific sample of older people, which may not have portrayed the difficulties in understanding by the older adults in the community.
LAM-BV may be incorporated in research and clinical practice nursing in different contexts in the health in Brazil, considering that the involvement in leisure depends on intersectoral actions. Although the psychometric analysis of the instrument is extremely important to ensure its validity, the present study represents a first step in the possibility of comparing the attitude of the Brazilian older adults related to leisure with individuals from other cultures. Furthermore, the production of standardized information in this regard in future studies will allow the development of retirement planning programs with significant involvement in leisure, aiming at a better quality of life for the older population.
- Funding:Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES – Finance Code 001).
REFERENCES
- 1Afshar PF, Foroughan M, Vedadhir A, Tabatabaie MG. Psychometric properties of the Persian version of Social Adaptation Self-evaluation Scale in community-dwelling older adults. Clin Interv Aging. 2017;12:579-84. https://doi.org/10.2147/CIA.S129407
» https://doi.org/10.2147/CIA.S129407 - 2Gvozd R, Rossaneis MA, Pissinati PSC, Guirardello EB, Haddad MCFL. Cultural adaptation of the Retirement Resources Inventory for Brazilian culture. Rev Saude Publica. 2019;53:60. https://doi.org/10.11606/s1518-8787.2019053000863
» https://doi.org/10.11606/s1518-8787.2019053000863 - 3Liberatti VM, Martins JT, Ribeiro RP, Scholze AL, Galdino MJQ, Trevisan GS. Qualidade de vida na concepção de docentes de enfermagem aposentadas por uma universidade pública. Cienc Cuid Saude. 2016;15(4):655-61. https://doi.org/10.4025/cienccuidsaude.v15i4.29968
» https://doi.org/10.4025/cienccuidsaude.v15i4.29968 - 4Lee C, Payne LL, Berdychevsky L. The roles of leisure attitudes and self-efficacy on attitudes toward retirement among retirees: a sense of coherence theory approach. Leis Sci. 2020;42(2):158-69. https://doi.org/10.1080/01490400.2018.1448025
» https://doi.org/10.1080/01490400.2018.1448025 - 5Lambrini K, Konstantinos K, Christos I, Areti T. Leisure activity in the third age. J Healthc Commun. 2018;3:12. https://doi.org/10.4172/2472-1654.100123
» https://doi.org/10.4172/2472-1654.100123 - 6Ferrari RFR, Baldissera VDA, Lange C, Carreira L, Silva ES. Attitude toward leisure among the community-dwelling elderly: an interface for health promotion. Texto Contexto Enferm. 2016;25(4):e1280015. https://doi.org/10.1590/0104-07072016001280015
» https://doi.org/10.1590/0104-07072016001280015 - 7Kao IC, Chang LC. Long-term effects of leisure education on leisure needs and stress in older adults. Educ Gerontol. 2017;43(7):356-64. https://doi.org/10.1080/03601277.2017.1299447
» https://doi.org/10.1080/03601277.2017.1299447 - 8Puke N, Marcellino NC. Possibilidades de interface entre lazer e fenomenologia. Movimento. 2014;20:307-27. https://doi.org/10.22456/1982-8918.40321
» https://doi.org/10.22456/1982-8918.40321 - 9Nimrod G, Shrira A. The paradox of leisure in later life. J Gerontol B Psychol Sci Soc Sci. 2016;71(1):106-11. https://doi.org/10.1093/geronb/gbu143
» https://doi.org/10.1093/geronb/gbu143 - 10Belo P, Navarro-Pardo E, Pocinho R, Carrana P, Margarido C. Relationship between mental health and the education level in elderly people: mediation of leisure attitude. Front Psychol. 2020;11:573. https://doi.org/10.3389/fpsyg.2020.00573
» https://doi.org/10.3389/fpsyg.2020.00573 - 11Hicks SA, Siedlecki KL. Leisure activity engagement and positive affect partially mediate the relationship between positive views on aging and physical health. J Gerontol B Psychol Sci Soc Sci. 2017;72(2):259-67. https://doi.org/10.1093/geronb/gbw049
» https://doi.org/10.1093/geronb/gbw049 - 12Adams EV, Taniguchi ST, Hite SJ, Ward PJ, Mugimu CB, Nsubuga YK. Leisure defined by perceiving choice: perspectives of Ugandan women. World Leis J. 2018;60(4):265-80. https://doi.org/10.1080/16078055.2018.1517106
» https://doi.org/10.1080/16078055.2018.1517106 - 13Castro VC, Carreira L. Leisure activities and attitude of institutionalized elderly people: a basis for nursing practice. Rev Lat Am Enfermagem. 2015;23(2):307-14. https://doi.org/10.1590/0104-1169.3650.2556
» https://doi.org/10.1590/0104-1169.3650.2556 - 14Munusturlar S. Examination of the leisure attitudes of individuals living in Turkey from an east to west perspective. Balt J Sport Health Sci. 2017;2(105):47-57. https://doi.org/10.33607/bjshs.v2i105.25
» https://doi.org/10.33607/bjshs.v2i105.25 - 15Ragheb MG, Beard JG. Measuring leisure attitude. J Leis Res. 1982;14(2):155-67. https://doi.org/10.1080/00222216.1982.11969512
» https://doi.org/10.1080/00222216.1982.11969512 - 16Deng J, Walker GJ, Swinnerton G. Leisure attitudes: a comparison between Chinese in Canada and Anglo-Canadians. Leisure (Waterloo, Ont). 2005;29(2):239-73. https://doi.org/10.1080/14927713.2005.9651331
» https://doi.org/10.1080/14927713.2005.9651331 - 17Walker GJ, Deng J, Chapman R. Leisure Attitudes: a follow-up study comparing Canadians, Chinese in Canada, and Mainland Chinese. World Leis J. 2007;49(4):207-15. https://doi.org/10.1080/04419057.2007.9674513
» https://doi.org/10.1080/04419057.2007.9674513 - 18Chiu LK, Kayat K. Psychological determinants of leisure time physical activity participation among public university students in Malaysia. AJTLHE. 2010;2(2):35-45.
- 19Freire T, Fonte C. Escala de atitudes face ao lazer em adolescentes e jovens adultos. Paideia (Ribeirão Preto). 2007;17(36):79-87. https://doi.org/10.1590/S0103-863X2007000100008
» https://doi.org/10.1590/S0103-863X2007000100008 - 20Teixeira A, Freire T. The Leisure Attitude Scale: psychometrics properties of a short version for adolescents and young adults. Leisure (Waterloo, Ont). 2013;37(1):57-67. https://doi.org/10.1080/14927713.2013.776748
» https://doi.org/10.1080/14927713.2013.776748 - 21Akgül BM, Gürbüz B. Bo Zaman Tutum Ölçe i: geçerlik ve güvenirlik çali masi. Gazi Beden E itimi ve Spor Bilimleri Dergisi. 2011 [cited 2020 May 7];16(1):37-43. Available from: https://dergipark.org.tr/tr/pub/gbesbd/issue/28050/305073
» https://dergipark.org.tr/tr/pub/gbesbd/issue/28050/305073 - 22Lino CRM, Brüggemann OM, Souza ML, Barbosa SFF, Santos EKA. The cross-cultural adaptation of research instruments, conducted by nurses in Brazil: an integrative review. Texto Contexto Enferm. 2017;26(4):e1730017. https://doi.org/10.1590/0104-07072017001730017
» https://doi.org/10.1590/0104-07072017001730017 - 23Machado RS, Fernandes ADBF, Oliveira ALCB, Soares LS, Gouveia MTO, Silva GRF. Cross-cultural adaptation methods of instruments in the nursing area. Rev Gaucha Enferm. 2018;39:e2017-0164. https://doi.org/10.1590/1983-1447.2018.2017-0164
» https://doi.org/10.1590/1983-1447.2018.2017-0164 - 24Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Recommendations for the cross-cultural adaptation of the DASH & QuickDASH Outcome Measures. Institute for Work & Health; 2007 [cited 2019 Jan 30]. Available from: http://dash.iwh.on.ca/sites/dash/files/downloads/cross_cultural_adaptation_2007.pdf
» http://dash.iwh.on.ca/sites/dash/files/downloads/cross_cultural_adaptation_2007.pdf - 25Cunha CM, Almeida Neto OPA, Stackfleth R. Principais métodos de avaliação psicométrica da validade de instrumentos de medida. Rev Aten Saude. 2016;14(47):75-83. https://doi.org/10.13037/ras.vol14n47.3391
» https://doi.org/10.13037/ras.vol14n47.3391 - 26Ortiz-Gutiérrez S, Cruz-Avelar A. Translation and cross-cultural adaptation of health assessment tools. Actas Dermosifiliogr. 2018;109(3):202-6. https://doi.org/10.1016/j.adengl.2018.02.003
» https://doi.org/10.1016/j.adengl.2018.02.003 - 27Freire F, Teixeira A. The influence of leisure attitudes and leisure satisfaction on adolescents' positive functioning: the role of emotion regulation. Front Psychol. 2018;9:1349. https://doi.org/10.3389/fpsyg.2018.01349
» https://doi.org/10.3389/fpsyg.2018.01349 - 28Muzindutsi PF, Viljoen D. Socio-demographic factors influencing leisure attitude among undergraduate students at a South African university. Int J Soc Sci Human Stud. 2016 [cited 2020 May 07];8(1):103-15. Available from: https://www.semanticscholar.org/paper/SOCIO-DEMOGRAPHIC-FACTORS-INFLUENCING-LEISURE-AMONG-Muzindutsi-Viljoen/46a2fbc10864cdf516a720cd08cb0a64cee809ba
» https://www.semanticscholar.org/paper/SOCIO-DEMOGRAPHIC-FACTORS-INFLUENCING-LEISURE-AMONG-Muzindutsi-Viljoen/46a2fbc10864cdf516a720cd08cb0a64cee809ba - 29Murathan T, Türker A, Murathan F. Investigation of university students attitudes toward leisure time activities (Ardahan University sample). Turkish J Sport Exerc. 2019;21(2):162-9. https://doi.org/10.15314/tsed.473228
» https://doi.org/10.15314/tsed.473228 - 30Choi S, Yoo Y. Leisure attitude and satisfaction with leisure and life: proposing leisure prioritization and justification. World Leis J. 2016;59(2):140-55. https://doi.org/10.1080/16078055.2016.1216886
» https://doi.org/10.1080/16078055.2016.1216886
Publication Dates
- Publication in this collection
20 Nov 2020 - Date of issue
2020
History
- Received
04 Feb 2020 - Accepted
15 June 2020