Psychometric analysis and adaptation of the generalized anxiety questionnaire (GAD-7) to the Collao Quechua language in Peru

Julio Cjuno Raúl Alexis Villegas-Mejía Jovita Coronado-Fernández About the authors

ABSTRACT

Objectives.

To adapt the 7-item Generalized Anxiety Questionnaire (GAD-7) from English to the cultural and linguistic context of Quechua Collao and to analyze its psychometric properties in Puno, Peru.

Material and methods.

The GAD-7 in its original version was translated into the Collao Quechua variety and its psychometric properties were analyzed. The participants were bilingual (Spanish and Quechua), over 18 years of age and of both sexes. The exploratory factor analysis (EFA) was evaluated using parallel analysis, confirmatory factor analysis (CFA) and goodness-of-fit indices; reliability was also analyzed using McDonald’s classic alpha and Omega.

Results.

Judges and focus group participants conducted the cultural and linguistic adaptation of the GAD-7 to Quechua Collao; the EFA reported the presence of a single factor (KMO=0.88, p=0.01); while the CFA confirmed adequate adjustments in the unifactorial model (CFI=0.994; TLI=0.991; SRMR=0.027; RMSEA=0.092), good reliability (α=0.896; ω=0.894) and was also invariant across age groups, sex, marital status and educational level.

Conclusions.

The questionnaire showed validity for a unidimensional model of the GAD-7 adapted to the Collao Quechua, as well as optimal reliability and invariance by the evaluated groups. Its use could benefit anxiety research and care.

Keywords:
Anxiety disorder; anxiety; GAD-7; indigenous peoples

KEY MESSAGES

Motivation for the study. Peru has the largest Quechua population; however, the measurement of anxiety is not validated in the cultural and linguistic context of the Collao Quechua variety mainly spoken in Puno.

Main findings. The adaptation of the 7-item Generalized Anxiety Questionnaire (GAD-7) to Collao Quechua reported adequate internal validity, invariance, and optimal reliability.

Implications. The GAD-7 adapted to Collao Quechua could be implemented in primary health care for screening for generalized anxiety symptoms.

Keywords:
Anxiety disorder; anxiety; GAD-7; indigenous peoples

INTRODUCTION

Anxiety is a common mental health disorder with a constant growth rate, and together with depression, affects more than 264 million people in the world 11. World Health Organization. Depression. 2020 [Consultado el 17 de junio de 2021]. Disponible en: https://www.who.int/news-room/fact-sheets/detail/depression.
https://www.who.int/news-room/fact-sheet...
. In the region of the Americas, 2.1% of the inhabitants suffer from anxiety; while prevalence rates of 5.3% have been reported in Peru 22. Organización Mundial de la Salud, Organización Panamericana de la Salud. La carga de los trastornos mentales en la Región de las Américas, 2018. WHO-PAHO; 2018 [Consultado el 17 de junio de 2021]. Disponible en: https://iris.paho.org/bitstream/handle/10665.2/49578/9789275320280_spa.pdf?sequence%20=%209&isAllowed%20=%20y.
https://iris.paho.org/bitstream/handle/1...
) and 16.2% in populations from the Peruvian highlands 33. Rondón MB. Salud mental: un problema de salud pública en el Perú. Rev Peru Med Exp Salud Publica. 2006;23(4):237-8.. In spite of being a frequent mental health problem, it is usually not diagnosed in time and treatment is almost always delayed, due to a lack of personnel with a command of the language and adequate instruments 44. Gutiérrez JR, Portillo CB. La ansiedad y la depresión como indicadores de problemas de salud mental en los Salvadoreños. Revista Electrónica de Psicología Iztacala. 2013;16(2):543-57.. An example of this is the Quechua, a native population in Peru, who despite having a large population of approximately 1,326,454 inhabitants 55. Instituto Nacional de Estadística e Informática. Perú: Perfil sociodemográfico Informe nacional - Censos nacionales 2017: XII de Población, VII de Vivienda y III de Comunidades Indígenas. Lima: INEI; 2018 [Consultado el 17 de junio de 2021]. Disponible en: https://censo2017.inei.gob.pe/.
https://censo2017.inei.gob.pe/ ...
, do not have anxiety screening instruments contextualized to their cultural and linguistic environment 66. Hambleton R, Merenda M, Spielberger C. Adapting Educational and Psychological Tests for Cross-Cultural Assessment. In Psychology Press; 2005. Disponible en: https://www.routledge.com/Adapting-Educational-and-Psychological-Tests-for-Cross-Cultural-Assessment/Hambleton-Merenda-Spielberger/p/book/9780805861761.
https://www.routledge.com/Adapting-Educa...
.

In this sense, it is important to highlight that the General Anxiety Disorder (GAD-7) has been widely used by different psychometric studies and is considered as a promising instrument for anxiety screening 77. van Ballegooijen W, Riper H, Cuijpers P, van Oppen P, Smit JH. Validation of online psychometric instruments for common mental health disorders: a systematic review. BMC Psychiatry. 2016;16(1):45. doi: 10.1186/s12888-016-0735-7.
https://doi.org/10.1186/s12888-016-0735-...
, because its theoretical construction was based on the diagnostic criteria of the DSM-IV for general anxiety 88. American Psychiatric Association. The Diagnostic and Statistical Manual of Mental Disorders, Fouth Edition,Text Revision (DSM-4) 2013. [Consultado el 13 de julio de 2022]. Disponible en: https://psychiatry.org:443/psychiatrists/practice/dsm.
https://psychiatry.org:443/psychiatrists...
. It was designed in English by a team of researchers from the United States 99. Johnson SU, Ulvenes PG, Øktedalen T, Hoffart A. Psychometric Properties of the General Anxiety Disorder 7-Item (GAD-7) Scale in a Heterogeneous Psychiatric Sample. Front Psychol. 2019;10(6):1713. doi: 10.3389/fpsyg.2019.01713.
https://doi.org/10.3389/fpsyg.2019.01713...
. Currently, it has adaptations that make possible its use in the screening of anxiety, due to several adaptation and validation studies in languages such as English 1010. Mossman SA, Luft MJ, Schroeder HK, Varney ST, Fleck DE, Barzman DH, et al. The Generalized Anxiety Disorder 7-item (GAD-7) scale in adolescents with generalized anxiety disorder: signal detection and validation. Ann Clin Psychiatry. 2017;29(4):227-234A., German 1111. Hinz A, Klein AM, Brähler E, Glaesmer H, Luck T, Riedel-Heller SG, et al. Psychometric evaluation of the Generalized Anxiety Disorder Screener GAD-7, based on a large German general population sample. J Affect Disord. 2017;210:338-44. doi: 10.1016/j.jad.2016.12.012.
https://doi.org/10.1016/j.jad.2016.12.01...
, Mandarin Chinese 1212. Gong Y, Zhou H, Zhang Y, Zhu X, Wang X, Shen B, et al. Validation of the 7-item Generalized Anxiety Disorder scale (GAD-7) as a screening tool for anxiety among pregnant Chinese women. J Affect Disord. 2021;282:98-103. doi: 10.1016/j.jad.2020.12.129.
https://doi.org/10.1016/j.jad.2020.12.12...
, Spanish 1313. García-Campayo J, Zamorano E, Ruiz MA, Pardo A, Pérez-Páramo M, López-Gómez V, et al. Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool. Health Qual Life Outcomes. 2010;8:8. doi: 10.1186/1477-7525-8-8.
https://doi.org/10.1186/1477-7525-8-8...
, Portuguese 1414. Moreno AL, DeSousa DA, Souza AMFLP de, Manfro GG, Salum GA, Koller SH, et al. Factor structure, reliability, and item parameters of the brazilian-portuguese version of the GAD-7 questionnaire. Temas em Psicologia. 2016;24(1):367-76. doi: 10.9788/TP2016.1-25.
https://doi.org/10.9788/TP2016.1-25...
, Bolivian Spanish 1515. Porto MF, Ocampo-Barba N, Flores-Valdivia G, Caldichoury N, López N, Porto MF, et al. Propiedades psicométricas del GAD-7 para detectar ansiedad generalizada en profesionales sanitarios de Bolivia. Rev Peru Med Exp Salud Publica. 2022;39(1):115-6. doi: 10.17843/rpmesp.2022.391.8620.
https://doi.org/10.17843/rpmesp.2022.391...
, Colombian Spanish 1616. Camargo L, Herrera-Pino J, Shelach S, Soto-Añari M, Porto MF, Alonso M, et al. Escala de ansiedad generalizada GAD-7 en profesionales médicos colombianos durante pandemia de COVID-19: validez de constructo y confiabilidad. Rev Colomb Psiquiatr. 2021;2(Jul):1-6. doi: 10.1016/j.rcp.2021.06.003.
https://doi.org/10.1016/j.rcp.2021.06.00...
. In all previous studies, the GAD-7 has shown single-factor behavior and adequate evidence of validity and reliability.

In Peru, the unifactorial model of the GAD-7 can be used in Spanish-speaking pregnant women 1717. Zhong QY, Gelaye B, Zaslavsky AM, Fann JR, Rondon MB, Sánchez SE, et al. Diagnostic Validity of the Generalized Anxiety Disorder - 7 (GAD-7) among Pregnant Women. PLOS ONE. 2015;10(4):e0125096. doi: 10.1371/journal.pone.0125096.
https://doi.org/10.1371/journal.pone.012...
) and in young university students 1818. Franco-Jimenez RA, Nuñez-Magallanes A, Franco-Jimenez RA, Nuñez-Magallanes A. Propiedades psicométricas del GAD-7, GAD-2 y GAD-Mini en universitarios peruanos. Propósitos y Representaciones. 2022;10(1):e1437. doi: 10.20511/pyr2022.v10n1.1437.
https://doi.org/10.20511/pyr2022.v10n1.1...
. The absence of anxiety measurements within Quechua-speaking communities in Peru creates a gap in the understanding of mental health in this population, which represents about 13.6% of the country’s total population 1919. Andrade L. Diez noticias sobre el quechua en el último censo peruano. Letras (Lima). 2019;90(132):41-70. doi: 10.30920/letras.90.132.2.
https://doi.org/10.30920/letras.90.132.2...
. The lack of an adaptation to Quechua limits the ability to identify and address mental health problems within these communities and hinders the development of effective and culturally sensitive interventions for anxiety. Given the importance of mental health in people’s quality of life and well-being 2020. Angell B, Muhunthan J, Eades AM, Cunningham J, Garvey G, Cass A, et al. The health-related quality of life of Indigenous populations: a global systematic review. Qual Life Res. 2016;25(9):2161-78. doi: 10.1007/s11136-016-1311-9.
https://doi.org/10.1007/s11136-016-1311-...
, it is essential to conduct studies that consider the experience and expression of anxiety in Quechua-speaking communities in order to ensure comprehensive and equitable care for all ethnic groups in the country.

Therefore, this study aimed to adapt the GAD-7 from English to the cultural and linguistic context of the Collao Quechua and to analyze its psychometric properties such as internal structure validity and reliability.

MATERIAL AND METHODS

Design and context

This was a psychometric study, developed in the Puno region of Peru, which is home to approximately 538,127 Quechua speakers of the Quechua Collao variety (57.0% of the population of Puno), who mainly engage in commercial activities, tourism, agriculture and livestock 55. Instituto Nacional de Estadística e Informática. Perú: Perfil sociodemográfico Informe nacional - Censos nacionales 2017: XII de Población, VII de Vivienda y III de Comunidades Indígenas. Lima: INEI; 2018 [Consultado el 17 de junio de 2021]. Disponible en: https://censo2017.inei.gob.pe/.
https://censo2017.inei.gob.pe/ ...
.

Translation

The original English version of the GAD-7 was translated directly into the Collao Quechua variety. Once the translation was completed, the two translators and a Quechua-speaking researcher met to discuss the differences in the translations. Once the discrepancies were resolved and the translations unified, they proceeded with the back translation from Quechua to English. Subsequently, the translators and the research team verified the back translation along with the first translation, fine-tuning details and giving their approval to the final version of the Quechua Collao GAD-7.

Cultural adaptation into Collao Quechua

By applying the Delphi method, we elaborated a cultural adaptation form (Supplementary Material: Annex 1) with some open questions for uncommon words in Quechua such as: “anxiety” and “nervous”. Additionally, we inquired about the change of the response category “more than half the days” to “ashka p’unchaykuna” and about the relationship between the Collao Quechua GAD-7 with the DSM-V for the diagnosis of generalized anxiety. Those questions were sent via e-mail. Three bilingual (i.e., Spanish and Quechua-speaking) psychologists participated, all of whom had at least one year of experience in the psychotherapeutic treatment of depression in Quechua-speaking patients of the Collao variety. The interaction between each expert and the research team took place in two rounds of emails, in addition to a meeting via Zoom with the research team in order to be able to assess suggestions and reach a consensus.

Subsequently, we conducted a virtual focus group to verify the clarity and understanding of the items in the target population, this meeting lasted approximately 60 minutes. At the beginning, after obtaining informed consent, we asked the participants to answer the Collao Quechua GAD-7 in an online version shared via Google Forms. Then, the moderator (a native Quechua-speaking psychologist with knowledge of qualitative methods) invited the participants to give their opinion on the clarity and comprehension of the items in a language that was common and simple for Quechua speakers. Five Quechua speakers over the age of 18 (three females and two males) who were bilingual (Quechua and Spanish) participated, this process is shown in the (Supplementary Material: Appendix 2).

Participants

The study had two samples. The final version of the GAD-7 translated into Collao Quechua was applied to a non-probabilistic sample of 206 participants for the exploratory factor analysis (EFA), following the recommendations for factor analysis of 20 participants per item, that is, 7 items X 20 participants each resulting in 140 participants 2121. Kyriazos TA. Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General. Psychology. 2018;09(08):2207. doi: 10.4236/psych.2018.98126.
https://doi.org/10.4236/psych.2018.98126...
. On the other hand, the sample included 454 participants for the confirmatory factor analysis (CFA), this number being higher than the number calculated by the application: https://wnarifin.shinyapps.io/ss_sem_cfi_equal/, where the sample size of the CFA for structural equation modeling (SEM) expected a sample of 414 participants, using a statistical power of 95% and a loss rate of 10%.

Both samples included participants over 18 years of age, of both sexes, living in urban and rural settings, located in the departments of Puno and Cusco, who reported being bilingual Quechua speakers (Quechua and Spanish), with sufficient academic background to read Quechua (e.g., at least incomplete primary school), excluding speakers of other varieties of Quechua and those who did not provide informed consent.

Instrument

The original English version of the GAD-7 2222. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7. Arch Intern Med. 2006;166(10):1092-7. doi: 10.1001/archinte.166.10.1092.
https://doi.org/10.1001/archinte.166.10....
, consists of 7 items that correspond to the generalized anxiety symptoms of the DSM-IV 88. American Psychiatric Association. The Diagnostic and Statistical Manual of Mental Disorders, Fouth Edition,Text Revision (DSM-4) 2013. [Consultado el 13 de julio de 2022]. Disponible en: https://psychiatry.org:443/psychiatrists/practice/dsm.
https://psychiatry.org:443/psychiatrists...
. Its response options evoke the frequency of occurrence of such symptoms in the last two weeks, considering the following Likert-type scale: 0 = not at all, 1 = several days, 2 = more than half the days, 3 = nearly every day. This defines a raw score between 0 and 21. The English version of the GAD-7 showed a sensitivity and specificity of 86% and 82% respectively, it also showed validity based on the relationship with other variables such as Beck’s Anxiety (r = 0.72) and the anxiety subscale of the Symptom Checklist-90 (r = 0.74) indicating external validity; likewise, it showed a good reliability of α = 0.92.

Procedures

For data collection, two surveyors, who were fifth-year psychology students, were trained in the use and application of the instrument. The surveyors identified WhatsApp groups of parents from educational institutions, Christian churches and groups of peasant community associations to whom they presented the survey in Google Forms format. The informed consent form was included at the beginning of the survey and only those who agreed to participate in the study were able to respond to it. Data collection began in January and ended in February 2023.

Statistical analysis

We conducted a descriptive analysis of the items (mean, standard deviation, skewness and kurtosis). Subsequently, we used the Exploratory Factor Analysis (EFA) to verify the number of factors through parallel analysis, additionally the Kaiser-Meyer-Olkin (KMO) test was estimated through the strength of the partial correlations between variables considering a KMO > 0.80 as adequate plausibility; additionally, Bartlett’s sphericity test (p < 0.05) should affirm that the correlation matrix is not an identity matrix 2323. Bandalos DL, Finney SJ. Factor Analysis: Exploratory and Confirmatory. In: The Reviewer's Guide to Quantitative Methods in the Social Sciences. 2nd ed. Routledge; 2018. p. 25. doi: 10.4324/9781315755649.
https://doi.org/10.4324/9781315755649...
.

We also conducted a Confirmatory Factor Analysis (CFA) of the unidimensional model using a WLSMV (Weighted Least Square Mean and Variance Adjusted) estimator, as we did previously 2424. Ledesma RD, Fernando P, Tosi JD. Uso del Análisis Factorial Exploratorio en RIDEP. Recomendaciones para Autores y Revisores. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica. 2019;52(3):173-80.. We report the standardized betas of the model and standard goodness-of-fit measures: the X2 for model versus baseline, considering values < 3 acceptable; the comparative fit index (CFI), which is adequate when > 0.90; the Tucker-Lewis Index (TLI), which is acceptable when > 0.90. Likewise, we used the standardized root mean square residual (SRMR) and the root mean square error of approximation (RMSEA), considering them adequate with values ≤ 0.08 2525. Hu LT, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. A Multidisciplinary Journal. 1999;6(1):1-55. doi: 10.1080/10705519909540118.
https://doi.org/10.1080/1070551990954011...
. SEM Structural Equation Modeling was used to graphically present the factor loadings 2626. Kline RB. Principles and Practice of Structural Equation Modeling, 4th ed. Guilford Publications; 2015. 553 p. Disponible en: https://psycnet.apa.org/record/2015-56948-000.
https://psycnet.apa.org/record/2015-5694...
) and finally, internal consistency was estimated using the classic Alpha and McDonald’s Omega 2727. Viladrich C, Angulo-Brunet A, Doval E. A journey around alpha and omega to estimate internal consistency reliability. Un viaje alrededor de alfa y omega para estimar la fiabilidad de consistencia interna. Anales de Psicologia. 2017;33(3):755-82. doi: 10.6018/analesps.33.3.268401.
https://doi.org/10.6018/analesps.33.3.26...
.

Measurement invariance was analyzed by means of the Multigroup CFA method considering groups defined by sex and marital status. The difference in CFI (ΔCFI) or RMSEA (ΔRMSEA) the criterion for determining models with more restrictions versus those with fewer restrictions, initially assuming the invariance configured as the base model, scaling to metric, strong and strict invariance. In order to determine an adequately constrained and appropriate model at each invariance stage the ΔCFI or ΔRMSEA had to have values < 0.01 2828. Widaman KF, Reise SP. Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In: The science of prevention: Methodological advances from alcohol and substance abuse research. Washington, DC, US: American Psychological Association; 1997. p. 281-324. doi: 10.1037/10222-009.
https://doi.org/10.1037/10222-009...
.

Additionally, MIMIC (Multiple Indicators and Multiple Causes) models were used to determine invariance by age and educational level (e.g., because they were numerical variables or when it was not feasible to use the Multigroup CFA). Therefore, the invariance of the intercepts of the indicators and the mean differences of the latent dimensions were evaluated, all through groups according to the aforementioned variables. Each covariate was evaluated separately, each comparing two types of models; the first was a saturated version where the covariate explains all the observed items, but not the latent dimensions, and the second, a version of the intercept invariance model where the covariate explained all the latent dimensions, but not the items. To determine a more restricted model we had to identify a ΔCFI or a ΔRMSEA <0.01 2828. Widaman KF, Reise SP. Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In: The science of prevention: Methodological advances from alcohol and substance abuse research. Washington, DC, US: American Psychological Association; 1997. p. 281-324. doi: 10.1037/10222-009.
https://doi.org/10.1037/10222-009...
. The statistical software R Studio version 3,3,0+ and its psych packages, GPArotation, polycor, lavaan, semTools, and semPlot were used for the analysis.

Ethical considerations

This study was evaluated and approved by the Ethics Committee of the Universidad Peruana Unión with report number 2022-CE-FCS - UPeU-059. Likewise, all the ethical principles of research in human beings stated in the Declaration of Helsinki 2929. Asociación Médica Mundial. Declaración de Helsinki. 2018 [Consultado el 21 de mayo de 2022]. Disponible en: https://www.wma.net/es/policies-post/declaracion-de-helsinki-de-la-amm-principios-eticos-para-las-investigaciones-medicas-en-seres-humanos/.
https://www.wma.net/es/policies-post/dec...
were respected.

RESULTS

Cultural adaptation phase

The experts who participated in the cultural adaptation by the Delphi method provided valuable suggestions that contributed to improve the adaptation of the word "anxiety" which was initially adapted as “phutisk’alla”, then the experts suggested to add “ansiedad nisqhawan” which, although is a combination between Spanish and Quechua, helps to achieve better understanding; therefore, the final term was “phutisk’alla/anxiety nisqhawan” in item 1. On the other hand, the expression “nervous” was changed to “Ancha mancharisqha” according to the recommendation by the experts. The focus group participants gave a favorable opinion regarding the clarity and comprehension of the Collao Quechua GAD-7, stating that it was clear and in accordance with their cultural context without any additional changes.

The response options (Likert type) of the Collao Quechua GAD-7 were also changed by the expert judges and the focus group participants, because the category “more than half of the days” presented clarity problems in its translation. The team’s final recommendation was to use the expressions “Mana hayk’aqpas”, “Wakin p’unchawkunalla”, “Ashka p'unchawkuna”, “Yaqa llapa p’un chawkuna”, representing the Spanish equivalent of “never”, “some days”, “several days”, “almost every day”. After improving the items and response categories of the Collao Quechua GAD-7, the experts who participated in the cultural adaptation, in a last interaction with the research team, rated each item with the maximum score (3/3 in a range from 0 to 3) for each content validity indicator such as: relevance, representativeness, clarity and cultural equivalence.

Characteristics of the participants

The EFA sample consisted of 206 participants with an average age of 31 years, of whom 57.3% (118) were female, 6.1% (132) were single/widowed/divorced and 59.7% (123) reported having a university education; on the other hand, the CFA sample consisted of 454 participants with an average age of 32 years, where 57.7% (262) were female, 57.0% (259) reported being single/widowed/divorced and 40.7% (185) indicated having university studies (Supplementary material: Appendix 3).

Exploratory factor analysis

The sedimentation plot, or parallel analysis, reported the presence of a single latent factor (Figure 1). On the other hand, the Kaiser-Meyer-Olkin test and Bartlett’s test of sphericity (KMO = 0.88, p = 0.01) confirmed the presence of that factor from the included variables (Supplementary material: Appendix 3) with factor weights ranging from (0.64 - 0.85) obtained by the maximum likelihood method and varimax rotation; indicating that, theoretically, the items form a factor (Table 1).

Figure 1
Parallel analysis of the GAD-7 adapted to Collao Quechua.

Table 1
Descriptive analysis of the GAD-7 items adapted to Collao Quechua.

Confirmatory factor analysis

The single factor model reported adequate goodness-of-fit values for the Collao Quechua GAD-7, (CFI = 0.994; TLI = 0.991; SRMR = 0.027; RMSEA = 0.092) (Table 2). Regarding factor loadings, a minimum of λ = 0.72 and a maximum of λ = 0.86 were obtained for the Collao Quechua GAD-7 items (Figure 2).

Table 2
Confirmatory Factor Analysis with WLSMV estimator for the GAD-7 adapted to Collao Quechua.

Figure 2
SEM graph of the GAD-7 unifactor model adapted to Col-laoQuechua with betas and errors.

Reliability

Additionally, the Collao Quechua GAD-7 reported good reliability, with Cronbach’s Alpha values of 0.896 and Omega of 0.894 (Table 2).

Measurement invariance

The multigroup CFA analysis for the Collao Quechua GAD-7 showed measurement invariance according to sex and marital status, presenting adequate goodness-of-fit indices (CFI > 0.98, RMSEA < 0.08) and values < 0.01 for at least the ΔCFI or ΔRMSEA; after verifying the configural invariance, we analyzed metric invariance (ΔCFI = 0.002), strong invariance (ΔCFI = 0.003) and strict invariance (ΔCFI < 0.004) (Table 3). On the other hand, by means of Multiple Indicator Multiple Causal Invariance (MIMIC), we confirmed the invariance by age and educational level since they presented adequate goodness-of-fit indices (CFI > 0.99, TLI > 0.99, RMSEA < 0.07, SRMR < 0.02) and all ΔCFI, ΔTLI, ΔMRSEA were < 0.01 in the saturated and intercept invariance (Table 4).

Table 3
Multigroup CFA invariance according to sex and marital status for the GAD-7 adapted to Collao Quechua.
Table 4
Invariance of Multiple Indicators and Multiple Causes (MIMIC) by age and level of education

DISCUSSION

The GAD-7 adapted to Collao Quechua showed adequate adjustments for a unifactorial model, likewise it was invariant by sex, marital status, age and educational level; it also presented optimal reliability for its use in Quechua speakers of the Collao variety. The Collao Quechua GAD-7 could be useful for assessing anxiety symptoms in the Quechua-speaking population in the departments of Puno, Cusco, Arequipa, Moquegua, Tacna and Puerto Maldonado.

The graphical analysis of parallels and factor loadings in the EFA and the goodness-of-fit indices in the CFA supported a unifactorial model; the RMSEA presented somewhat elevated values but not to the point of reaching a mismatch 3030. Chen F, Curran PJ, Bollen KA, Kirby J, Paxton P. An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models. Sociological Methods & Research [Internet]. 2008;36(4):462-94. doi: 10.1177/0049124108314720.
https://doi.org/10.1177/0049124108314720...
. The RMSEA usually tolerates some mismatch due to the sample size 3131. Rigdon EE. CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal. 1996;3(4):369-79. doi: 10.1080/10705519609540052.
https://doi.org/10.1080/1070551960954005...
. However, this behavior was not only reported by our study; studies conducted in Germany 1111. Hinz A, Klein AM, Brähler E, Glaesmer H, Luck T, Riedel-Heller SG, et al. Psychometric evaluation of the Generalized Anxiety Disorder Screener GAD-7, based on a large German general population sample. J Affect Disord. 2017;210:338-44. doi: 10.1016/j.jad.2016.12.012.
https://doi.org/10.1016/j.jad.2016.12.01...
, Spain 1313. García-Campayo J, Zamorano E, Ruiz MA, Pardo A, Pérez-Páramo M, López-Gómez V, et al. Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool. Health Qual Life Outcomes. 2010;8:8. doi: 10.1186/1477-7525-8-8.
https://doi.org/10.1186/1477-7525-8-8...
, Brazil 1414. Moreno AL, DeSousa DA, Souza AMFLP de, Manfro GG, Salum GA, Koller SH, et al. Factor structure, reliability, and item parameters of the brazilian-portuguese version of the GAD-7 questionnaire. Temas em Psicologia. 2016;24(1):367-76. doi: 10.9788/TP2016.1-25.
https://doi.org/10.9788/TP2016.1-25...
, Colombia 1616. Camargo L, Herrera-Pino J, Shelach S, Soto-Añari M, Porto MF, Alonso M, et al. Escala de ansiedad generalizada GAD-7 en profesionales médicos colombianos durante pandemia de COVID-19: validez de constructo y confiabilidad. Rev Colomb Psiquiatr. 2021;2(Jul):1-6. doi: 10.1016/j.rcp.2021.06.003.
https://doi.org/10.1016/j.rcp.2021.06.00...
and Bolivia 1515. Porto MF, Ocampo-Barba N, Flores-Valdivia G, Caldichoury N, López N, Porto MF, et al. Propiedades psicométricas del GAD-7 para detectar ansiedad generalizada en profesionales sanitarios de Bolivia. Rev Peru Med Exp Salud Publica. 2022;39(1):115-6. doi: 10.17843/rpmesp.2022.391.8620.
https://doi.org/10.17843/rpmesp.2022.391...
also found high values. Despite the high RMSEA trend in those studies, the RMSEA value reported by our study could be attributed to the size of the sample 3030. Chen F, Curran PJ, Bollen KA, Kirby J, Paxton P. An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models. Sociological Methods & Research [Internet]. 2008;36(4):462-94. doi: 10.1177/0049124108314720.
https://doi.org/10.1177/0049124108314720...
and the differences in the proportions of subsamples, mainly by educational level. Another reason could be the inclusion of uneducated people, even though they could read, it is possible that some items are not easy for them to understand; further studies could seek to adapt the instrument in an audible format for native Quechua speakers without schooling. Despite this, the Collao Quechua GAD-7 is a valid screening instrument with a unifactorial structure because the other goodness-of-fit indices strictly met the expected parameters.

Our findings show optimal reliability of the scores derived from the Collao Quechua GAD-7 in the Peruvian population. This is consistent with other findings in previous GAD-7 validation studies such as Germany (α = 0.85) 1111. Hinz A, Klein AM, Brähler E, Glaesmer H, Luck T, Riedel-Heller SG, et al. Psychometric evaluation of the Generalized Anxiety Disorder Screener GAD-7, based on a large German general population sample. J Affect Disord. 2017;210:338-44. doi: 10.1016/j.jad.2016.12.012.
https://doi.org/10.1016/j.jad.2016.12.01...
, China (α = 0.84) 1212. Gong Y, Zhou H, Zhang Y, Zhu X, Wang X, Shen B, et al. Validation of the 7-item Generalized Anxiety Disorder scale (GAD-7) as a screening tool for anxiety among pregnant Chinese women. J Affect Disord. 2021;282:98-103. doi: 10.1016/j.jad.2020.12.129.
https://doi.org/10.1016/j.jad.2020.12.12...
, Spain (α = 0.94) 1313. García-Campayo J, Zamorano E, Ruiz MA, Pardo A, Pérez-Páramo M, López-Gómez V, et al. Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool. Health Qual Life Outcomes. 2010;8:8. doi: 10.1186/1477-7525-8-8.
https://doi.org/10.1186/1477-7525-8-8...
, Brazil (α = 0.92) (14), Colombia (α = 0.92) 1616. Camargo L, Herrera-Pino J, Shelach S, Soto-Añari M, Porto MF, Alonso M, et al. Escala de ansiedad generalizada GAD-7 en profesionales médicos colombianos durante pandemia de COVID-19: validez de constructo y confiabilidad. Rev Colomb Psiquiatr. 2021;2(Jul):1-6. doi: 10.1016/j.rcp.2021.06.003.
https://doi.org/10.1016/j.rcp.2021.06.00...
who identified reliability values similar to those we found. In spite of all these studies being carried out in culturally and linguistically different populations, the measures of internal consistency by the classic alpha are similar. This allows for comparability of the intercultural anxiety variable by having achieved an optimal instrument in its internal consistency for Quechua-speaking populations of the Collao variety. The Classic Alpha was probably chosen to estimate reliability due to the search for comparability of the data for the purpose of meta-analysis 3232. COnsensus-based Standards for the selection of health Measurement INstruments. COSMIN. 2021 [Consultado el 18 de marzo de 2022]. I'm conducting a study on measurement properties. Disponible en: https://www.cosmin.nl/.
https://www.cosmin.nl/...
. We additionally estimated the McDonald Omega with the aim of evaluating internal consistency from factor loadings, as it is more appropriate for estimating reliability 2727. Viladrich C, Angulo-Brunet A, Doval E. A journey around alpha and omega to estimate internal consistency reliability. Un viaje alrededor de alfa y omega para estimar la fiabilidad de consistencia interna. Anales de Psicologia. 2017;33(3):755-82. doi: 10.6018/analesps.33.3.268401.
https://doi.org/10.6018/analesps.33.3.26...
, without being affected by the number of items or response categories, as is the case with the classical alpha 3333. Ventura-León JL, Caycho-Rodríguez T. El coeficiente Omega: un método alternativo para la estimación de la confiabilidad. Rev Latinoam Cienc Soc Niñez Juv. 2017;15(1):625-7.. Having optimal reliability values shows that this version of the GAD-7 is consistent and reliable for measuring anxiety in Quechua speakers of the Collao variety, because the construct items are consistently correlated with each other in the measurement of the underlying construct, thus being able to offer reliable and consistent measurements.

The Collao Quechua GAD-7 also presented configural, metric, scalar and strict invariance for sex and marital status groups, and reported saturated and intercept invariance by age and educational level. Previous studies have reported similar results, such as the one conducted in Italy where the GAD-7 was invariant by age and gender 3434. Bolgeo T, Di Matteo R, Simonelli N, Dal Molin A, Lusignani M, Bassola B, et al. Psychometric properties and measurement invariance of the 7-item General Anxiety Disorder scale (GAD-7) in an Italian coronary heart disease population. J Affect Disord. 2023;334:213-9. doi: 10.1016/j.jad.2023.04.140.
https://doi.org/10.1016/j.jad.2023.04.14...
; in China the GAD-7 showed evidence of invariance by age, gender, education level and residence 3535. Zhang C, Wang T, Zeng P, Zhao M, Zhang G, Zhai S, et al. Reliability, Validity, and Measurement Invariance of the General Anxiety Disorder Scale Among Chinese Medical University Students. Front Psychiatry. 2021;12. doi: 10.3389/fpsyt.2021.648755.
https://doi.org/10.3389/fpsyt.2021.64875...
; likewise, another study conducted in the United Kingdom, Ireland, Spain and Italy found saturated and intercept invariance by country, age and gender groups 3636. Shevlin M, Butter S, McBride O, Murphy J, Gibson-Miller J, Hartman TK, et al. Measurement invariance of the Patient Health Questionnaire (PHQ-9) and Generalized Anxiety Disorder scale (GAD-7) across four European countries during the COVID-19 pandemic. BMC Psychiatry. 2022;22(1):154. doi: 10.1186/s12888-022-03787-5.
https://doi.org/10.1186/s12888-022-03787...
. Those similarities between previous studies and ours suggest that in Quechua speakers of the Collao variety, the GAD-7 Quechua Collao allows comparisons of the measure of anxiety without variation in the measurement between males and females, married/cohabiting and single/widowed/divorced, age group and in those with no education, with primary, secondary, higher technical and university education.

The present study has some limitations such as the self-report data collection technique, which could have influenced the participants to be dishonest when answering the questions; however, the participants were previously sensitized to the importance of being honest when answering the instrument. On the other hand, it is possible that the interviewer did not respect the self-reporting application protocol, and in some cases may have added explanations to the participants’ questions, which could have influenced the somewhat high RMSEA values; for this reason, we sought to control the effect of the evaluator with training in the presentation and data collection process, as well as constant monitoring of his/her role to ensure that the self-reporting collection technique was respected. It is also important to state that the MIMIC models we used for the assessment of invariance by age and educational level are limited to assessing the invariant intercept models and factor means, and assume that the rest of the structural and measurement parameters such as factor loadings, variance or covariance error, variance or covariance factor are the same across all levels of these variables.

In conclusion, our findings show validity for a unidimensional model of the Collao Quechua GAD-7, with optimal reliability and invariance by the evaluated groups such as age, sex, marital status and educational level. Its use could benefit research and care of patients with anxiety in the Quechua-speaking population from the Puno region of Peru.

References

  • 1
    World Health Organization. Depression. 2020 [Consultado el 17 de junio de 2021]. Disponible en: https://www.who.int/news-room/fact-sheets/detail/depression
    » https://www.who.int/news-room/fact-sheets/detail/depression
  • 2
    Organización Mundial de la Salud, Organización Panamericana de la Salud. La carga de los trastornos mentales en la Región de las Américas, 2018. WHO-PAHO; 2018 [Consultado el 17 de junio de 2021]. Disponible en: https://iris.paho.org/bitstream/handle/10665.2/49578/9789275320280_spa.pdf?sequence%20=%209&isAllowed%20=%20y
    » https://iris.paho.org/bitstream/handle/10665.2/49578/9789275320280_spa.pdf?sequence%20=%209&isAllowed%20=%20y
  • 3
    Rondón MB. Salud mental: un problema de salud pública en el Perú. Rev Peru Med Exp Salud Publica. 2006;23(4):237-8.
  • 4
    Gutiérrez JR, Portillo CB. La ansiedad y la depresión como indicadores de problemas de salud mental en los Salvadoreños. Revista Electrónica de Psicología Iztacala. 2013;16(2):543-57.
  • 5
    Instituto Nacional de Estadística e Informática. Perú: Perfil sociodemográfico Informe nacional - Censos nacionales 2017: XII de Población, VII de Vivienda y III de Comunidades Indígenas. Lima: INEI; 2018 [Consultado el 17 de junio de 2021]. Disponible en: https://censo2017.inei.gob.pe/.
    » https://censo2017.inei.gob.pe/
  • 6
    Hambleton R, Merenda M, Spielberger C. Adapting Educational and Psychological Tests for Cross-Cultural Assessment. In Psychology Press; 2005. Disponible en: https://www.routledge.com/Adapting-Educational-and-Psychological-Tests-for-Cross-Cultural-Assessment/Hambleton-Merenda-Spielberger/p/book/9780805861761
    » https://www.routledge.com/Adapting-Educational-and-Psychological-Tests-for-Cross-Cultural-Assessment/Hambleton-Merenda-Spielberger/p/book/9780805861761
  • 7
    van Ballegooijen W, Riper H, Cuijpers P, van Oppen P, Smit JH. Validation of online psychometric instruments for common mental health disorders: a systematic review. BMC Psychiatry. 2016;16(1):45. doi: 10.1186/s12888-016-0735-7.
    » https://doi.org/10.1186/s12888-016-0735-7
  • 8
    American Psychiatric Association. The Diagnostic and Statistical Manual of Mental Disorders, Fouth Edition,Text Revision (DSM-4) 2013. [Consultado el 13 de julio de 2022]. Disponible en: https://psychiatry.org:443/psychiatrists/practice/dsm
    » https://psychiatry.org:443/psychiatrists/practice/dsm
  • 9
    Johnson SU, Ulvenes PG, Øktedalen T, Hoffart A. Psychometric Properties of the General Anxiety Disorder 7-Item (GAD-7) Scale in a Heterogeneous Psychiatric Sample. Front Psychol. 2019;10(6):1713. doi: 10.3389/fpsyg.2019.01713.
    » https://doi.org/10.3389/fpsyg.2019.01713
  • 10
    Mossman SA, Luft MJ, Schroeder HK, Varney ST, Fleck DE, Barzman DH, et al. The Generalized Anxiety Disorder 7-item (GAD-7) scale in adolescents with generalized anxiety disorder: signal detection and validation. Ann Clin Psychiatry. 2017;29(4):227-234A.
  • 11
    Hinz A, Klein AM, Brähler E, Glaesmer H, Luck T, Riedel-Heller SG, et al. Psychometric evaluation of the Generalized Anxiety Disorder Screener GAD-7, based on a large German general population sample. J Affect Disord. 2017;210:338-44. doi: 10.1016/j.jad.2016.12.012.
    » https://doi.org/10.1016/j.jad.2016.12.012
  • 12
    Gong Y, Zhou H, Zhang Y, Zhu X, Wang X, Shen B, et al. Validation of the 7-item Generalized Anxiety Disorder scale (GAD-7) as a screening tool for anxiety among pregnant Chinese women. J Affect Disord. 2021;282:98-103. doi: 10.1016/j.jad.2020.12.129.
    » https://doi.org/10.1016/j.jad.2020.12.129
  • 13
    García-Campayo J, Zamorano E, Ruiz MA, Pardo A, Pérez-Páramo M, López-Gómez V, et al. Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool. Health Qual Life Outcomes. 2010;8:8. doi: 10.1186/1477-7525-8-8.
    » https://doi.org/10.1186/1477-7525-8-8
  • 14
    Moreno AL, DeSousa DA, Souza AMFLP de, Manfro GG, Salum GA, Koller SH, et al. Factor structure, reliability, and item parameters of the brazilian-portuguese version of the GAD-7 questionnaire. Temas em Psicologia. 2016;24(1):367-76. doi: 10.9788/TP2016.1-25.
    » https://doi.org/10.9788/TP2016.1-25
  • 15
    Porto MF, Ocampo-Barba N, Flores-Valdivia G, Caldichoury N, López N, Porto MF, et al. Propiedades psicométricas del GAD-7 para detectar ansiedad generalizada en profesionales sanitarios de Bolivia. Rev Peru Med Exp Salud Publica. 2022;39(1):115-6. doi: 10.17843/rpmesp.2022.391.8620.
    » https://doi.org/10.17843/rpmesp.2022.391.8620
  • 16
    Camargo L, Herrera-Pino J, Shelach S, Soto-Añari M, Porto MF, Alonso M, et al. Escala de ansiedad generalizada GAD-7 en profesionales médicos colombianos durante pandemia de COVID-19: validez de constructo y confiabilidad. Rev Colomb Psiquiatr. 2021;2(Jul):1-6. doi: 10.1016/j.rcp.2021.06.003.
    » https://doi.org/10.1016/j.rcp.2021.06.003
  • 17
    Zhong QY, Gelaye B, Zaslavsky AM, Fann JR, Rondon MB, Sánchez SE, et al. Diagnostic Validity of the Generalized Anxiety Disorder - 7 (GAD-7) among Pregnant Women. PLOS ONE. 2015;10(4):e0125096. doi: 10.1371/journal.pone.0125096.
    » https://doi.org/10.1371/journal.pone.0125096
  • 18
    Franco-Jimenez RA, Nuñez-Magallanes A, Franco-Jimenez RA, Nuñez-Magallanes A. Propiedades psicométricas del GAD-7, GAD-2 y GAD-Mini en universitarios peruanos. Propósitos y Representaciones. 2022;10(1):e1437. doi: 10.20511/pyr2022.v10n1.1437.
    » https://doi.org/10.20511/pyr2022.v10n1.1437
  • 19
    Andrade L. Diez noticias sobre el quechua en el último censo peruano. Letras (Lima). 2019;90(132):41-70. doi: 10.30920/letras.90.132.2.
    » https://doi.org/10.30920/letras.90.132.2
  • 20
    Angell B, Muhunthan J, Eades AM, Cunningham J, Garvey G, Cass A, et al. The health-related quality of life of Indigenous populations: a global systematic review. Qual Life Res. 2016;25(9):2161-78. doi: 10.1007/s11136-016-1311-9.
    » https://doi.org/10.1007/s11136-016-1311-9
  • 21
    Kyriazos TA. Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General. Psychology. 2018;09(08):2207. doi: 10.4236/psych.2018.98126.
    » https://doi.org/10.4236/psych.2018.98126
  • 22
    Spitzer RL, Kroenke K, Williams JBW, Löwe B. A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7. Arch Intern Med. 2006;166(10):1092-7. doi: 10.1001/archinte.166.10.1092.
    » https://doi.org/10.1001/archinte.166.10.1092
  • 23
    Bandalos DL, Finney SJ. Factor Analysis: Exploratory and Confirmatory. In: The Reviewer's Guide to Quantitative Methods in the Social Sciences. 2nd ed. Routledge; 2018. p. 25. doi: 10.4324/9781315755649.
    » https://doi.org/10.4324/9781315755649
  • 24
    Ledesma RD, Fernando P, Tosi JD. Uso del Análisis Factorial Exploratorio en RIDEP. Recomendaciones para Autores y Revisores. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica. 2019;52(3):173-80.
  • 25
    Hu LT, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. A Multidisciplinary Journal. 1999;6(1):1-55. doi: 10.1080/10705519909540118.
    » https://doi.org/10.1080/10705519909540118
  • 26
    Kline RB. Principles and Practice of Structural Equation Modeling, 4th ed. Guilford Publications; 2015. 553 p. Disponible en: https://psycnet.apa.org/record/2015-56948-000
    » https://psycnet.apa.org/record/2015-56948-000
  • 27
    Viladrich C, Angulo-Brunet A, Doval E. A journey around alpha and omega to estimate internal consistency reliability. Un viaje alrededor de alfa y omega para estimar la fiabilidad de consistencia interna. Anales de Psicologia. 2017;33(3):755-82. doi: 10.6018/analesps.33.3.268401.
    » https://doi.org/10.6018/analesps.33.3.268401
  • 28
    Widaman KF, Reise SP. Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In: The science of prevention: Methodological advances from alcohol and substance abuse research. Washington, DC, US: American Psychological Association; 1997. p. 281-324. doi: 10.1037/10222-009.
    » https://doi.org/10.1037/10222-009
  • 29
    Asociación Médica Mundial. Declaración de Helsinki. 2018 [Consultado el 21 de mayo de 2022]. Disponible en: https://www.wma.net/es/policies-post/declaracion-de-helsinki-de-la-amm-principios-eticos-para-las-investigaciones-medicas-en-seres-humanos/
    » https://www.wma.net/es/policies-post/declaracion-de-helsinki-de-la-amm-principios-eticos-para-las-investigaciones-medicas-en-seres-humanos/
  • 30
    Chen F, Curran PJ, Bollen KA, Kirby J, Paxton P. An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models. Sociological Methods & Research [Internet]. 2008;36(4):462-94. doi: 10.1177/0049124108314720.
    » https://doi.org/10.1177/0049124108314720
  • 31
    Rigdon EE. CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal. 1996;3(4):369-79. doi: 10.1080/10705519609540052.
    » https://doi.org/10.1080/10705519609540052
  • 32
    COnsensus-based Standards for the selection of health Measurement INstruments. COSMIN. 2021 [Consultado el 18 de marzo de 2022]. I'm conducting a study on measurement properties. Disponible en: https://www.cosmin.nl/
    » https://www.cosmin.nl/
  • 33
    Ventura-León JL, Caycho-Rodríguez T. El coeficiente Omega: un método alternativo para la estimación de la confiabilidad. Rev Latinoam Cienc Soc Niñez Juv. 2017;15(1):625-7.
  • 34
    Bolgeo T, Di Matteo R, Simonelli N, Dal Molin A, Lusignani M, Bassola B, et al. Psychometric properties and measurement invariance of the 7-item General Anxiety Disorder scale (GAD-7) in an Italian coronary heart disease population. J Affect Disord. 2023;334:213-9. doi: 10.1016/j.jad.2023.04.140.
    » https://doi.org/10.1016/j.jad.2023.04.140
  • 35
    Zhang C, Wang T, Zeng P, Zhao M, Zhang G, Zhai S, et al. Reliability, Validity, and Measurement Invariance of the General Anxiety Disorder Scale Among Chinese Medical University Students. Front Psychiatry. 2021;12. doi: 10.3389/fpsyt.2021.648755.
    » https://doi.org/10.3389/fpsyt.2021.648755
  • 36
    Shevlin M, Butter S, McBride O, Murphy J, Gibson-Miller J, Hartman TK, et al. Measurement invariance of the Patient Health Questionnaire (PHQ-9) and Generalized Anxiety Disorder scale (GAD-7) across four European countries during the COVID-19 pandemic. BMC Psychiatry. 2022;22(1):154. doi: 10.1186/s12888-022-03787-5.
    » https://doi.org/10.1186/s12888-022-03787-5

  • Funding.

    Self-funded.

  • Cite as.

    Cjuno J, Villegas-Mejía RA, Coronado-Fernández J. Psychometric analysis and adaptation of the generalized anxiety questionnaire (GAD-7) to the Collao Quechua language in Peru. Rev Peru Med Exp Salud Publica. 2024;41(2):121-8. doi: 10.17843/rpmesp.2024.412.13373.

Publication Dates

  • Publication in this collection
    19 Aug 2024
  • Date of issue
    Apr-Jun 2024

History

  • Received
    18 Oct 2023
  • Accepted
    24 Apr 2024
Instituto Nacional de Salud Lima - Lima - Peru
E-mail: revmedex@ins.gob.pe