Abstract

INTRODUCTION

Good quality questionnaires are prerequisite to validity of study results.⁵5. Bjorner JB, Olsen J. Questionnaires in epidemiology. In: Olsen J, Saracci R, Trichopoulos D, editors. Teaching epidemiology: a guide for teachers in epidemiology, public health and clinical medicine. 3.ed. New York: Oxford University Press; 2010. p.93-103.Information from general objective questions (e.g., smoke/do not smoke) or questions on abstract concepts (e.g., job stress), collected using rating scales, can only be captured using a questionnaire as compared with assessments or measurements. A questionnaire can provide information on, for example, psychosocial aspects of childhood and adolescence or the respondent's perceived health. Understanding causal mechanisms of health outcomes as complex and comprehensive and determinants that are not closely related to the outcome of interest (e.g., parental education) have gained momentum in the investigation of chronic diseases.

The questionnaire is a key component of the Brazilian Longitudinal Study for Adult Health (ELSA-Brasil). The relationships of its contents and results of assessments and measurements can substantially contribute to understanding factors that may cause, aggravate or protect against cardiovascular diseases and type 2 diabetes in the Brazilian context. Potential variability of results between the six ELSA investigation centers in different Brazilian regions may improve the original contribution of the study. Bearing in mind distinctive characteristics of the Brazilian scenario, we opted to focus on social determinants of health. Capturing the potential impact of some factors (e.g., socioeconomic status), social and racial inequalities and health-related living conditions on outcomes of interest over the course of life may make the difference between what is already known and what the ELSA findings can add to scientific knowledge.¹1. Aquino EM, Barreto SM, Benseñor IM, Carvalho MS, Chor D, Duncan BB, et al Brazilian Longitudinal Study of Adult Health (ELSA-Brasil): objectives and design. Am J Epidemiol. 2011;175(4):315-24. DOI:10.1093/aje/kwr294
https://doi.org/10.1093/aje/kwr294...

In the following sections we describe the work of the ELSA Exposures Committee (EEC) advising and collaborating with the ELSA Research Steering Committee (ERSC) in developing the questionnaire. The development of questionnaires for different study designs has common steps such as selection of topics and pilot testing. But a longitudinal design requires a different approach from baseline. Interview duration is a common concern in all designs, but it is particularly important when subject participation and retention over many years are vital to validity of results. In the ELSA, some topics were considered less relevant and were left out to be included in the second round of data collection (wave 2) from 2012. The content of the questionnaire for wave 2 was determined based on criteria for covering the same topics included in the questionnaire for wave 1: significant changes in frequency and intensity of exposure (e.g., 30% of subjects moved after wave 1 of the study with an impact on the assessment of living conditions) and required exposure status update. In other circumstances, a topic was reassessed with a different instrument (e.g., job stress). All these issues have to be examined to determine the inclusion of new topics that were not investigated in wave 1. Therefore, there should be a balance between concerns of subject retention, affected by the length of wave 2, and investigation of all relevant exposures.

DATA COLLECTION STRATEGY

One of the first decisions of the ERSC was to conduct data collection through interviews and a self-administered questionnaire. Both approaches have been applied in a preliminary study with a sample of 144 subjects (24 in each ELSA study site). Comparisons were made between answers to key questions (e.g., smoking and income), surveys with participants were conducted, and interviewers' comments were examined. Based on these findings, we opted to use in the study face-to-face interviews.

SELECTION OF TOPICS

Selection of topics is usually a major challenge in questionnaire development. Many times some questions included in questionnaires are never analyzed, wasting time, and human (e.g., interviewer training, interview work) and financial resources.

For selecting topics to be investigated we took into consideration the knowledge available on the events of interest, comparability with similar studies, the complex network of causal study outcomes and interview duration. Information from a wide variety of sources was assessed to ensure that interrelationships between different dimensions associated to these outcomes, mediating factors, effect modifiers, and potential confounders would be investigated.

The work involved face-to-face workshops, EEC teleconferences, consultations with experts, and reference to questionnaires and manuals of major international epidemiological.⁴4. Bild DE, Bluemke DA, Burke GL, Detrano R, Diez-Roux AV, Folsom AR, et al Multi-ethnic study of atherosclerosis: objectives and design. Am J Epidemiol. 2002;156(9):871-81. DOI:10.1093/aje/kwf113
https://doi.org/10.1093/aje/kwf113... ^{, 8}8. Clarke R, Breeze E, Sherliker P, Shipley M, Youngman L, Fletcher A, et al Design, objectives, and lessons from a pilot 25 year follow up re-survey of survivors in the Whitehall study of London Civil Servants. J Epidemiol Community Health. 1998;52(6):364-9. ^{, 22}22. The ARIC investigators. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol.1989;129(4):687-702.

After selecting a number of topics and instruments (Table 1) the ERSC developed a "Quality Control Form" (Table 2) to systematize information regarding each potential subject to be included in the ELSA questionnaire for assisting them with decision making. This information consisted of rationale, main characteristics, quality of measuring instruments, and key references on the topic. These forms listed all topics assessed - either they were included or not in the study questionnaire - and were key to the questionnaire development. This inventory is a valuable lesson learned to be shared with other researchers.

Some topics were included by consensus of the EEC and ERSC as they were classic subjects of the epidemiology of cardiovascular diseases and diabetes (e.g., family history of disease, previous medical diagnoses). Other more recent topics (e.g., job stress) were included because of the type of cohort population (workers) as well as evidence showing its association with the outcomes of interest. General topics such as self-rated health, reproductive health and access to health services, more general, were also consensually included.

The decision to give priority to social determinants of health among exposures investigated in the ELSA was also consensual. Not only because of the Brazilian background, but because they are novel exposures less investigated in other cohort studies. In addition to the classic variables associated to socioeconomic conditions - education and income - other relevant aspects of childhood (e.g., parental education and occupation) were included, which gives a life course approach to the study of chronic disease epidemiology. Location information (e.g., city, neighborhood) were collected at key time points (e.g., birth, early school life) to complement information on current area of residence for georeferencing. In addition to social capital measures²³ and social class categorization,¹⁰10. Goldthorpe JH, Llewellyn C, Payne C. Social mobility and class structure in modern Britain. Oxford: Clarendon Press;1987.questions on health-related environment characteristics (neighborhood variables)¹⁸18. Sampson RJ, Raudenbush SW, Earls F. Neighborhoods and violent crime: a multilevel study of collective efficacy. Science. 1997;277(5328):918-24. DOI:10.1126/science.277.5328.918
https://doi.org/10.1126/science.277.5328... are included in a questionnaire for the first time in Brazil. ¹⁹19. Santos SM, Griep RH, Cardoso LO, Alves MGM, Fonseca MJM, Giatti L, et al. Adaptação transcultural e confiabilidade de medidas de características autorreferidas de vizinhança no ELSA-Brasil. Rev Saude Publica. 2013;47(Supl 2):122-30.

The inclusion of instruments for assessment of cognitive function is exceptional. Despite its relevance, the assessment of cognitive function requires the administration of memory (learning and word retention), language (verbal fluency tests), and executive function tests (trail-making test) that are relatively complex and time-consuming, a reason to hesitate to include them in wave 1. But we pondered that it might allow to examine cognitive function cross-sectionally and longitudinally in different birth cohorts especially in those over 35 years of age, which is uncommon in similar studies, and to investigate its relationship with cardiovascular diseases and diabetes. Following intense debate, the assessment of cognitive function was included in the ELSA.³3. Bertolucci PHF, Okamoto IH, Brucki SMD, Siviero MO, Toniolo Neto J, Ramos LR. Applicability of the CERAD neuropsychological battery to Brazilian elderly. Arq Neuropsiquiatr. 2001;59(3A):532-6. DOI:10.1590/S0004-282X2001000400009
https://doi.org/10.1590/S0004-282X200100...

Regarding other topics, the discussion was about what approach to take and instrument to use. For instance, as for food intake, it was defined that our main goal was to assess food consumption patterns. Also, we were aware there were no ideal gold standards to determine the validity of instruments of food intake data collection. We opted for the food frequency questionnaire (FFQ) because it provides information on regular dietary intake (food and nutrients) and allows classification of individuals into intake categories and has been widely used in studies similar to the ELSA. The FFQ, that was validated by Sichieri & Everhart (1998),²⁰20. Sichieri R, Everhart J. Validity of a Brazilian food frequency questionnaire against dietary recalls and estimated energy intake. Nutr Res. 1998;18(10):1649-59. DOI:10.1016/S0271-5317(98)00151-1
https://doi.org/10.1016/S0271-5317(98)00... was used compared to the 24-hour recall. It was found Pearson coefficients of correlation ranging between 0.55 and 0.18 for calcium and vitamin A, respectively.²⁰20. Sichieri R, Everhart J. Validity of a Brazilian food frequency questionnaire against dietary recalls and estimated energy intake. Nutr Res. 1998;18(10):1649-59. DOI:10.1016/S0271-5317(98)00151-1
https://doi.org/10.1016/S0271-5317(98)00... In the ELSA-Brasil validity study, the modified FFQ was compared to three food records. Most coefficients of intraclass correlation were higher than 0.35, and ranged between 0.72 and 0.20 for calcium and selenium (unpublished results).

The inclusion of common non-psychotic mental disorders (CMD) - depression and anxiety - as exposures was based on evidence suggesting an association between CMD and cardiovascular diseases and diabetes.¹²12. Jain S, Mills PJ, Von Känel R, Hong S, Dimsdale JE. Effects of perceived stress and uplifts on inflammation and coagulability. Psychophysiology. 2007;44(1):154-60. DOI:10.1111/j.1469-8986.2006.00480.x
https://doi.org/10.1111/j.1469-8986.2006... ^{, 17}17. Rosengren A, Hawken S, Ounpuu S, Sliwa K, Zubaid M, Almahmeed WA, et al Association of psychosocial risk factors with risk of acute myocardial infarction in 11119 cases and 13648 controls from 52 countries (the INTERHEART study): case-control study. Lancet. 2004;364(9438):953-62. DOI:10.1016/S0140-6736(04)17019-0
https://doi.org/10.1016/S0140-6736(04)17... Similar to what happened with food intake, the discussion was about which measure instrument to use. There are short instruments such as the 12-item General Health Questionnaire-12 (GHQ-12)²⁴24. Werneke U, Goldberg DP, Yalcin I, Ustün BT. The stability of the factor structure of the General Health Questionnaire. Psychol Med. 2000;30(4):823-9.and the Self-Reporting Questionnaire (SRQ-20)²⁶26. World Health Organization, Division of Mental Health. A user's guide to Self-Reporting Questionnaire (SRQ). Geneva; 1994.that can assess whether subjects are non-cases or probable CMD cases, but they do not allow to diagnose the type of disorder. However, the Clinical Interview Schedule-Revised (CIS-R)¹³13. Lewis G, Pelosi AJ, Araya R, Dunn G. Measuring psychiatric disorder in the community: a standardized assessment for use by lay interviewers. Psychol Med. 1992;22(2):465-86.follows the International Statistical Classification of Diseases and Related Health Problems-Tenth Revision criteria to classify subjects into six categories of disorders - generalized anxiety disorder, mixed anxiety-depressive disorder, depression, phobias, obsessive-compulsive disorder, and panic disorder - and is less subject to sociocultural bias than the GHQ-12 and the SRQ-20.²2. Araya R, Wynn R, Lewis G. Comparison of the two self administered psychiatric questionnaires (GHQ-12 and SRQ-20) in primary care in Chile. Soc Psychiatry Psychiatr Epidemiol. 1992;27(4):168-73. ^{, 14}14. Ludemir AB, Lewis G. Investigating the effect of demographic and socioeconomic variables on misclassification by the SRQ-20 compared with a psychiatric interview. Soc Psychiatry Psychiatr Epidemiol. 2005;40(1):36-41. DOI:10.1007/s00127-005-0840-2
https://doi.org/10.1007/s00127-005-0840-... CIS-R is limited by the fact that it is lengthy and requires good training to minimize systematic or random errors²⁰ and to address sensitive situations (e.g., suicidal thoughts). The CIS-R was chosen to be administered in the ELSA because of CMD growing relevance in Brazil and its advantages over other instruments. It was retranslated into Brazilian Portuguese for the ELSA.¹⁶16. Nunes MA, Alves MGM, Chor D, Schmidt MI, Duncan BB. Adaptação transcultural do CIS-R (Clinical Interview Schedule- Revised version) para o português no Estudo Longitudinal de Saúde do Adulto (ELSA). Rev HCPA. 2011;31(4):515-8.

Given the ample scope of topics (Table 2) and the questionnaire length - administered in two stages lasting 40 minutes and 2 hours, respectively - topics such as quality of life were excluded. ELSA's longitudinal design will allow to including this and other topics in subsequent waves of data collection.

TRANSLATION AND ADAPTATION OF MEASURING INSTRUMENTS

Once topics and related instruments were chosen to be included in the ELSA-Brasil questionnaire, we followed standard guidelines¹¹11. Herdman M, Fox-Rushby J, Badia X. A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Qual Life Res. 1998;7(4):323-35. to translate some instruments into Brazilian Portuguese including neighborhood environment rating scales and CIS-R or adapt them (e.g., FFQ) to the cohort population and food intake changes in Brazil over time.

Environment neighborhood rating scales

There was no Portuguese translation available of the rating scales used in the Project on Human Development in Chicago Neighborhoods (PHDCN)18 and the Multi-Ethnic Study of Atherosclerosis (MESA).15 These scales evaluate different neighborhood environment dimensions, for example, availability of public spaces for physical activities and access to fresh fruits and vegetables, which have potential impact on cardiovascular diseases. They were selected because of their ease of administration, very good reproducibility of the English version, and potential comparability of results.

The scales were cross-culturally adapted following Herdman et al (1998) recommendations:11 translation of the original English version into Brazilian Portuguese; retranslation (Portuguese version was translated into English by a second translator) with process and result evaluation; probing; and pretesting. Furthermore, a test-retest reliability study was conducted in a subsample of 261 volunteers to evaluate reproducibility of the Portuguese scales. The intraclass correlation coefficients for each scale were: social cohesion 0.83 (95% confidence interval {95%CI} 0.78;0.87); availability of healthy foods 0.89 (95%CI 0.86;0.91); open space for physical activities 0.90 (95%CI 0.87;0.92); safety 0.86 (95%CI 0.82;0.89); and perceived violence: 0.87 (95%CI 0.84;0.90). ¹⁹19. Santos SM, Griep RH, Cardoso LO, Alves MGM, Fonseca MJM, Giatti L, et al. Adaptação transcultural e confiabilidade de medidas de características autorreferidas de vizinhança no ELSA-Brasil. Rev Saude Publica. 2013;47(Supl 2):122-30.

Clinical Interview Schedule-Revised version (CIS-R)

Many (though not all) sections of the CIS-R had been translated/adapted into Brazilian Portuguese but for use in hospital settings.⁶6. Botega NJ, Zomignani MA, Garcia Junior C, Bio MR, Pereira WAB. Morbidade psiquiátrica no hospital geral: utilização da edição revisada da 'Clinical Interview Schedule CIS-R'. Rev ABP-APAL. 1994;16(2):57-62.We opted for adapting all sections of the original scale version using a standard approach¹¹11. Herdman M, Fox-Rushby J, Badia X. A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Qual Life Res. 1998;7(4):323-35. for the ELSA population, which have a quite different profile from that of inpatients.

The adaptation¹⁶16. Nunes MA, Alves MGM, Chor D, Schmidt MI, Duncan BB. Adaptação transcultural do CIS-R (Clinical Interview Schedule- Revised version) para o português no Estudo Longitudinal de Saúde do Adulto (ELSA). Rev HCPA. 2011;31(4):515-8. comprised steps similar to those described in the adaptation of environment neighborhood rating scales and discussion with experts for final adjustments.

Food Frequency Questionnaire (FFQ)

The FFQ original version was adapted to the population of Rio de Janeiro²⁰ and of five other cities where ELSA study sites are located. A preliminary study was conducted to identify food items to be added to the original list and assess the suitability of household measures (a cup, a ladle) in the light of changes in portion sizes in Brazil over recent decades.

A pilot study of a 24-hour dietary food recall (24HFR)²⁵25. Willett WC. Nutritional epidemiology. 2.ed. New York: Oxford University Press; 1998. was carried out at each study site (total of six 24HFRs) in individuals with age, gender, and education similar to the ELSA population. Based on 24HFR results up to two regional food items or regional marker foods from each ELSA site/city were included (e.g., mate, acarajé {black-eyed pea fritter}, cheese bread, São Paulo couscous, Espírito Santo stew). In addition, a set of standard-size tools was used in all ELSA investigation centers (Figure).

THEME BLOCK SEQUENCING

One of the challenges in developing structured questionnaires is arranging questions so as to make them consistent, facilitate event recall, and minimize losses or refusals.⁷7. Bowling A. Research methods in health: investigating health and health services. Maidenhead: Open University Press; 1998.

Questions in the ELSA questionnaire were sequenced taking into consideration that the first half would last 30 to 45 minutes and be administered at the workplace (Stage 1) and the second more extensive half would last 120 to 180 minutes and be administered at the investigation centers along with assessments and measurements (Stage 2).

Stage 1 questions were designed to facilitate empathy between interviewer and respondent and be simple and easy to answer so that respondents would be willing to participate in the study. Given it is a health survey for adults, the opening questions of the questionnaire were about personal and family history of disease. They were intended at the same time to collect data that would allow to defining the profile of losses compared to cohort subjects in case of withdrawal or loss during follow-up. Thus, questions on sociodemographic and family aspects and health-related behaviors and occupational history were also included.

Stage 2 questions asked about specific health problems (e.g., headache and intermittent claudication). These questions were followed by cognitive function testing that required subjects to be rested, and by many other assessments (Table 2). The CMD assessment scale (CIS-R) as well as other scales were applied uninterruptedly after a trusting relationship was established between interviewer and respondent to minimize losses and refusals. The FFQ was administered right after a snack was offered following the second blood collection because it was not advisable to recall food consumption while the subject was fasting or after drinking the solution for glucose tolerance test. The closing questions of the interview were about experiencing social discrimination because it is a sensitive issue.

For constructing the flowchart alternating assessments and questions of the interview, we sought to ensure efficiency by optimizing the utilization of rooms and human resources and the same interviewer whenever possible. But, above all, we tried to provide the most comfortable and less exhausting interaction between respondents and interviewers to ensure quality of data collected.

PRETESTING AND PILOT STUDIES

Pretesting and pilot studies are designed to reduce the likelihood of measurement errors1 by checking if the questions are clear, the respondent's response to sensitive questions, adequacy of answer choices, "skip" of questions required when the respondent do not have to answer certain question, and the sequence of theme blocks. In addition, as ELSA is a multicenter study, specific aspects that characterize the population of each participating city were only identified in these "trials"20 (for example, the inclusion of synonymous food items in the FFQ).

Specific sections of the questionnaire draft were successively pretested at the six ELSA investigation centers. Three rounds of concurrent pretesting were then carried out, totaling 73 interviews with outsourced employees of participating institutions or servers of other public institutions with age, gender and education profiles similar to those of ELSA population.

Questions, comments and praise of volunteers and interviewers on the questionnaire structure, arrangement and administration were reviewed and any changes were incorporated into the different versions of the questionnaire. For example, shift work schedule and which kinship ties should be considered as "family members" in the social capital scale required to be clearly defined.

It also became evident after pretesting that detailed guidelines should be provided in the Interview Guide. Interviewers were trained to give cues in a consistent manner to help the respondent recall information, for example, age at menarche, using the following question: "Do you remember in which school grade you were when you had your period for the first time?". Another helpful guidance was on the intonation of "not" in questions that remained negative in the translation to comply with the original English questionnaire (e.g., "People in your neighborhood generally DO NOT get along with each other.").

Pretests also suggested that answer cards should be handed to the respondent for rating scales with similar choices in different questions and in those questions for which there was an extensive list of choices, as in CIS-R.

Rounds of questionnaire pretesting were carried out until all problems were identified and corrected. Then a pilot study with a three-stage strategy and a "full-scale rehearsal"9,21 were simultaneously conducted at the six ELSA investigation centers. New assessments and measurements were added at each new stage of the pilot study. A simulation of the real interview including all study procedures in sequence was carried out in a sample of 360 volunteers who had characteristics similar to the pretesting population. All volunteers signed a simplified free informed consent form to comply with ethics concerns in all stages of ELSA preparation.

The first pilot study was conducted in June 2008. It included 12 subjects who were outsourced employees of participating institutions or public servers of other public institutions. There were conducted only actions of stage 1 of the study - recruitment and part I of the interview - at the workplace, which allowed to assess privacy for the interview and estimated duration of this stage.

The second pilot study (n = 30) included the study assessments and measurements and part II of the interview to be conducted during the subject's visit to the study site. The third pilot study was carried out in August 2008 with 107 subjects. All procedures that were part of the study visit were tested. We sought to identify uncomfortable situations, for example, administering the questionnaire while the subject was fasting.

A major goal of the pilot study was to test the interview flow including assessments, sections of the questionnaire, and measurements (pace and duration) as it would certainly have an impact on a subject's willingness to participate, quality of data provided, and participation of his/her colleagues (word of mouth was the strategy used to advertise the study). At this stage, volunteers were actively asked to give their impressions and suggestions. This feedback was recorded and was crucial for improving the flow, alternating interview sections, clinical and biological assessments, and snacks.

The pilot studies also assessed interviewer workload. Short breaks between each section of the interview were allowed and, in Rio de Janeiro, training was provided to improve verbal communication and prevent vocal cord conditions.

Feedback and comments from each stage of the pilot studies were summarized in a report at each study site and forwarded to the EEC and Quality Control Committee. All relevant changes were reviewed and implemented by the ERSC. Pretesting and pilot studies were key for greater internal validity of results through standardization of interviews in the six investigation centers. They also facilitated fieldwork as most issues were addressed and helped build team confidence.

FINAL CONSIDERATIONS

It is not an easy task to develop a survey questionnaire to collect valid information. Like in many studies, the ELSA had a permanent supervision team to review unsuspected interpretations of questions and answer choices. Learning from experience can be more enriching than learning from books or manuals, and all those involved in the development and quality control of the questionnaire and interviews have learned a great deal. In this article we shared our experience with readers to contribute to the improvement of epidemiological research in Brazil.

References

Publication Dates

History