Abstract
OBJECTIVE:
To estimate the test-retest reliability of items of the Resource Generator scale for assessing social capital in the Brazilian Longitudinal Study for Adult Health (ELSA-Brasil).
METHODS:
The social capital was applied in a subsample of 281 participants from six ELSA investigation centers, on two occasions with an interval of seven to 14 days. The instrument consists of 31 items that represent concrete situations to evaluate the access to different types of resources. In addition, it evaluates the strength of ties (family, friends or acquaintances) for the available resources. Statistical analyses were performed through use of the kappa statistic (k) and prevalence-adjusted kappa (ka).
RESULTS:
A high frequency was found for social resources (above 50%). Regarding the presence or absence of resources, prevalence-adjusted reliability (ka) varied from 0.54 to 0.97. With regard to the source for the resource, the reliability estimates ranged from ka = 0.45 ("someone who has good contacts with the media") to ka = 0.86 ("someone who completed secondary education").
CONCLUSIONS:
The scale presentedadequate levels of reliability, which varied according to the type of resource.
Questionnaires, utilization; Reproducibility of Results; Validity of Tests; Validation Studies as Topic; Social capital
INTRODUCTION
In the literature, there are various definitions of social capital and it has been measured at both a collective and individual level.10 It is multidimensional and includes trust, social norms and reciprocity in access to resources (which may be, material, emotional or informational), through networks of relationships.1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29. According to Lin (1999),13 social capital is composed of three components: the existence of a network, the individual's involvement in the network and the availability of resources for those participating in the network. Relationships which are rich in social capital may offset the disadvantage of a lack of privately owned goods.14 Therefore, in more egalitarian societies, with adequate provision of public requirements and social welfare, social capital may be less important for the health of its population than in societies with high socioeconomic inequality.99. Islam MK, Merlo J, Kawachi I, Lindström M, Burström K, Gerdtham UG. Does it really matter where you live? A panel data multilevel analysis of Swedish municipality-level social capital on individual health-related quality of life. Health Econ Policy Law. 2006;1(Pt 3):209-35. DOI:10.1017/S174413310600301X
https://doi.org/10.1017/S174413310600301... , 1414. Lin N. Inequality in social capital. Contemp Sociol. 2000;29(6):785-95.
Social capital can be measured using complex mapping of name identification (Name Generator): of social position relating to high-status professions (Position Generator) of those who make up the social network: or even of the perception of the availability of different resources in the individual's social network (Resource Generator).1313. Lin N. Building a network theory of social capital. Connections. 1999;22(1):28-51. , 1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29.
The Resource Generator scale identifies the type of resource available, as well as the degree of proximity (family, friend of acquaintance) of the person who may be able to provide the resource in case of need. In contrast to other measures of social capital, and aiming to overcome their limitations, this scale does not include complete mapping of the network's components, significantly reducing interview time. Moreover, it refers to the various sources of sources of access to social capital, instead of being restricted to just the status of the occupations.18 It is a new instrument, used in the investigation of links between access to social capital and health care outcomes.1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29. , 2020. Webber MP, Huxley PJ, Tirril H. Social capital and the course of depression: six-month prospective cohort study. J Affect Disord. 2011;129(1-3):149-57. DOI:10.1016/j.jad.2010.08.005
https://doi.org/10.1016/j.jad.2010.08.00...
Associations between social capital and common types of mental illness,22. Aslund C, Starrin B, Nilsson KW. Social capital in relation to depression, musculoskeletal pain, and psychosomatic symptoms: a cross-sectional study of a large population-based cohort of Swedish adolescents. BMC Public Health. 2010;10:715. DOI:10.1186/1471-2458-10-715
https://doi.org/10.1186/1471-2458-10-715... , 1212. Lin N, Ye X, Ensel WM. Social support and depressed mood: a structural analysis. J Health Soc Behav. 1999;40(4):344-59. , 1919. Webber MP, Huxley PJ. Measuring access to social capital: the validity and reliability of the Resouce Generator-UK and its association with common mental disorder. Soc Sci Med. 2007;65(3):481-92. DOI:10.1016/j.socscimed.2007.03.030
https://doi.org/10.1016/j.socscimed.2007... depression,66. Dutt K, Webber M. Access to social capital and social support among South East Asian women with severe mental health problems: a cross-sectional survey. Int J Soc Psychiatry. 2010;56(6):594-605. DOI:10.1177/0020764009106415
https://doi.org/10.1177/0020764009106415... , 2020. Webber MP, Huxley PJ, Tirril H. Social capital and the course of depression: six-month prospective cohort study. J Affect Disord. 2011;129(1-3):149-57. DOI:10.1016/j.jad.2010.08.005
https://doi.org/10.1016/j.jad.2010.08.00... self-reported health conditions,77. Fujiwara T, Kawachi I. Social capital and health: a study of adult twins in the U.S. Am J Prev Med. 2008;35(2):139-44. DOI:10.1016/j.amepre.2008.04.015
https://doi.org/10.1016/j.amepre.2008.04... fruit and vegetable consumption,1717. Sorensen G, Stoddard AM, Dubowitz T, Barbeau EM, Bigby J, Emmons KM, et al. The influence of social context on changes in fruit and vegetables consumption: results of the healthy directions studies. Am J Public Health. 2007;97(7):1216-27. DOI:10.2105/AJPH.2006.088120
https://doi.org/10.2105/AJPH.2006.088120... alcohol consumption and smoking,5 obesity and diabetes,88. Holtgrave DR, Crosby R. Is social capital a protective factor against obesity and diabetes? Findings from an exploratory study. Ann Epidemiol. 2006;16(5):406-8. DOI:10.1016/j.annepidem.2005.04.017
https://doi.org/10.1016/j.annepidem.2005... and mortality rates for coronary disease1616. Rutledge T, Reis SE, Olson M, Owens J, Kelsey S, Pepine C, et al. Social networks are associated with lower mortality rates among women with suspected coronary disease: The National Heart, Lung, and Blood Institute-Sponsored Women's Ischemia Syndrome Evaluation Study. Psychosom Med. 2004;66(6):882-8. DOI:10.1097/01.psy.0000145819.94041.52
https://doi.org/10.1097/01.psy.000014581... have already been found.
Of the social determinants of health, social capital is one which is of interest to the Brazilian Longitudinal Study for Adult Health (ELSA-Brasil). The Brazilian version of the Resource Generator Scale was included in the questionnaire for the baseline study. The instrument was developed in Holland18 and contains 33 questions representing practical situations with which to evaluate access to various types of resources. Together, they cover four dimensions of life considered essential to social capital: 1) status and education (e.g. knowing someone who speaks a foreign language); 2) political and financial abilities (e.g., contact with a political activist); 3) personal abilities (e.g., knowing someone who can fix a bicycle); 4) personal support (e.g., someone who can give advice on conflict in the workplace).18 Moreover, it evaluates the strength of the connection between the individuals and members of the network through whom resources could be obtained (family, friends or acquaintances).1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29. , 1919. Webber MP, Huxley PJ. Measuring access to social capital: the validity and reliability of the Resouce Generator-UK and its association with common mental disorder. Soc Sci Med. 2007;65(3):481-92. DOI:10.1016/j.socscimed.2007.03.030
https://doi.org/10.1016/j.socscimed.2007...
In every study, reliability needs to be investigated, especially if the instrument has been translated from another language. This is because it should be measured according to the dynamic at the time of the interview and is an important stage in the psychometric evaluation of instruments used in epidemiology.1515. Reichenheim ME, Moraes CL. Alguns pilares para a apreciação da validade de estudos epidemiológicos. Rev Bras Epidemiol. 1998;1(2):131-48. DOI:10.1590/S1415-790X1998000200004
https://doi.org/10.1590/S1415-790X199800... This article reports the measuring of the test-retest reliability of the items on the scale of social capital resource providers in participants in the ELSA-Brasil.
METHODS
Study and sample design
During the ELSA-Brasil interviews and tests, the interviewees responded to parts of the questionnaire again, including the social capital scale, for which test-retest reliability is to be estimated. The study interview was considered to be "test" and then the interviewees responded to the same questionnaire again (retest), with the same interviewer, seven to 14 days after the first interview.
The parameters necessary for calculating the sample size were based on the study by Weber & Huxley (2007),2020. Webber MP, Huxley PJ, Tirril H. Social capital and the course of depression: six-month prospective cohort study. J Affect Disord. 2011;129(1-3):149-57. DOI:10.1016/j.jad.2010.08.005
https://doi.org/10.1016/j.jad.2010.08.00... using the sskdlg routine of STATA software (version 10). The expected value of kappa for the items was between 0.67 and 0.75,2020. Webber MP, Huxley PJ, Tirril H. Social capital and the course of depression: six-month prospective cohort study. J Affect Disord. 2011;129(1-3):149-57. DOI:10.1016/j.jad.2010.08.005
https://doi.org/10.1016/j.jad.2010.08.00... the proportion of positive results expected in the first and second interviews was expected to be 0.25 and two-tailed alpha error of 0.05. The minimum sample size was estimated to be 270 participants for the six investigation centers, also considering the strata of functional categories, age groups and sex of the ELSA-Brasil participants.11. Aquino EML, Barreto SM, Bensenor IM, Carvalho MS, Chor D, Duncan BB, et al. Brazilian Longitudinal Study of Adult Health (ELSA-Brasil): objectives and design. Am J Epidemiol. 2012;175(4):315-2. DOI:10.1093/aje/kwr294
https://doi.org/10.1093/aje/kwr294... The final sample for studying the test-retest reliability contained 281 participants from the six Investigation Centers, recruited between November 2009 and November 2010, who agreed to respond to the questionnaire again.
Interview Quality Control measures
During the stage of selecting and training the field researchers, at least 40 hours of theoretical and practical training were carried out. Only those researchers scoring higher than 70% of the expected performance, according to a checklist previously drawn up, were kept on the team. An extensive handbook of interview and questionnaire completion guidelines was drawn up, based on pretests and pilot studies carried out in all of the Investigation Centers before data collection began. The questionnaires applied in the test and the retest were revised twice before data entry: once by the interviewer and a second time by the interview supervisor. The team of interviewers were supported daily by the presence of a supervisor to clear up doubts on filling out the questionnaire.44. Chor, D. Questionário do ELSA-Brasil: desafios na elaboração de instrumento multidimensional. Rev Saude Publica. 2013;47(Supl 2):27-36.
Resource Generator Scale
The English version1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29. was translated into Brazilian Portuguese by a team of researchers from the Instituto Universitário de Pesquisas do Rio de Janeiro. In this version, four questions from the original version were excluded ("Can give advice on matters of law {problems with landlord, boss, municipality}", "Can give medical advice when you are dissatisfied with your doctor", "Has higher vocational education", "Knows about soccer") and two new questions were introduced ("Can talk with you about very important topics" and "You can visit socially {party, barbecue, dance etc.}"). The team of experts (one sociologist and three epidemiologists with experience using scales, with a mastery of English) who evaluated the appropriateness of the items and made small adjustments in the questions forms, made few additional changes to the Portuguese version for the instrument to be included in ELSA. In addition, the item "Do you know someone who can facilitate a hospital admission or test?" was included (Table 1).
The final version of the instrument used in ELSA was composed of 31 items, the response options of which had two stages: 1 - yes or no: 2 - if yes, they were requested to identify the source of the resource using three options: someone from their family, their friend or an acquaintance. These options were presented to the interviewees on a card at the start of the process of completing the questionnaire.
The participant was given the following guidance: 1 - do not consider themselves to be in possession of that resource; 2 - use the standardized definition of 'acquaintance', i.e., "someone with whom the interviewee would stop and talk in the street and whose name they know"; 3 - if they had a family member and a friend and an acquaintance who possessed some kind of resource (for example, a car) they should give the closest, that is, the family member.
Data analysis
Overall stability and stability stratified by sex, age and schooling of the items on the presence of a resource (dichotomized: yes/no) and the source of the resource (family member, friend or acquaintance) were estimated using the kappa statistic (k) and kappa adjusted for prevalence (ka) and for interviewer bias (Prevalence-adjusted and Bias-adjusted kappa - PABAK).33. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423-9. DOI:10.1016/0895-4356(93)90018-V
https://doi.org/10.1016/0895-4356(93)900... Confidence intervals of 95% were estimated using the bootstrap method. In order to classify the degree of concordance, the criteria employed by Webber & Huxley2020. Webber MP, Huxley PJ, Tirril H. Social capital and the course of depression: six-month prospective cohort study. J Affect Disord. 2011;129(1-3):149-57. DOI:10.1016/j.jad.2010.08.005
https://doi.org/10.1016/j.jad.2010.08.00... adapted from Landis & Koch1111. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74. were used: excellent: > 0.74; good: 0.59 to 0.74; moderate: 0.40 to 0.58; and poor: < 0.40. Confidence intervals of 95% were estimated for all of the statistics. The R program was used to calculate these statistics.
The questionnaires were revised and codified in a standardized way. Data were entered twice, independently, using the EpiInfo program, with inconsistencies corrected using the "validate" subprogram.
RESULTS
Around half of the participants in the study were male; 15.3% were aged between 36 and 44; 37.4% between 45 and 54; 35.2% between 55 and 64 and 12.1% between 65 and 74. More than half had university education (54.5%); 31% had finished high school and 13.5% primary education.
The percentage of participants who reported knowing someone able to cooperate with the resources in question was high for the majority of items. Some items were less frequent, for example, "know someone who could arrange a temporary job for a family member" (51.6%) and "has good contacts in the media" (57.7%). On the other hand, there were items with a very high frequency, such as "someone who has a car" (100%) and someone who finished high school" (98.9%) (Table 2). For these items, family was the most frequently cited source. For items related to contacts" in the media, and advice about work and the workplace, friends were the most commonly cited source. Acquaintances were the most common source in repairing cars and bicycles, being a political activist and having more than R$ 2,000.00 in savings.
Test-retest, description of the items and the link with the provider of the social capital resource (Resource Generator Scale) in ELSA-Brazil.
Estimates of reliability varied from k = 0.24 to k = 0.68 in relation to the presence or absence of the resource (Table 3). Lower estimates were identified for "someone to help move house" (k = 0.24 and "someone to provide a job reference" (k = 0.27; and higher for "has more than R$ 2,000.00 in savings" (k = 0.68) and "someone who has good contacts in the media (k = 0.67) (Table 3). When adjusted for prevalence, stability varied between ka = 0.54 (someone who could arrange a temporary job for a family member) and ka = 0.97 (someone who finished high school). With regards to the source of the resources, estimates of reliability varied between k = 0.35 (someone who finished high school) to k = 0.68 (someone who plays a musical instrument). When adjusted for prevalence, the values for reliability varied between ka = 0.45 (someone who has good contacts in the media) to ka = 0.86 (someone who finished high school). No differences were observed in stratification by sex, age and schooling (data not shown).
DISCUSSION
The levels of test-retest reliability of the items on the Resource Generator proved to be adequate, varying according to the type of resource. The availability of the resources in question was high for the majority of items, as was to be expected given the characteristics of the study population (state employees in research and education facilities). The study with the Dutch population showed similar results.1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29.
In addition to measuring access, other objectives of the scale are to capture the nature of the network and the strength of the connection, investigating the relationship between the participant and the resource provider - family member, friend or acquaintance. With this information, it is possible to know the "distance" between the participant and the resource. It is acknowledged that there is a greater possibility of more distant connections (friends or acquaintances) providing some typed of resources which are not so frequently needed ("someone who has good contacts in the media"), and that family members are more available to provide every day, more available resources (for example, someone to do the shopping when you are poorly, let you stay with them for a week or take care of the children).1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29. Our results confirmed these expectations,
Incorporating measures which allow the different types of resources which make up social capital to be distinguished is recommended.1010. Kawachi I, Kim D, Coutts A, Subramanian SV. Commentary: reconciling the three accounts of social capital. Int J Epidemiol. 2004;33(4):682-90. DOI:10.1093/ije/dyh177.
https://doi.org/10.1093/ije/dyh177... Thus, the social capital scale included in ELSA-Brasil contributes to increasing understanding of different, valued social resources are achieved, or not, in networks of relationships and how close or distant these resources are from the reach of the individuals.
The test-retest reliability varied from moderate to good for the majority of the items referring to the presence of the resource. Similar results were obtained in the adaption of the English scale, the Resource Generator, the reliability of whose items varied between 0.33 and 0.85.1919. Webber MP, Huxley PJ. Measuring access to social capital: the validity and reliability of the Resouce Generator-UK and its association with common mental disorder. Soc Sci Med. 2007;65(3):481-92. DOI:10.1016/j.socscimed.2007.03.030
https://doi.org/10.1016/j.socscimed.2007... However, in the English study, reliability was classified as excellent for the majority of items and only two of them had poor reliability. In the case of ELSA, four items had poor reliability evaluated using unadjusted kappa. For the majority of these items, these results were partly related to the high frequency of positive responses in this population. Using prevalence- adjusted kappa, recommended for this situation, the reliability of the items in ELSA-Brasil varied between good and excellent for almost all of the items.
Questions regarding the source of the resources (family, friends or acquaintances) showed brute concordance values (k) classified as "moderate". However, when adjusted for prevalence, concordance became good or excellent for the majority of items. Similar results were reported by the authors of the original instrument.1919. Webber MP, Huxley PJ. Measuring access to social capital: the validity and reliability of the Resouce Generator-UK and its association with common mental disorder. Soc Sci Med. 2007;65(3):481-92. DOI:10.1016/j.socscimed.2007.03.030
https://doi.org/10.1016/j.socscimed.2007...
Due to operational issues, it was not possible to analyze reliability between interviewers, as it was necessary that the same participant responded to the questionnaire twice with different interviewers each time, who may have been in different cities.
To conclude, adequate levels of temporal stability were estimated varying according to the type of resource. However, the greater difficulty found in applying this scale s in understanding what is useful in each social network in different contexts, meaning that local adaptations are necessary.1818. Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29. Thus, complementary psychometric evaluations of the instrument are underway in the context of the study population of ELSA-Brasil. At this stage, the validity of the construct is being evaluated, including the dimensional structure and the items belonging to these dimensions. Because of its multi-centric character, regional performance specificities for this scale will also be able to be analyzed.
References
- 1Aquino EML, Barreto SM, Bensenor IM, Carvalho MS, Chor D, Duncan BB, et al. Brazilian Longitudinal Study of Adult Health (ELSA-Brasil): objectives and design. Am J Epidemiol 2012;175(4):315-2. DOI:10.1093/aje/kwr294
» https://doi.org/10.1093/aje/kwr294 - 2Aslund C, Starrin B, Nilsson KW. Social capital in relation to depression, musculoskeletal pain, and psychosomatic symptoms: a cross-sectional study of a large population-based cohort of Swedish adolescents. BMC Public Health 2010;10:715. DOI:10.1186/1471-2458-10-715
» https://doi.org/10.1186/1471-2458-10-715 - 3Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol 1993;46(5):423-9. DOI:10.1016/0895-4356(93)90018-V
» https://doi.org/10.1016/0895-4356(93)90018-V - 4Chor, D. Questionário do ELSA-Brasil: desafios na elaboração de instrumento multidimensional. Rev Saude Publica 2013;47(Supl 2):27-36.
- 5Cohen S, Lemay EP. Why would social networks be linked to affect and health practices? Health Psychol 2007;26(4):410-7. DOI:10.1037/0278-6133.26.4.410
» https://doi.org/10.1037/0278-6133.26.4.410 - 6Dutt K, Webber M. Access to social capital and social support among South East Asian women with severe mental health problems: a cross-sectional survey. Int J Soc Psychiatry 2010;56(6):594-605. DOI:10.1177/0020764009106415
» https://doi.org/10.1177/0020764009106415 - 7Fujiwara T, Kawachi I. Social capital and health: a study of adult twins in the U.S. Am J Prev Med 2008;35(2):139-44. DOI:10.1016/j.amepre.2008.04.015
» https://doi.org/10.1016/j.amepre.2008.04.015 - 8Holtgrave DR, Crosby R. Is social capital a protective factor against obesity and diabetes? Findings from an exploratory study. Ann Epidemiol 2006;16(5):406-8. DOI:10.1016/j.annepidem.2005.04.017
» https://doi.org/10.1016/j.annepidem.2005.04.017 - 9Islam MK, Merlo J, Kawachi I, Lindström M, Burström K, Gerdtham UG. Does it really matter where you live? A panel data multilevel analysis of Swedish municipality-level social capital on individual health-related quality of life. Health Econ Policy Law 2006;1(Pt 3):209-35. DOI:10.1017/S174413310600301X
» https://doi.org/10.1017/S174413310600301X - 10Kawachi I, Kim D, Coutts A, Subramanian SV. Commentary: reconciling the three accounts of social capital. Int J Epidemiol. 2004;33(4):682-90. DOI:10.1093/ije/dyh177.
» https://doi.org/10.1093/ije/dyh177 - 11Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74.
- 12Lin N, Ye X, Ensel WM. Social support and depressed mood: a structural analysis. J Health Soc Behav. 1999;40(4):344-59.
- 13Lin N. Building a network theory of social capital. Connections. 1999;22(1):28-51.
- 14Lin N. Inequality in social capital. Contemp Sociol. 2000;29(6):785-95.
- 15Reichenheim ME, Moraes CL. Alguns pilares para a apreciação da validade de estudos epidemiológicos. Rev Bras Epidemiol. 1998;1(2):131-48. DOI:10.1590/S1415-790X1998000200004
» https://doi.org/10.1590/S1415-790X1998000200004 - 16Rutledge T, Reis SE, Olson M, Owens J, Kelsey S, Pepine C, et al. Social networks are associated with lower mortality rates among women with suspected coronary disease: The National Heart, Lung, and Blood Institute-Sponsored Women's Ischemia Syndrome Evaluation Study. Psychosom Med. 2004;66(6):882-8. DOI:10.1097/01.psy.0000145819.94041.52
» https://doi.org/10.1097/01.psy.0000145819.94041.52 - 17Sorensen G, Stoddard AM, Dubowitz T, Barbeau EM, Bigby J, Emmons KM, et al. The influence of social context on changes in fruit and vegetables consumption: results of the healthy directions studies. Am J Public Health 2007;97(7):1216-27. DOI:10.2105/AJPH.2006.088120
» https://doi.org/10.2105/AJPH.2006.088120 - 18Van der Gaag M, Snijders TAB. The Resource Generator: measuring social capital with concrete items. Soc Networks. 2005;27:1-29.
- 19Webber MP, Huxley PJ. Measuring access to social capital: the validity and reliability of the Resouce Generator-UK and its association with common mental disorder. Soc Sci Med. 2007;65(3):481-92. DOI:10.1016/j.socscimed.2007.03.030
» https://doi.org/10.1016/j.socscimed.2007.03.030 - 20Webber MP, Huxley PJ, Tirril H. Social capital and the course of depression: six-month prospective cohort study. J Affect Disord 2011;129(1-3):149-57. DOI:10.1016/j.jad.2010.08.005
» https://doi.org/10.1016/j.jad.2010.08.005
- Article available from: www.scielo.br/rsp
Publication Dates
- Publication in this collection
June 2013
History
- Received
05 Oct 2011 - Accepted
05 June 2012