Abstract

The proportion of non-participation in cohort studies, if associated with both the exposure and the probability of occurrence of the event, can introduce bias in the estimates of interest. The aim of this study is to evaluate the impact of participation and its characteristics in longitudinal studies. A systematic review (MEDLINE, Scopus and Web of Science) for articles describing the proportion of participation in the baseline of cohort studies was performed. Among the 2,964 initially identified, 50 were selected. The average proportion of participation was 64.7%. Using a meta-regression model with mixed effects, only age, year of baseline contact and study region (borderline) were associated with participation. Considering the decrease in participation in recent years, and the cost of cohort studies, it is essential to gather information to assess the potential for non-participation, before committing resources. Finally, journals should require the presentation of this information in the papers.

Selection Bias; Cohort Studies; Epidemiologic Methods

Resumo

A proporção de não-participação em estudos de coorte está associada também à exposição e à probabilidade de ocorrência do evento poder gerar viés nas estimativas de interesse. O objetivo do presente trabalho é realizar uma revisão sistemática e metanálise de artigos que descrevem a participação em estudos de coorte e avaliar as características associadas à participação. Foi realizada uma revisão sistemática (MEDLINE, Scopus e Web of Science), buscando-se artigos que descrevessem a proporção de participação na linha de base de estudos de coorte. De 2.964 artigos inicialmente identificados, foram selecionados 50. Entre esses, a proporção média de participação foi de 64,7%. Utilizando-se o modelo de metarregressão com efeitos mistos, somente a idade, ano da linha de base e a região do estudo (limítrofe) estiveram associados à participação. Considerando a diminuição na participação em anos mais recentes e o custo dos estudos de coorte, é essencial buscar informações que permitam avaliar o potencial de não-participação antes de comprometer os recursos.

Viés de Seleção; Estudos de Coortes; Métodos Epidemiológicos

Resumen

La proporción de no participación en estudios de cohorte se asocia también con la exposición y probabilidad de ocurrencia de hechos que pueden generar sesgos en las estimaciones de interés. El objetivo de este estudio es realizar una revisión sistemática y un metaanálisis de artículos que describen la participación en estudios de cohortes y evaluar las características asociadas con la participación. Una revisión sistemática fue realizada (MEDLINE, Scopus y Web of Science), en busca de artículos que describen la relación de participación basada en estudios de cohortes. Se seleccionaron 2964 artículos, de los cuales se identificaron preliminarmente 50. Entre estos, la proporción promedio de participación fue de un 64,7%. Utilizando la metarregresión, sólo la edad, años de referencia y la región de estudio (borderline) se asociaron con la participación. Teniendo en cuenta la disminución de la participación en los últimos años, y el coste de los estudios de cohortes, es esencial buscar información para evaluar el potencial de la no participación antes de comprometer recursos.

Sesgo de Selección; Estudios de Cohortes; Métodos Epidemiológicos

Background

Among observational studies, the advantages of prospective cohort studies are that they are able to estimate incidence measures directly and are less vulnerable to information bias. However, participation refusal at baseline or follow-up can introduce selection bias when simultaneously associated with both exposure and the outcome ¹1. Kelsey JL. Methods in observational epidemiology. New York: Oxford University Press; 1996.^,22. Greenland S. Response and follow-up bias in cohort studies. Am J Epidemiol 1977; 106:184-7.. As a result, the association between exposure and outcome may differ between participants and non-participants.

Morton et al. ³3. Morton LM, Cahill J, Hartge P. Reporting participation in epidemiologic studies: a survey of practice. Am J Epidemiol 2006; 163:197-203. observed a tendency for participation in cohort studies to decrease between 1970 and 2003. As the non-participation proportion rises, vulnerability to selection bias tends to increase. Therefore, it is recommended reporting participation proportion in observational studies ⁴4. Stang A. Nonresponse research: an underdeveloped field in epidemiology. Eur J Epidemiol 2003; 18:929-31.,designing methodological studies to evaluate the impacts of non-participation and evaluating study characteristics that may influence participation ⁵5. Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol 2007; 17:643-53..

To the best of our knowledge, and in spite of its importance, no systematic evaluation of participation in observational cohort studies is available to guide choices and scientific assessment of validity of conclusions. This present study aims to perform a systematic review and meta-regression of papers describing non-participation bias in cohort studies, and evaluate the studies’ characteristics associated with participation proportion.

Methods

We performed a systematic review and meta-regression following the methodology proposed by Higgins & Green ⁶6. Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Volume 5. Chichester: Wiley-Blackwell; 2008. and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) criteria ⁷7. Moher D, Liberati A, Tetzlaff J, Altman DG; The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. PLoS Med 2009; 6:e1000097..

Search strategy

We searched MEDLINE, Scopus and Web of Science data bases for papers published between January 1978 and November 2014. The query used for the MEDLINE search strategy was: (cooperation[Title/Abstract/MESH] or noncooperation[Title/Abstract/MESH] or non-cooperation[Title/Abstract/MESH] or participant*[Title/Abstract/MESH] or nonparticipant*[Title/Abstract/MESH] or non-participant*[Title/Abstract/MESH] or compliance[Title/Abstract/MESH] or noncompliance[Title/Abstract/MESH] or non-compliance[Title/Abstract/MESH]) AND bias*[Title/Abstract/MESH] AND (cohort*[Title/Abstract/MESH] OR prospective [Title/Abstract/MESH] OR longitudinal [Title/Abstract/MESH]). For the other data bases, the specific syntaxes corresponding to each base were used.

Article titles and abstracts were evaluated by two reviewers working independently in order to ascertain whether they met the criteria for inclusion in the study. Disagreements were assessed by a third reviewer.

Eligibility criteria and data extraction

As specific populations and health problems may induce large differences in participation proportions related to theses specificities, we only included population-based cohort studies on adult (18 to 75 years old) healthy people. We excluded studies that addressed specific populations (eg. pregnant women, patients with specific ailments), review studies and others (eg. genetic studies, surgery, drug therapies). Figure 1 depicts the review flow chart.

The references identified were stored and processed using the JabRef 2.10 software (http://jabref.sourceforge.net/). We collected the participation proportion, the general characteristics of the study (year of baseline contact, place, selection strategy and study outcome). We also evaluated the characteristics of the study population including type (general population vs. working population), participation of women and the mean age. The relevant data was extracted reading the full paper.

Data analysis

A meta-analysis of participation proportion was conducted using mixed-effects models, often called binominal-normal models ⁸8. De Ridder MAJ, Stijnen T, Hokken-Koelega ACS. A new method to determine mean adult height from incomplete follow-up data. Horm Res 2007; 67:205-10.. Given the heterogeneity of the studies (I2 = 99.97%; τ² = 0.54; p < 0.001), we investigated the variables associated with the participation proportion, initially by simple meta-regression models. When the value of variance accounted for (VAF) by the model was greater than 5%, the variable was included in the multiple model. VAF indicates the percentage of total heterogeneity that is explained by each moderator. The goodness of fit of the multiple model was evaluated by the likelihood ratio test (LRT).

We analyzed the following variables: year of the baseline contact, participant mean age, proportion of women, selection strategy, population type (general population vs. employees population), study outcome – cardiovascular (baseline category), general health or others (cancer, accident, substance use, incapacity and smoking) – and study region, as divided by United Nations Statistics Division ⁹9. United Nations Statistics Division. Composition of macro geographical (continental) regions, geographical sub-regions, and selected economic and other groupings. https://unstats.un.org/unsd/methods/m49/m49regin.htm (accessed on 01/Mar/2013).
https://unstats.un.org/unsd/methods/m49/... into Continental Europe (baseline category), Northern Europe, USA, and Others (Asia or Oceania). Spearman correlation coefficient was used to evaluate the relation between the year of the baseline contact and the participation proportion.

The analyses were performed using the metafor ¹⁰10. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw 2010; 36(3). http://www.jstatsoft.org/v36/i03/paper.
http://www.jstatsoft.org/v36/i03/paper... library of R software (The R Foundation for Statistical Computing, Vienna, Austria; http://www.r-project.org).

Results

Of the 2,964 original papers initially identified, 50 were selected. Figure 1 summarizes the study selection process.

Table 1 describes the objectives, database, analysis and main results of the selected papers. To evaluate participation, 29 (58%) papers compared participants and non-participants using secondary databases, 15 (30%) used the information available at baseline, and six (12%) used some way of contacting the non-respondents with small questionnaires. Logistic regression models were the most used technique to evaluate participation, used in 18 (40%) of the papers. Passive follow-up studies applied survival (7) and Poisson regression models (4), and a few some combination of different techniques. In eight papers the evaluation was based on frequencies comparison, using baseline characteristics and/or questionnaires. Imputation, weighted regression and simulations were applied in four papers to evaluate and propose analytical methods for correcting potential bias.

Table 2 describes of the overall study characteristics and sample characteristics potentially associated with participation proportion. Most of the publications are concentrated in the years from 2005 to 2014, the oldest having been published in 1978. The studies comprised 40 (80%) geographically population-based, while the remainder were of workers (8), students (1) and recruits (1).

Most of the studies were conducted in Northern Europe (40%). Regarding participant selection, 60% were random sample, the remainder census-based. The most frequent outcomes were overall health condition in twenty-three (46%), and cardiovascular health in forteen. Other outcomes included cancer, accident, substance use, incapacity and smoking. Participant mean age was 49.5 years (SD = 8.2 years). Mean participation proportion was 64.7%, and ranged from 32.2% to 87.3%. Women participation was slightly larger (52.6%) (Table 2).

A negative correlation was found between study year and participation proportion (ρ = -0.38). Figure 2 shows the downward trend in participation proportion. The dotted line indicates the linear regression, an annual rate of decrease of 0.66% (R² = 0.1; p = 0.01). The continuous line (a smooth spline) indicates a downward trend in participation, since 1985. The diameters of the circles of each study, identified by the number of the study (id) in Table 1, is proportional to the inverse of the corresponding standard errors in the meta-regression. The larger circles are more influential in the meta-regression.

The simple meta-regression showed association only between participation proportion and year of the baseline contact (OR = 0.97; 95%CI: 0.95-0.99). The multiple meta-regression showed an association between participation proportion, year of the baseline contact (OR = 0.97; 95%CI: 0.95-0.99) and age (OR = 0.97; 95%CI: 0.95-1.00) (Table 3). In other words, for one-year increase in the year of the baseline contact of the study we expect a 3% decrease in the odds of study participation. Likewise, for one-year increase in the mean age of the study participants we expect a 3% reduction in the odds of study participation.

The analysis shows residual heterogeneity τ² = 0.41 (p < 0.001) for the participation proportion, suggesting that 18.1% of total heterogeneity can be accounted for by including year of the baseline contact and age. The test for residual heterogeneity is significant (LRT = 42,252.5, df = 33, p = 0.00), indicating that other covariates not considered in the model are influencing the participation proportion.

Discussion

We found a high heterogeneity in participation proportions among the papers evaluating non-participation bias. The most referred characteristics described in the systematic reviewed papers were sociodemographic profile, hospitalization and cancer incidence. Mortality was larger among non-participants. However, in the meta-regression performed only year of the baseline contact and age was associated with participation.

Several strategies involving comparison between participants and non-participants have been proposed to evaluate the potential selection bias in cohort studies: questionnaires to non-participants, comparison of participants according to recruitment moment ⁴4. Stang A. Nonresponse research: an underdeveloped field in epidemiology. Eur J Epidemiol 2003; 18:929-31. and passive monitoring of the eligible population using secondary database to assess the outcome ¹¹11. Bopp M, Braun J, Faeh D. Variation in mortality patterns among the general population, study participants, and different types of nonparticipants: evidence from 25 years of follow-up. Am J Epidemiol 2014; 180:1028-35., the majority of papers in our study.

The results show a decrease in participation in studies over time. The reasons for this decline are not clear, but social changes, and changes in selection and recruitment and in study designs may influence participation ³3. Morton LM, Cahill J, Hartge P. Reporting participation in epidemiologic studies: a survey of practice. Am J Epidemiol 2006; 163:197-203.. The decrease in participation may be related particularly to the increasing number of studies in recent decades, as well as the proliferation of political and marketing surveys ⁵5. Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol 2007; 17:643-53.. In addition, increased requests for biological material in epidemiological studies may influence adherence negatively ³3. Morton LM, Cahill J, Hartge P. Reporting participation in epidemiologic studies: a survey of practice. Am J Epidemiol 2006; 163:197-203..

Previous studies have reported the association between young age and participation cohort studies. Contrary to other articles ¹²12. Harald K, Salomaa V, Jousilahti P, Koskinen S, Vartiainen E. Non-participation and mortality in different socioeconomic groups: the FINRISK population surveys in 1972-92. J Epidemiol Community Health 2007; 61:449-54.^,1313. Carlsson F, Merlo J, Lindström M, Ostergren P-O, Lithman T. Representativity of a postal public health questionnaire survey in Sweden, with special reference to ethnic differences in participation. Scand J Public Health 2006; 34:132-9.^,1414. Lundberg I, Damström Thakker K, Hällström T, Forsell Y. Determinants of non-participation, and the effects of non-participation on potential cause-effect relationships, in the PART study on mental disorders. Soc Psychiatry Psychiatr Epidemiol 2005; 40:475-83.^,1515. Jackson R, Chambless LE, Yang K, Byrne T, Watson R, Folsom A, et al. Differences between respondents and nonrespondents in a multicenter community-based study vary by gender ethnicity. The Atherosclerosis Risk in Communities (ARIC) Study Investigators. J Clin Epidemiol 1996; 49:1441-6. the proportion of women in the studies showed no association with participation, not even in the simple model. The outcome of the studies was not associated with participation, in spite of its importance in some of them ¹¹11. Bopp M, Braun J, Faeh D. Variation in mortality patterns among the general population, study participants, and different types of nonparticipants: evidence from 25 years of follow-up. Am J Epidemiol 2014; 180:1028-35.^,1818. Boshuizen HC, Viet AL, Picavet HSJ, Botterweck A, van Loon AJM. Non-response in a survey of cardiovascular risk factors in the Dutch population: determinants and resulting biases. Public Health 2006; 120:297-308.^,1919. Knudsen AK, Hotopf M, Skogen JC, Overland S, Mykletun A. The health status of nonparticipants in a population-based health study: the Hordaland Health Study. Am J Epidemiol 2010; 172:1306-14.^,2020. Goldberg M, Chastang JF, Zins M, Niedhammer I, Leclerc A. Health problems were the strongest predictors of attrition during follow-up of the GAZEL cohort. J Clin Epidemiol 2006; 59:1213-21.^,2121. Walker M, Shaper AG, Cook DG. Non-participation and mortality in a prospective study of cardiovascular disease. J Epidemiol Community Health 1987; 41:295-9.^,2222. Hara M, Sasaki S, Sobue T, Yamamoto S, Tsugane S. Comparison of cause-specific mortality between respondents and nonrespondents in a population-based prospective study: ten-year follow-up of JPHC Study Cohort I. Japan Public Health Center. J Clin Epidemiol 2002; 55:150-6.^,2323. Ferrie JE, Kivimäki M, Singh-Manoux A, Shortt A, Martikainen P, Head J, et al. Non-response to baseline, non-response to follow-up and mortality in the Whitehall II cohort. Int J Epidemiol 2009; 38:831-7.^,2424. Barchielli A, Balzi D. Nine-year follow-up of a survey on smoking habits in Florence (Italy): higher mortality among non-responders. Int J Epidemiol 2002; 31:1038-42.^,2525. Putnam RD. Tuning in, tuning out: the strange disappearance of social capital in America. PS Polit Sci Polit 1995; 28:664-83..

Study region showed no association with participation, in spite of the diversity of places evaluated. Participating in studies voluntarily, giving time, information and biological material is all related to ideas of social capital and volunteering ¹⁶16. Stang A, Moebus S, Dragano N, Beck EM, Möhlenkamp S, Schmermund A, et al. Baseline recruitment and analyses of nonresponse of the Heinz Nixdorf Recall Study: identifiability of phone numbers as the major determinant of response. Eur J Epidemiol 2005; 20:489-96., and we expected variation according to local cultural components.

Participation in studies has also been associated with behavioral variables and with general state of health. Non-participants report greater consumption of alcohol, smoking and poor general state of health ¹²12. Harald K, Salomaa V, Jousilahti P, Koskinen S, Vartiainen E. Non-participation and mortality in different socioeconomic groups: the FINRISK population surveys in 1972-92. J Epidemiol Community Health 2007; 61:449-54.^,1515. Jackson R, Chambless LE, Yang K, Byrne T, Watson R, Folsom A, et al. Differences between respondents and nonrespondents in a multicenter community-based study vary by gender ethnicity. The Atherosclerosis Risk in Communities (ARIC) Study Investigators. J Clin Epidemiol 1996; 49:1441-6.^,2019,2121. Walker M, Shaper AG, Cook DG. Non-participation and mortality in a prospective study of cardiovascular disease. J Epidemiol Community Health 1987; 41:295-9.^,2222. Hara M, Sasaki S, Sobue T, Yamamoto S, Tsugane S. Comparison of cause-specific mortality between respondents and nonrespondents in a population-based prospective study: ten-year follow-up of JPHC Study Cohort I. Japan Public Health Center. J Clin Epidemiol 2002; 55:150-6.^,2323. Ferrie JE, Kivimäki M, Singh-Manoux A, Shortt A, Martikainen P, Head J, et al. Non-response to baseline, non-response to follow-up and mortality in the Whitehall II cohort. Int J Epidemiol 2009; 38:831-7.^,2626. Shahar E, Folsom AR, Jackson R. The effect of nonresponse on prevalence estimates for a referent population: insights from a population-based cohort study. Atherosclerosis Risk in Communities (ARIC) Study Investigators. Ann Epidemiol 1996; 6:498-506.^,2727. Garcia M, Fernandez E, Schiaffino A, Borrell C, Marti M, Borras JM. Attrition in a population-based cohort eight years after baseline interview: The Cornella Health Interview Survey Follow-up (CHIS.FU) Study. Ann Epidemiol 2005; 15:98-104.^,2828. Veenstra MY, Friesema IHM, Zwietering PJ, Garretsen HFL, Knottnerus JA, Lemmens PHHM. Lower prevalence of heart disease but higher mortality risk during follow-up was found among nonrespondents to a cohort study. J Clin Epidemiol 2006; 59:412-20.^,2929. Manjer J, Carlsson S, Elmståhl S, Gullberg B, Janzon L, Lindström M, et al. The Malmö Diet and Cancer Study: representativity, cancer incidence and mortality in participants and non-participants. Eur J Cancer Prev 2001; 10:489-99.^,3030. Van Loon AJM, Tijhuis M, Picavet HSJ, Surtees PG, Ormel J. Survey non-response in the Netherlands: effects on prevalence estimates and associations. Ann Epidemiol 2003; 13:105-10.^,3131. Jacobsen TN, Nohr EA, Frydenberg M. Selection by socioeconomic factors into the Danish National Birth Cohort. Eur J Epidemiol 2010; 25:349-55.^,3232. Kjøller M, Thoning H. Characteristics of non-response in the Danish Health Interview Surveys, 1987-1994. Eur J Public Health 2005; 15:528-35.^{, 33}33. Carter KN, Imlach-Gunasekara F, McKenzie SK, Blakely T. Differential loss of participants does not necessarily cause selection bias. Aust N Z J Public Health 2012; 36:218-22.^,3434. Forthofer RN. Investigation of nonresponse bias in NHANES II. Am J Epidemiol 1983; 117:507-15.. This information, however, are not available in most publications, limiting the scope of our study.

Strategies to increase participation proportion have been proposed in terms of persuading individuals who are reluctant or hesitant; however, willingness to participate is not always accompanied by commitment to adhere to the study in the long term ³⁵35. Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Q 2008; 72:167-89.. Lastly, we agree with the argument of Morton et al. ³3. Morton LM, Cahill J, Hartge P. Reporting participation in epidemiologic studies: a survey of practice. Am J Epidemiol 2006; 163:197-203. that more information should be requested on the profile of participation and its potential bias.

There is a major need to pursue methodological studies to evaluate the impacts of non-participation on measures of effect in cohort studies. Strategies for that kind of evaluation include comparing participants with non-participants through administrative data bases (sex, age, place of residence), application of summary questionnaires and passive follow-up of eligible population to evaluate mortality ⁴4. Stang A. Nonresponse research: an underdeveloped field in epidemiology. Eur J Epidemiol 2003; 18:929-31.. Recent publications from journals with high impact factors show that nonparticipation is mostly ignored or dismissed by many authors, although some are attempting to reduce it or mention it as a limitation in their study ³⁶36. Keeble C, Barber S, Law GR, Baxter PD. Participation bias assessment in three high-impact journals. SAGE Open 2013; 3(4):2158244013511260..

In conclusion, our findings suggest that the drive for participation and compliance should be assessed previously to funding the cohort study, and specific local knowledge should be included in addressing the potential participants.

Acknowledgments

We acknowledge Wolfgang Viechtbauer (at Maastricht University, Maastricht, Netherlands) for his help with the R metafor package, and Israel Souza for revising the selected paper. C. M. Coeli was supported by research fellowship grants from CNPq (304101/2011-7) and Faperj (E26/102.771/2012).

References

Publication Dates

History