Abstract:

This study illustrates the use of a recently developed sensitivity index, the E-value, helpful in strengthening causal inferences in observational epidemiological studies. The E-value aims to determine the minimum required strength of association between an unmeasured confounder and an exposure/outcome to explain the observed association as non-causal. Such parameter is defined as E-value=RR+RRRR-1, where RR is the risk ratio between the exposure and the outcome. Our work illustrates the E-value using observational data from a recently published study on the relationship between indicators of prenatal care adequacy and the outcome low birthweight. The E-value ranged between 1.45 and 5.63 according to the category and prenatal care index evaluated, showing the highest value for the “no prenatal care” category of the GINDEX index and the minimum value for “intermediate prenatal care” of the APNCU index. For “inappropriate prenatal care” (all indexes), the E-value ranged between 2.76 (GINDEX) and 4.99 (APNCU). These findings indicate that only strong confounder/low birthweight associations (more than 400% increased risk) would be able to fully explain the prenatal care vs. low birthweight association observed. The E-value is a useful, intuitive sensitivity analysis tool that may help strengthening causal inferences in epidemiological observational studies.

Keywords:
Measures of Association, Exposure, Risk or Outcome; Observational Studies as Topic; Health Care Outcome Assessment

Resumo:

O estudo ilustra o uso de um índice recém-desenvolvido para análise de sensibilidade, o valor de E, útil para embasar inferências causais em estudos epidemiológicos observacionais. O valor de E busca identificar a força mínima da associação entre um fator de confusão não medido e uma exposição/desfecho que seria necessária para explicar a associação observada enquanto não causal, definido como valor de E-value=RR+RRRR-1, onde RR é a razão de riscos entre a exposição e o desfecho. Aqui, o valor de E é exemplificado através de dados observacionais de um estudo recém-publicado sobre a relação entre indicadores de adequação da assistência pré-natal e o baixo peso ao nascer enquanto desfecho. O valor de E variou entre 1,45 e 5,63, a depender da categoria e do índice de assistência pré-natal avaliado, com o valor máximo para a categoria “sem assistência pré-natal” do índice GINDEX e o mínimo para “assistência pré-natal intermediária” do índice APNCU. Na categoria “assistência pré-natal inadequada” (todos os índices), o valor de E variou entre 2,76 e 4,99 (índices GINDEX e APNCU, respectivamente). Isso indica que apenas associações fortes entre um fator de confusão e baixo peso ao nascer (aumento de risco maior que 400%) seria capaz de explicar plenamente a associação entre assistência pré-natal e baixo peso ao nascer. O valor de E é uma ferramenta útil e intuitiva para análise de sensibilidade, que pode ajudar a fortalecer as inferências causais em estudos epidemiológicos observacionais.

Palavras-chave:
Medidas de Associação, Exposição, Risco ou Desfecho; Estudos Observacionais como Assunto; Avaliação de Resultados em Cuidados de Saúde

Resumen:

El presente estudio ilustra el uso de un índice desarrollado recientemente para el análisis de la sensibilidad, el E-value, útil para el fortalecimiento de las inferencias causales en los estudios epidemiológicos observacionales. El E-value tiene por objetivo identificar la fortaleza de asociación mínima necesaria entre un factor de confusión no calculable y una exposición/resultado que sería necesario para explicar la asociación observada como no-causal, y está definida como E-value=RR+RRRR-1, donde RR es la ratio de riesgo entre la exposición y el resultado. Aquí, el E-value es ilustratado con datos observacionales de un estudio recientemente publicado para mostrar la relación entre indicadores de adecuación de cuidado prenatal y el resultado de bajo peso al nacer. El E-value varió entre 1,45 and 5,63, dependiendo de la categoría y el índice cuidado prenatal evaluado, con su valor más alto para la categoría “no cuidado prenatal” del índice GINDEX y el mínimo para el “cuidado prenatal intermedio” del índice APNCU. En la categoría “inapropiado cuidado prenatal” (todos los índices) el E-value varió entre 2,76 y 4,99 (los índices GINDEX y APNCU, respectivamente). Esto indica que solamente un factor de confusión fuerte/asociaciones bajo peso al nacer (incrementaron el riesgo más de un 400%) habría hecho posible explicar completamente la asociación observada cuidado prenatal vs bajo peso al nacer. El E-value es una herramienta de análisis de sensibilidad útil, intuitiva, que puede ayudar a fortalecer las inferencias causales en los estudios epidemiológicos observacionales.

Palabras clave:
Medidas de Asociación, Exposición, Riesgo o Desenlace; Estudios Observacionales como Asunto; Evaluación de Resultados en la Atención de Salud

Introduction

Sensitivity analyses are commonly used in observational epidemiologic studies to quantify the robustness of an investigated association to unmeasured or uncontrolled confounders ¹1. VanderWeele TJ, Ding P. Sensitivity analysis in observational research: Introducing the E-value. Ann Intern Med 2017; 167:268-74.. Traditional sensitivity analysis methods estimate the strength of association between the unmeasured confounder and the outcome (RR _UD ) and between the unmeasured confounder and the exposure (RR _EU ). After specifying these associations, one can calculate the influence of a given pair RR _UD and RR _EU on the risk ratio between exposure and outcome (RR _ED ) (Figure 1). The confounding factor (B) - maximum relative amount by which unmeasured confounders could reduce an observed and - is calculated as follows ²2. Ding P, VanderWeele TJ. Sensitivity analysis without assumptions. Epidemiology 2016; 27:368-77.:

By dividing the observed by B, one obtains the maximum value by which a set of confounding factors could alter the observed RR _ED . However, some author express concern about the subjectivity underlying sensitivity parameters choice (RR _UD and RR _EU ) ¹1. VanderWeele TJ, Ding P. Sensitivity analysis in observational research: Introducing the E-value. Ann Intern Med 2017; 167:268-74.^,22. Ding P, VanderWeele TJ. Sensitivity analysis without assumptions. Epidemiology 2016; 27:368-77.. These parameters also entail simplifications related to unmeasured confounders, such as being defined as a binary variable or requiring the assumption of a single confounder ³3. Bross IDJ. Spurious effects from an extraneous variable. J Chronic Dis 1966; 19:637-47.^,44. Schelesselman JJ. Assessing effects of confounding variables. Am J Epidemiol 1978; 108:3-8.^,55. Rosenbaum PR, Rubin DB. Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. J R Stat Soc Series B Stat Methodol 1983; 45:212-8., which negatively impact the robustness of sensitivity analyses and the causal inferences sought in observational studies.

Seeking to develop a simple and intuitive tool that waves the need for strong assumptions, Ding & VanderWeele ²2. Ding P, VanderWeele TJ. Sensitivity analysis without assumptions. Epidemiology 2016; 27:368-77. proposed a new sensitivity analysis technique for observational studies - the E-value. This tool aims to determine the “...minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates” ¹1. VanderWeele TJ, Ding P. Sensitivity analysis in observational research: Introducing the E-value. Ann Intern Med 2017; 167:268-74. (p. 268). The E-value can be calculated as shown in Equation ²2. Ding P, VanderWeele TJ. Sensitivity analysis without assumptions. Epidemiology 2016; 27:368-77.,

Where RR is the risk ratio between exposure and outcome. The E-value is conditional on the measured covariates and calculated based on the risk ratio scale used in the analysis. When the effect measure is the odds ratio (OR) and the outcome is relatively rare (prevalence below 15% in the population), OR can be used in Equation 2, defining the following:

Equation 3 is also applicable to a confidence interval (CI) parameter. For cases where the lower limit (LL) of the CI is lower than or equal to “1”, the E-value is considered one; otherwise, the E-value is determined as follows:

For large E-values, the unmeasured confounder will need a considerable impact to fully explain the effect estimate. Conversely, small values​ indicate that little impact would already be able to explain the effect estimate, indicating weak causal relations between the study variables.

Next, we will illustrate the use of the E-value using observational data from a recently-published study on the relationship between indicators of prenatal care adequacy and the outcome low birthweight ⁶6. Vale CCR, Almeida NKO, Almeida RMVR. Association between pre-natal care adequacy indexes and low birth weight outcome. Rev Bras Ginecol Obstet 2021; in press..

Methods

An observational study conducted by Vale et al. ⁶6. Vale CCR, Almeida NKO, Almeida RMVR. Association between pre-natal care adequacy indexes and low birth weight outcome. Rev Bras Ginecol Obstet 2021; in press. used multiple logistic modeling to investigate low birthweight in 368,093 singleton term live births in Rio de Janeiro, Brazil, from 2015 to 2016. Box 1 summarizes the study covariate variables and prenatal care indexes. The E-value was used to determine the minimum strength of association between possible unmeasured confounders and the outcome capable of altering results interpretation. E-values were calculated based on the adjusted OR and the lower limit of the 95% confidence interval (95%CI) of each prenatal care adequacy index (only statistically significant categories). The “Adequate prenatal care” category was used as reference (OR = 1.00). Analyzes were performed using the R Studio v.1.2.5001 (http://www.r-project.org) and the SPSS v.23 (https://www.ibm.com/).

Results

The estimated E-value ranged between 1.45 and 5.63 according to the category and index evaluated (Table 1), showing the highest value for the “no prenatal care” category of the GINDEX index and the minimum value for “intermediate prenatal care” of the adequacy of prenatal care utilization (APNCU) index. For “inappropriate prenatal care” (all indexes), the E-value ranged between 2.76 (GINDEX) and 4.99 (APNCU).

Discussion

Based on the E-value parameter, researchers were able to determine the minimum required association between unmeasured potential confounders and low birthweight for explaining the observed associations. For instance, E-value reached its maximum value (4.99) when APNCU (the most discriminatory index) was considered “inappropriate prenatal care,” and only very strong confounders/low birthweight associations would be able to explain the prenatal care vs. low birthweight association observed.

The E-value method allows these results to be contrasted with association values for known risk factors not included in an traditional analysis. Studies approaching smoking during pregnancy, for example, reported an adjusted OR ranging from 1.23 to 2.63 ⁷7. Bell JF, Zimmerman FJ. Selection bias in prenatal care use by Medicaid recipients. Matern Child Health J 2003; 7:239-52.^,88. Zambonato AMK, Pinheiro RT, Horta BL, Tomasi E. Risk factors for small-for-gestational age births among infants in Brazil. Rev Saúde Pública 2004; 38:24-9.^,99. Reichman NE, Teitler JO. Timing of enhanced prenatal care and birth outcomes in New Jersey's Health Start program. Matern Child Health J 2005; 9:151-8.^,1010. Xaverius P, Alman C, Holtz L, Yarber L. Risk factors associated with very low birth weight in a large urban area, stratified by adequacy of prenatal care. Matern Child Health J 2016; 20:623-9.^,1111. Xaverius PK, O'Reilly Z, Li A, Flick LH, Arnold LD. Smoking cessation and pregnancy: timing of cessation reduces or eliminates the effect on low birth weight. Matern Child Health J 2019; 23:1434-41. - lower than that found by E-value parameters regarding the “inadequate prenatal care” category of all evaluated indexes, except for that of Ciari Jr. et al. ¹²12. Ciari Jr. C, Santos JLF, Almeida PAM. Avaliação quantitativa de serviços de pré-natal. Rev Saúde Pública 1972; 6:361-70. and Kessner et al. ¹³13. Kessner DM, Singer J, Kalk CE, Schlesinger ER. Infant death: an analysis by maternal risk and health care. Washington DC: Institute of Medicine/National Academy of Sciences; 1973.. This suggests that, alone, smoking during pregnancy is not capable of explaining the prenatal care vs. low birthweight association observed, thus strengthening causal inferences.

Likewise, studies reported adjusted ORs for alcohol and drug use during pregnancy ranging between 1.04 and 1.68 ⁹9. Reichman NE, Teitler JO. Timing of enhanced prenatal care and birth outcomes in New Jersey's Health Start program. Matern Child Health J 2005; 9:151-8., so that similar causal inferences may be made: the consumption of alcohol and drugs during pregnancy per se also would not comprise confounding factors capable of fully explaining the effects estimate.

Limitations

Just as any new metric, the E-value could be potentially misused, and should only be applied provided that researchers have a clear understanding of its scope and limitations. The recent literature on parameters for E-value has pointed out the following caveats for its use ¹⁴14. Ioannidis JPA, Tan YJ, Blum MR. Limitations and misinterpretations of E-values for sensitivity analyses of observational studies. Ann Intern Med 2019; 170:108-11.^,1515. Blum MR, Tan YJ, Ioannidis JPA. Use of E-values for addressing confounding in observational studies - an empirical assessment of the literature. Int J Epidemiol 2020; 49:1482-94.^,1616. VanderWeele TJ, Mathur MB, Ding P. Correcting misinterpretations of the E-Value. Ann Intern Med 2019; 170:131-2.^,1717. VanderWeele TJ, Ding P, Mathur MB. Technical considerations in the use of the E-value. J Causal Infer 2019; 7:20180007.^,1818. VanderWeele TJ, Mathur MB. Commentary: developing best-practice guidelines for the reporting of E-values. Int J Epidemiol 2020; 49:1495-97.:

(1) The E-value is strictly concerned with the impact of unobserved confounders, evaluating no other biases such as sample bias, selective reporting, or other design flaws. These factors should be considered when interpreting an E-value, so that a good study with a low E-value may produce more reliable results than poorly designed and controlled studies with a high E-value.

(2) The E-value may be less useful in the presence of multiple, possibly interacting unmeasured confounders, in which case “...one should perhaps question whether the data available are in fact adequate to get a reasonable estimate of the causal effect at all (...) it is perhaps time to leave that study data alone and pursue other data sources more adequate”, as stated by VanderWeele et al. ¹⁷17. VanderWeele TJ, Ding P, Mathur MB. Technical considerations in the use of the E-value. J Causal Infer 2019; 7:20180007. (p. 4).

(3) Another limitation concerns the assumption of the same value for the confounder x exposure and confounder x outcome association. When this is not the case, more complicated methods were developed for applying E-value-like metrics ¹1. VanderWeele TJ, Ding P. Sensitivity analysis in observational research: Introducing the E-value. Ann Intern Med 2017; 167:268-74.^,22. Ding P, VanderWeele TJ. Sensitivity analysis without assumptions. Epidemiology 2016; 27:368-77.. Using the index under these circumstances is valid upon the assumption that the E-value is a heuristic filter for the total maximum effect of all unknown confounders ¹⁴14. Ioannidis JPA, Tan YJ, Blum MR. Limitations and misinterpretations of E-values for sensitivity analyses of observational studies. Ann Intern Med 2019; 170:108-11.^,1515. Blum MR, Tan YJ, Ioannidis JPA. Use of E-values for addressing confounding in observational studies - an empirical assessment of the literature. Int J Epidemiol 2020; 49:1482-94.^,1616. VanderWeele TJ, Mathur MB, Ding P. Correcting misinterpretations of the E-Value. Ann Intern Med 2019; 170:131-2..

Conclusion

Many are the available procedures for conducting sensitivity analyses. However, for being considered “...too complicated to describe in reports, (...) too difficult to present, occupy too much space” and given that “reviewers and editors were often unsympathetic and believed that they could not be understood”, as emphasized by VanderWeele et al. ¹⁶16. VanderWeele TJ, Mathur MB, Ding P. Correcting misinterpretations of the E-Value. Ann Intern Med 2019; 170:131-2. (p. 131-2), these procedures are not commonly used. Before this scenario, our study illustrated the use of a recently developed sensitivity index: the E-value, an intuitive tool of easy implementation that assemble the toolbox for dealing with causality inferences in non-experimental settings. “Statistical significance” metrics such as the p-value determines the existence of possible relationships between exposure and outcome, but fails in addressing potential bias arising from unmeasured confounders - to which end the E-value could be used.

Acknowledgments

We thank the Brazilian Graduate Studies Coordinating Board (CAPES) for the financial support offered for this research (grant code 001).

References

Publication Dates

History