Abstracts

Introduction

In clinical and epidemiological studies one often wishes to estimate the effects of factors in binary outcomes. For these cases, when one is interested in estimating the relative risk (RR) or prevalence ratio (PR), it has already been well established that the logistic regression is not the most suitable statistical analysis, particularly when the outcome is common (> 10%). Several authors have shown that the odds ratio (OR) obtained by the logistic model can underestimate or overestimate the RR or the PR ¹1 Barros AJD, Hirakata VN. Alternatives for logistic regression in cross-sectional studies: an empirical comparison of models that directly estimate the prevalence ratio. BMC Med Res Methodol 2003; 3:21. ^{, 2}2 Greenland S. Model-based estimation of relative risks and other epidemiologic measures in studies of common outcomes and in case-control studies. Am J Epidemiol 2004; 160:301-5. ^{, 3}3 McNutt L, Wu C, Xue X, Hafner JP. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol 2003; 157:940-3. ^{, 4}4 Skov T, Deddens J, Petersen MR, Endahl L. Prevalence proportion ratios: estimation and hypothesis testing. Int J Epidemiol 1998; 27:91-5.. Alternatives have also been suggested and discussed, such as the use of the Poisson regression with robust variance and the log-binomial model ¹1 Barros AJD, Hirakata VN. Alternatives for logistic regression in cross-sectional studies: an empirical comparison of models that directly estimate the prevalence ratio. BMC Med Res Methodol 2003; 3:21. ^{, 3}3 McNutt L, Wu C, Xue X, Hafner JP. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol 2003; 157:940-3. ^{, 4}4 Skov T, Deddens J, Petersen MR, Endahl L. Prevalence proportion ratios: estimation and hypothesis testing. Int J Epidemiol 1998; 27:91-5. ^{, 5}5 Deddens JA, Petersen MR. Approaches for estimating prevalence ratios. Occup Environ Med 2008; 65:501-6. ^{, 6}6 Deddens JA, Petersen MR, Lei X. Estimation of prevalence ratios when PROC GENMOD does not converge. In: Proceedings of the 28th Annual SAS Users Group International Conference. http://www2.sas.com/proceedings/sugi28/270-28.pdf (accessed on 16/Apr/2010).
http://www2.sas.com/proceedings/sugi28/2... ^{, 7}7 Petersen MR, Deddens JA. A comparison of two methods for estimating prevalence ratios. BMC Med Res Methodol 2008; 8:9. ^{, 8}8 Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol 2005; 162:199-200. ^{, 9}9 Zocchetti C, Consonni D, Bertazzi PA. Estimation of prevalence rate ratios from cross-sectional data. Int J Epidemiol 1995; 24:1064-7. ^{, 10}10 Zou G. A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol 2004; 159:702-6..

Although recommended methods seem to be well established for dichotomous outcomes, there is still little discussion concerning which method should be used to estimate the RR or the PR for outcomes with three or more categories. In such circumstances, one usually uses the multinomial logistic regression which, unlike the binary logistic model, estimates the OR, which is then used as an approximation of the RR or the PR.

Blizzard & Hosmer ¹¹11 Blizzard L, Hosmer DW. The log multinomial regression model for nominal outcomes with more than two attributes. Biom J 2007; 49:889-902. proposed the log-multinomial regression model, which directly estimates the RR or PR when the outcome is multinomial. To evaluate whether this leads to an underestimation or overestimation of the RR or PR when we use the OR obtained through the multinomial logistic regression, ideally one would compare the results obtained using this method with those obtained using the log-multinomial model. However, we are not aware of the availability of a computational routine for this model. Moreover, Blizzard and Hosmer ¹¹11 Blizzard L, Hosmer DW. The log multinomial regression model for nominal outcomes with more than two attributes. Biom J 2007; 49:889-902. concluded that separate robust Poisson regressions produce estimates very similar to those of the log-multinomial model.

Therefore, this work aims to study through simulation the accuracy and precision of OR, obtained through the multinomial logistic regression, to estimate RR and to compare it with the RR estimated by separate robust Poisson regressions and those estimated by the separate log-binomial models. The separate log-binomial models have been included in the study as they have proved to be the most suitable for estimating the RR or the PR for binary outcomes. Another reason for choosing the robust Poisson regression and the log-binomial model is that both of them are implemented in various statistical programs.

Methodology

It is well known that RR and PR have very different epidemiological meanings, but the mathematical definitions are equivalent. The accuracy and precision of the model to estimate the RR are the same as for estimating the PR, and so henceforth we will only refer to the RR.

In order to illustrate the three techniques evaluated, without loss of generality, we will consider a multinomial outcome Y with the categories A, B and C, and a binary exposure factor X (yes x no). We are interested in estimating the RR for the exposure X with the categories B (RR _B ) and C (RR _C ) of the outcome and we are not interested in the association with category A.

The first method studied in this work was the multinomial logistic regression. The definition of the model and details about the properties of their estimators can be found in Hosmer & Lemeshow ¹²12 Hosmer D, Lemeshow S. Applied logistic regression. 2 nd Ed. New York: John Wiley & Sons; 2000. . To adjust it, we use the library VGAM of program R, version 2.9.0 (The R Foundation for Statistical Computing, Vienna, Austria; http://www.r-project.org ) ¹³13 Yee TW. The VGAM package for categorical data analysis. Journal of Statistical Software 2010; 32: 1-34.. The syntax for adjusting the model is:

library(VGAM)

model <- vglm(Y ~ X, family=multinomial())

summary(model)

As in Blizzard & Hosmer ¹¹11 Blizzard L, Hosmer DW. The log multinomial regression model for nominal outcomes with more than two attributes. Biom J 2007; 49:889-902., the use of separate Poisson regressions, according to the second method evaluated, consists of creating binary outcomes from the original multinomial outcome, and then fitting separate robust Poisson regressions between each binary outcome and the factors. For the aforementioned outcome Y, we create the dichotomous variables Y _B and Y _C as follows:

To fit the models we used the libraries lmtest ¹⁴14 Zeileis A, Hothorn T. Diagnostic checking in regression relationships. R News 2002; 2:7-10. and sandwich ¹⁵15 Zeileis A. Econometric computing with HC and HAC covariance matrix estimators. Journal of Statistical Software 2004; 11:1-17. ^{, 16}16 Zeileis A. Object-oriented computation of sandwich estimators. Journal of Statistical Software 2006; 16:1-16. of R. The syntax for estimating RR _B is as follows. For RR _C it is analogous.

library(lmtest)

library(sandwich)

model.pois.B <- glm(YB ~ X, family=poisson)

coef.beta.B <- coeftest(model.pois.B, vcov= sandwich)

coef.beta.B

The third technique compared, namely the adjustment of separate log-binomial models, is analogous to the method of separate Poisson regressions, only adjusting one log-binomial model for each binary outcome. The syntax for estimating RR _B using R is:

model.bin.B <- glm(YB ~ X, family=binomial (link=log))

summary(model.bin.B)

It should be pointed out that the Poisson regression with robust variance and the log-binomial model are also available in other statistical software such as SAS (SAS Inst., Cary, USA), SPSS (SPSS Inc., Chicago, USA) and Stata (Stata Corp., College Station, USA).

Simulation

In the simulation study we considered an outcome (Y) with three categories (A, B and C) and a binary exposure factor (X). An RR as established between exposed and non-exposed individuals equal to 1.5 for the outcome B (RR _B = 1.5) and equal to 2 for the outcome C (RR _C = 2). The simulated scenarios combined three probabilities of exposure, say P(X=+), equal to 0.1, 0.5 and 0.9 and three marginal distributions for the outcome Y: P(Y=A) = 0.90, P(Y=B) = 0.05 and P(Y=C) = 0.05; P(Y=A) = 0.60, P(Y=B) = 0.35 and P(Y=C) = 0.05, and P(Y=A) = 0.4, P(Y=B) = 0.30 and P(Y=C) = 0.30. Thus, nine different scenarios were simulated, each one being repeated for three sample sizes (100, 500 and 1,000).

Considering these fixed RRs and probabilities for each response category, the conditional probabilities of each outcome given the exposure factor were calculated. Using the R program, for each sample size and scenario, 10,000 contingency tables 2x3 were simulated, in which the two lines are defined as presence and absence of the exposure factor (X), and the three columns are the categories of the outcome (Y). These tables were simulated using two independent multinomial distributions (one for those exposed and the other for those not exposed) with probabilities specified in Table 1 .

In order to avoid the occurrences of estimates of RRs equal to zero or infinity, if a simulated table contained a cell with zero frequency, it was discarded and another one generated. The consequences of this will be discussed further on.

For each simulated table, the point estimate and respective 95% confidence interval (95%CI) of the RRs obtained by each method were recorded. From the 10,000 estimates resulting for each RR (RR _B and RR _C ) and each method, the following measures of accuracy and precision were calculated: averages of estimates, absolute average biases and coverage of the confidence intervals as a percentage. The absolute average bias of the estimators for each parameter q, i.e., each true RR, was calculated as

where is the estimated RR of the i-th table from the respective model. Whereas the coverage percentage of the confidence intervals was calculated by:

%Coverage = (number of confidence intervals which contained q / 10,000) x 100.

Results

The separate robust Poisson regression and separate log-binomial models had practically identical measures of accuracy and precision. Therefore, we will only display here the results of the separate log-binomial model and the multinomial logistic model. The tables with the results for the robust Poisson regression can be obtained at www.mat.ufrgs.br/~camey/Poisson . The results of the simulations for the parameters RR _B and RR _C are summarized in Tables 2 and 3 , respectively.

For both parameter RR _B , and parameter RR _C , in nearly all the scenarios, the log-binomial model had more accurate estimates (smaller averages of absolute biases) than the multinomial logistic regression model. In some cases the average absolute bias of the multinomial logistic regression model was more than 2,000 times the value of the average absolute bias of the log-binomial model ( Tables 2 and 3 , scenario 7, n = 500 and n = 1,000). The exceptions occurred only for RR _C in scenarios 3 and 6 with n = 500 and n = 100, respectively. Generally speaking, the ratio between the biases is nearer 1 when the incidence of the outcomes is low [ and/or ].

For the parameter RR _B , except for scenario 3, the averages of the absolute biases diminished as we increased the sample size. Whereas for the parameter RR _C , there were many cases in which increasing the sample size did not lead to the bias being reduced: scenarios 2, 3, 5, 6 and 9.

In the log-binomial model, for both the parameters, in only three situations did the coverage percentage of the intervals remain below the nominal value of 95% (see Table 2 , n = 100, third scenario and Table 3 , n = 100, third and sixth scenarios), always with a difference of less than 10%. However, for the multinomial logistic model, the coverage percentage of the confidence interval was only near the nominal value of 95% when there was a low prevalence of outcomes and/or sample size of n = 100. In some cases the coverage % was near zero (see scenarios 7 and 8, in Tables 2 and 3 ).

It should be pointed out that, for all the tables, there were no problems of convergence in the log-binomial model or any unallowable solutions for the probabilities estimated by the robust Poisson regression model, i.e., all the probabilities assumed values between 0 and 1.

A fourth approach was also studied in this work, which was the robust Poisson regression with offset ¹⁷17 Woodward M. Epidemiology: study design and data analysis. Boca Raton: Chapman & Hall; 1999.. We have not detailed this method, or displayed its results here (available at www.mat.ufrgs.br/~camey/Poisson ), as they were practically identical to those obtained using the separate robust Poisson regression without offset. Moreover, the calculation of the offset is a practical disadvantage, as it is increasingly complex, since categorical exposure (or confusion) factors are added, and cannot be calculated in the presence of continuous exposure (or confusion) factors.

Comparison using real data

As an example, we used a data set of the Study of Food Intake and Eating Behavior in Pregnancy (ECCAGE) which is a prospective cohort study of pregnant women followed until the puerpe- rium ¹⁸18 Nunes MA, Ferri CP, Manzolli P, Soares RM, Drehmer M, Buss C, et al. Nutrition, mental health and violence: from pregnancy to postpartum cohort of women attending primary care units in southern Brazil: ECCAGE Study. BMC Psychiatry 2010; 10:66.. These pregnant women were receiving care in 18 primary care units in two cities in Southern Brazil. They were first evaluated between the 16th and 36th week of pregnancy at a prenatal visit. In this example we use a sample of 667 pregnant women.

Total weight gain was classified according to the recommendation of the Institute of Medicine 2009 ¹⁹19 Institute of Medicine. Weight gain during pregnancy: reexamining the guidelines. Washington DC : Institute of Medicine; 2009 . . Total weight gain between 12.5 and 18kg was considered adequate for women with pre-pregnancy body mass index (BMI) below 18.5kg/m2, between 11.5 and 16kg for women with pre-pregnancy BMI between 18.5 and 24.9kg/m2, and between 7 and 11.5kg/m2 for women with pre-pregnancy BMI between 25.0 and 29.9kg/m2. Total weight gain between 5 and 9kg was considered appropriate when pre-pregnancy BMI was higher or equal than 30.0kg/m2. Incidences of pregnancy weight gain according to the categories of pre-pregnancy BMI ²⁰20 Drehmer M, Camey S, Schmidt MI, Olinto MTA, Giacomello A, Buss C, et al. Socioeconomic, demographic and nutritional factors associated with maternal weight gain in general practices in southern Brazil. Cad Saúde Pública 2010; 26:1024-34. are showed in Table 4 .

The results obtained of the adjustment by separate log-binomial model and multinomial logistic regression are shown in Table 5 . In this example we can perceive that the choice of model influenced the direction, magnitude and significance of the effect (RR).

The first difference between the models can be observed in the RR of the insufficient weight gain of the women who had a pre-pregnancy BMI greater than or equal to 30 (obese), which in spite of not being significant is rather illustrative. When the multinomial regression was used, being obese was a risk factor (RR = 1.42) for insufficient weight gain and by the log-binomial model it becomes a protective factor (RR = 0.81).

The influence of the choice of model in the magnitude of the effect can be seen in Table 5 in several comparisons between the estimates of the two models, as for example, in the RR of excessive gain of women with a BMI greater than or equal to 25 and less than 30kg/m2 (overweight). In this case there is a reduction of approximately 70% in the magnitude of the effect when the log-binomial model is used.

The illustration that the choice of model influences the significance of the RR can be seen through the RR for insufficient weight gain of women with a BMI greater than or equal to 25 and less than 30kg/m2 (overweight). By the multinomial logistic model, being overweight is not a statistically significant protective factor for insufficient weight gain, different from what occurs when the log-binomial model is used.

This example makes it clear that the suitable choice of model influences the result of a study in several ways.

Conclusions

As in the case in which the outcome is dichotomous, the results showed that the multinomial logistic model should not be used to estimate the RR or RP with polytomous outcomes, as its estimates generally have greater bias. As a result of the bias of the estimator, the estimate of the confidence interval (CI) is centered upon a distant value of the parameter, reducing the coverage of the CI.

The ratio between the biases of the RR estimated by the multinomial logistic model compared with those estimated by the log-binomial model is nearly always greater than 1, and this ratio increases to the extent that the incidence of the outcomes increases. This also occurs when the outcome is dichotomous.

In spite of it being generally agreed that increasing the sample size improves the features of the estimates, we can clearly see that when the multinomial logistic regression is used, increasing the sample size produces a CI which in most cases does not contain the parameter. Three cases occurred with the larger sample size in which the coverage is near to zero, meaning that of the 10,000 CI estimated practically none contained the parameter.

As mentioned above, a problem of non-convergence of the log-binomial model did not occur in our simulations, something which it is known does not always occur in practice ⁵5 Deddens JA, Petersen MR. Approaches for estimating prevalence ratios. Occup Environ Med 2008; 65:501-6. ^{, 6}6 Deddens JA, Petersen MR, Lei X. Estimation of prevalence ratios when PROC GENMOD does not converge. In: Proceedings of the 28th Annual SAS Users Group International Conference. http://www2.sas.com/proceedings/sugi28/270-28.pdf (accessed on 16/Apr/2010).
http://www2.sas.com/proceedings/sugi28/2... ^{, 7}7 Petersen MR, Deddens JA. A comparison of two methods for estimating prevalence ratios. BMC Med Res Methodol 2008; 8:9. ^{, 21}21 Petersen MR, Deddens JA. A revised SAS macro for maximum likelihood estimation of prevalence ratios using the COPY method. Occup Environ Med 2009; 66:639-40.. We believe that this occurred due to the design of our simulation, with only one binary factor, and to the discarding of the contingency table with null cells. For the same reason, there were no situations in which the estimated probabilities were greater than one using the robust Poisson regression. We would like to draw your attention to the fact that when there is a convergence problem in the log-binomial model, in general, the Poisson regression estimates probabilities greater than one ⁵5 Deddens JA, Petersen MR. Approaches for estimating prevalence ratios. Occup Environ Med 2008; 65:501-6. . This occurs because in the Poisson regression the average of the Poisson distribution is modeled, which is a positive number, but not restricted to being less than one.

It is known that there already exist proposals for addressing the problem of non-convergence of the log-binomial model, such as the COPY method ⁶6 Deddens JA, Petersen MR, Lei X. Estimation of prevalence ratios when PROC GENMOD does not converge. In: Proceedings of the 28th Annual SAS Users Group International Conference. http://www2.sas.com/proceedings/sugi28/270-28.pdf (accessed on 16/Apr/2010).
http://www2.sas.com/proceedings/sugi28/2... ^{, 21}21 Petersen MR, Deddens JA. A revised SAS macro for maximum likelihood estimation of prevalence ratios using the COPY method. Occup Environ Med 2009; 66:639-40. and the use of non-linear programming ²²22 Lumley T, Kronmal R, Shuangge M. Relative risk regression in medical research: models, contrasts, estimators, and algorithms. Washington DC: University of Washington; 2006. (UW Biostatistics Working Paper Series, 293). ^{, 23}23 Yu B, Wang Z. Estimating relative risks for common outcome using PROC NLP. Comput Methods Programs Biomed 2008; 90:179-86.. More realistic simulations, including more factors of exposure, which are discrete and quantitative, are already being executed.

We would like to stress that for the choice of method of analysis, it is important to review the conceptual aspects of the measure of association which one wishes to estimate, as already pointed out by other authors ¹1 Barros AJD, Hirakata VN. Alternatives for logistic regression in cross-sectional studies: an empirical comparison of models that directly estimate the prevalence ratio. BMC Med Res Methodol 2003; 3:21. ^{, 3}3 McNutt L, Wu C, Xue X, Hafner JP. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol 2003; 157:940-3. ^{, 8}8 Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol 2005; 162:199-200. ^{, 24}24 Cummings P. Analysis of matched cohort data. Stata J 2004; 4:274-81., avoiding or minimizing the erroneous interpretation of OR as RR or PR, especially by readers. For a multinomial outcome this becomes even more important, as the interpretation of the OR is performed concerning a reference category of the outcome and in the RR we have the risk ratio for the category of interest among levels of the factor, but without having a reference category. Indeed we could estimate RR even for the reference category. But as the main issue of this paper is to compare the results with multinomial logistic regression, where this is impossible, we did not estimate.

This study is relevant regarding the discussion of the estimate of RR or PR when the outcomes are multinomial, showing that there is no justification for using the multinomial logistic regression to estimate them, once the last one estimates OR. Our present recommendation is analogous to that of Spiegelman & Hertzmark ⁸8 Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol 2005; 162:199-200. for binary outcomes: one must try to adjust the separate log-binomial models, but if they do not converge, the separate robust Poisson regressions are adjusted.

References

Publication Dates

History