Abstract

The objective of this study was to assess the agreement between the Brazilian Healthy Eating Index – Revised (BHEI-R), estimated by a food frequency questionnaire (FFQ) and multiple 24-hour recalls (24h-R). The Wilcoxon paired test, partial correlations (PC), intraclass correlation coefficient (ICC), and Bland-Altman method were used. The total BHEI-R scores and its components (“total fruits”, “whole fruits”, “total vegetables”, “integral cereals”, “saturated fat”, “sodium”, and “energy intake derived from solid fat, added sugar, and alcoholic beverages”) were statistically different, with the ICC and PC indicating poor concordance and correlation. The mean concordance estimated for the total BHEI-R and its components varied from 68% for “integral cereals” to 147% for “whole fruits”. The suitable concordance limits were violated for most of the components of the BHEI-R. Poor concordance was observed between the BHEI-R estimated by the FFQ and by multiple 24h-R, which indicated a strong reliability of the BHEI-R on the instrument used to collect information on food consumption.

Diet; Validation Studies; Adolescent

Resumo

O objetivo do estudo foi avaliar a concordância entre o Índice de Qualidade da Dieta Revisado (IQD-R), estimado por questionário de frequência alimentar (QFA) e recordatório de 24 horas (R24h). O teste de Wilcoxon, correlações parciais (CP), coeficiente de correlação intra-classe (CCI) e o método Bland-Altman foram usados. Os escores totais do IQD-R e seus componentes (“frutas totais”, “frutas inteiras”, “vegetais totais”, “cerais integrais”, “gordura saturada”, “sódio” e “ingestão de energia derivada de gordura sólida, adicionados de açúcar e bebidas alcoólicas” foram estatisticamente diferentes, com o CCI e as CP indicando baixa concordância e correlação. A concordância média estimada para o IQD-R total e seus componentes variou de 68% para “cereais integrais” e 147% para as “frutas inteiras”. Os limites adequados de concordância foram ultrapassados para a maior parte dos componentes do IQD-R. Observou-se baixa concordância entre o IQD-R estimado pelo QFA e por múltiplos R24h, o que indica uma forte confiabilidade do IQD-R no instrumento usado para coletar informações sobre o consumo alimentar.

Dieta; Estudos de Validação; Adolescente

Resumen

El objetivo del estudio fue evaluar la correlación entre el Índice de Calidad de la Dieta Revisado (ICD-R), estimado por cuestionario de frecuencia de alimentos (CFA) y recordatorio de 24 horas (R24h). Se utilizó la prueba de Wilcoxon, correlaciones parciales (CP), coeficiente de correlación intraclase (CCI) y el método de Bland-Altman. Las puntuaciones totales del ICD-R y sus componentes (“total de frutas”, “frutas enteras”, “vegetales totales”, “granos integrales”, “grasa saturada”, “sodio” y “consumo de energía derivada de grasa sólida, agregado de azúcar y alcohol”) fueron estadísticamente diferentes, con el CCI y el CP indicando baja concordancia y correlación. La concordancia media estimada para el ICD-R total y sus componentes varió de 68% para los “granos integrales” y 147% para las “frutas enteras”. Los límites adecuados de concordancia fureon excedidos para la mayor parte de los componentes del ICD-R. Se observó baja concordancia entre el ICD-R estimado por el CFA y múltiples R24h, lo que indica una fuerte confiabilidad del ICD-R en el instrumento que se utiliza para recopilar información sobre el consumo de alimentos.

Dieta; Estudios de Validación; Adolescente

Introduction

Given the complexity of individual food consumption, different methods have been proposed to assess dietary quality, with the aim of providing more efficient analyses of the correlation between diet and health outcomes ¹1. Fransen HP, Ocké MC. Indices of diet quality. Curr Opin Clin Nutr Metab Care 2008; 11:559-65.^,22. Lazarou C, Newby PK. Use of dietary indices among children in developed countries. Adv Nutr 2011; 2:295-303.^,33. Wirfält E, Drake I, Wallström P. What do review papers conclude about food and dietary patterns? Food Nutr Res 2013; 57:20523.. One of the methods proposed is to estimate dietary indices elaborated from dietary guideline recommendations. The aim of these indices is to characterize food consumption, estimate overall dietary quality, and represent adherence to dietary recommendations ²2. Lazarou C, Newby PK. Use of dietary indices among children in developed countries. Adv Nutr 2011; 2:295-303.^,44. Hu FB. Dietary pattern analysis: a new direction in nutritional epidemiology. Curr Opin Lipidol 2002; 13:3-9..

The Brazilian Healthy Eating Index – Revised (BHEI-R) for the Brazilian population ⁵5. Previdelli AN, Andrade SC, Pires MM, Vívolo SRF, Fisberg RM, Marchioni DML. A revised version of the Healthy Eating Index for the Brazilian population. Rev Saúde Pública 2011; 45:794-8.characterizes dietary quality based on components related to food and nutrient groups, considering the recommendations regarding nutritional aspects affording protection against non-communicable chronic diseases. This index was developed according to the Dietary Guidelines for the Brazilian Population⁶6. Coordenação-Geral da Política de Alimentação e Nutrição, Secretaria de Atenção à Saúde, Ministério da Saúde. Guia alimentar para a população brasileira: promovendo a alimentação saudável. Brasília: Ministério da Saúde; 2006. (Série A. Normas e Manuais Técnicos). and the recommendations of the Brazilian Society of Cardiology ⁷7. Sociedade Brasileira de Cardiologia. IV diretriz brasileira sobre dislipidemias e prevenção da aterosclerose. Arq Bras Cardiol 2007; 88 Suppl 1:2-19.and U.S. Institute of Medicine ⁸8. Institute of Medicine. Food and nutrition board dietary reference intakes for water, potassium, sodium, chloride, and sulfate. Washington DC: The National Academic Press; 2004.for saturated fat and sodium consumption, respectively. The BHEI-R is also based on the method proposed by Guenther et al. ⁹9. Guenther PM, Reedy J, Krebs-Smith SM, Reeve BB, Basiotis PP. Development and evaluation of the Healthy Eating Index-2005: technical report. http://www.cnpp.usda.gov/HealthyEatingIndex.htm (accessed on 25/Mar/2014).
http://www.cnpp.usda.gov/HealthyEatingIn... for the Healthy Eating Index¹⁰10. Kennedy ET, Ohls J, Carlson S, Fleming K. The Healthy Eating Index: design and applications. J Am Diet Assoc 1995; 95:1103-8., which takes into consideration dietary guidelines for Americans in 2005 and emphasizes aspects such as the consumption of whole grains, vegetables, good quality fats, and energy intake derived from solid fat, added sugar, and alcoholic beverages (SoFAS). Furthermore, Andrade et al. ¹¹11. Andrade SC, Previdelli AN, Marchioni DML, Fisberg RM. Evaluation of the reliability and validity of the Brazilian Healthy Eating Index Revised. Rev Saúde Pública 2013; 47:675-83. found that the BHEI-R is a reliable instrument and structurally valid to assess and monitor the quality of Brazilian diets.

However, few studies evaluated the concordance of dietary indices by comparing the results obtained by different instruments used to assess food consumption. According to Moeller et al. ¹²12. Moeller SM, Reedy J, Millen AE, Dixon LB, Newby PK, Tucker KL, et al. Dietary patterns: challenges and opportunities in dietary patterns research. J Am Diet Assoc 2007; 107:1233-9., the reproducibility and validity of dietary indices rely on the dietary assessment instrument and the objective evaluation of the diet. Thus, the quality of information on food consumption is essential for the success of epidemiological studies aimed at estimating the relationship between diet quality and health outcomes, and may be ensured by studies evaluating the concordance between dietary data estimated from two different instruments ¹³13. Cardoso MA. Desenvolvimento, validação e aplicações de questionários de frequência alimentar em estudos epidemiológicos. In: Kac G, Sichieri R, Gigante DP, organizadores. Epidemiologia nutricional. Rio de Janeiro: Editora Fiocruz/São Paulo: Atheneu; 2007. p. 201-12.. A reasonable concordance and a high correlation coefficient was observed for dietary quality index obtained from food frequency questionnaires (FFQ) and from food records ¹⁴14. Newby PK, Hu FB, Rimm EB, Smith-Warner SA, Feskanich D, Sampson L, et al. Reproducibility and validity of the Diet Quality Index Revised as assessed by use of a food-frequency questionnaire. Am J Clin Nutr 2003; 78:941-9.^,1515. Huybrechts I, Vereecken C, De Bacquer D, Vandevijvere S, Van Oyen H, Maes L, et al. Reproducibility and validity of a diet quality index for children assessed using a FFQ. Br J Nutr 2010; 104:135-44.. Therefore, the aim of this study was to assess the concordance between BHEI-R estimated from an FFQ and from adjusted means of multiple 24-hour recalls (24h-R).

Methods

This study used data from a study designed to validate an FFQ (applied with the support of food photographs elaborated to assist the application of the FFQ) developed for adolescents. The study was performed in a public elementary school located in the municipality of Niterói, Rio de Janeiro, Brazil. The study was approved by the Ethics Research Committee of the Institute of Social Medicine at the State University of Rio de Janeiro, and participation was dependent on the responsible adult or individual, aged 18 years or older, who provided informed consent. In addition, authorization to develop the study in this teaching unit was obtained from the State Department of Education and the School Director.

Study design and participant selection

The original study was conducted in 2009 to validate an FFQ, with supporting food photographs. For the study, 120 adolescents of both sexes (60 boys and 60 girls) between 12 and 18 years old were selected. The sample size met the recommendations of Cade et al. ¹⁶16. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilization of food-frequency questionnaires – a review. Public Health Nutr 2002; 5:567-87. for FFQ validation studies of at least 50 individuals and preferably 100 or more when the Bland-Altman method is used for the analyses¹⁷17. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 8:307-10., and a sample of between 100 and 200 individuals for the correlation coefficient. A list of students, including information on their sex and age, from the sixth grade of elementary school up to the third year of high school, was obtained for the random selection of the adolescents.

Food consumption assessment

Data collection was performed by applying a semi-quantitative FFQ developed for adolescents of Rio de Janeiro, which consisted of 93 food items and was properly analyzed for reproducibility ¹⁸18. Araújo MC, Ferreira DM, Pereira RA. Reliability of a semi-quantitative food frequency questionnaire designed for adolescents from the Rio de Janeiro Metropolitan Area, Brazil. Cad Saúde Pública 2008; 24:2775-86. and validity ¹⁹19. Araújo MC, Yokoo EM, Pereira RA. Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc 2010; 110:1170-7.. The detailed description of the FFQ elaboration used in this study can be found in the study by Araújo et al. ²⁰20. Araújo MC, Veiga GV, Sichieri R, Pereira RA. Development of a semiquantitative food frequency questionnaire for adolescents from the Rio de Janeiro metropolitan area, Brazil. Rev Nutr 2010; 23:179-89.. The FFQ was applied in personal interviews held by trained nutritionists, with the support of food photographs validated for FFQ application in adolescents ²¹21. Brito AP, Guimarães CP, Pereira RA. Evaluation of photographs supporting an FFQ developed for adolescents. Public Health Nutr 2014; 17:139-44..

Three nonconsecutive 24h-R (two weekdays, one weekend day) were obtained between three weeks and one month after the FFQ administration. The multiple-pass method was used to apply 24h-R ²²22. Conway JM, Ingwersen LA, Vinyard BT, Moshfegh AJ. Effectiveness of the US Department of Agriculture 5-step multiple-pass method in assessing food intake in obese and nonobese women. Am J Clin Nutr 2003; 77:1171-8.. Photographs of common tools and containers were presented to support the estimate of quantities reported by the 24h-R. The application sequence of the methods used followed Nelson’s guidelines ²³23. Nelson M. The validation of dietary questionnaires. In: Margetts BM, Nelson M, editors. Design concepts in nutritional epidemiology. 2nd Ed. Oxford: Oxford University Press; 1997. p. 266-95., in which the FFQ is applied before the reference method to maintain independence in the questionnaire responses.

Energy and nutrient consumptions were assessed by using the NutWin software (São Paulo Federal University, São Paulo, Brazil). When foods were not available on the NutWin software list, the information was obtained from theBrazilian Food Composition Table²⁴24. Núcleo de Estudos e Pesquisas em Alimentação, Universidade Estadual de Campinas. Tabela de composição de alimentos – TACO. 4a Ed. Campinas: Núcleo de Estudos e Pesquisas em Alimentação, Universidade Estadual de Campinas; 2011. and the Nutrition Data System for Research (Nutrition Coordinating Center, University of Minnesota, Minneapolis, USA). Foods were classified according to the groups presented in the Dietary Guidelines for the Brazilian Population⁶6. Coordenação-Geral da Política de Alimentação e Nutrição, Secretaria de Atenção à Saúde, Ministério da Saúde. Guia alimentar para a população brasileira: promovendo a alimentação saudável. Brasília: Ministério da Saúde; 2006. (Série A. Normas e Manuais Técnicos).. The ingredients of preparations involving more than one food group, such as sandwiches and pizzas, were segregated into their ingredients, which were classified into their respective groups. Standardized recipes, proposed by Fisberg & Villa ²⁵25. Fisberg RM, Villar BS. Manual de receitas e medidas caseiras para cálculo de inquérito alimentar. São Paulo: Signus; 2002. and Novaes ²⁶26. Novaes EM. Manual de orientação para a porcionamento de alimentos e preparações de restaurante coletivo. São Paulo: Universidade Metodista de São Paulo; 2003., were used for this procedure.

The intra- and inter-individual variabilities were estimated for the set of three 24h-R and means of energy and nutrients were deattenuated for intra-individual variability, according to the method proposed by the Iowa State University ²⁷27. Nusser SM, Carriquiry AL, Dodd KW, Fuller WA. A semiparametric transformation approach to estimating usual intake distributions. J Am Stat Assoc 1996; 91:1440-9.^,2828. Nusser SM, Fuller WA, Guenther PM. Estimating usual dietary intake distributions: adjusting for measurement error and non-normality in 24-hour food intake data. In: Lyberg L, Biemer PP, Collins M, De Leeuw ED, Dippo C, Schwarz N, et al., editors. Survey measurement and process quality. New York: Wiley and Sons; 1997. p. 689-709., by using the Software for Intake Distribution Estimation (PC-Side version 1.0, Iowa State University, Ames, USA) ²⁹29. Carriquiry AL. Estimation of usual intake distributions of nutrients and foods. J Nutr 2003; 133:601S-8S..

Diet quality

Dietary quality was evaluated by using the BHEI-R for Brazilians ⁵5. Previdelli AN, Andrade SC, Pires MM, Vívolo SRF, Fisberg RM, Marchioni DML. A revised version of the Healthy Eating Index for the Brazilian population. Rev Saúde Pública 2011; 45:794-8., assessed by attributing a score to 12 components belonging to groups of foods, nutrients, and the concomitant presence of nutrients and foods, which characterize different aspects of the diet.

The score assigned to each component is expressed in three levels (maximum, intermediate, and minimum), and the method of calculation is based on nutrient or food group density per 1,000kcal. The score is attributed to different food groups according to the Brazilian dietary guidelines recommendations ⁶6. Coordenação-Geral da Política de Alimentação e Nutrição, Secretaria de Atenção à Saúde, Ministério da Saúde. Guia alimentar para a população brasileira: promovendo a alimentação saudável. Brasília: Ministério da Saúde; 2006. (Série A. Normas e Manuais Técnicos). which consider a 2,000kcal diet and corresponding number of portions per 1,000kcal, to which the highest score was assigned. According to the recommendations of the Brazilian Society of Cardiology ⁷7. Sociedade Brasileira de Cardiologia. IV diretriz brasileira sobre dislipidemias e prevenção da aterosclerose. Arq Bras Cardiol 2007; 88 Suppl 1:2-19., the maximum score for “saturated fat” corresponds to a maximum consumption of 7% of the total energy intake. For “sodium,” the maximum score corresponds to diets with less than 0.75g/1,000kcal, as recommended by the U.S. Institute of Medicine ⁸8. Institute of Medicine. Food and nutrition board dietary reference intakes for water, potassium, sodium, chloride, and sulfate. Washington DC: The National Academic Press; 2004.. Finally, for SoFAS, the limits (maximum and minimum) were based on results related to the consumption of these components, obtained in a population-based survey that was conducted in 2003 in the city of São Paulo ³⁰30. Castro MA, Barros RR, Bueno MB, César CLG, Fisberg RM. Trans fatty acid intake among the population of the city of Sao Paulo, Brazil. Rev Saúde Pública 2009; 43:991-7.. The maximum score for this component was assigned to diets in which solid fats, alcohol, and added sugar provided less than 10% of the total energy consumption, corresponding to the 85^th distribution percentile for the consumption of these components by the population of São Paulo. Detailed descriptions of the criteria for minimum, intermediate, and maximum scores of each component can be found in Previdelli et al. ⁵5. Previdelli AN, Andrade SC, Pires MM, Vívolo SRF, Fisberg RM, Marchioni DML. A revised version of the Healthy Eating Index for the Brazilian population. Rev Saúde Pública 2011; 45:794-8..

Moreover, it is important to highlight that Previdelli et al. ⁵5. Previdelli AN, Andrade SC, Pires MM, Vívolo SRF, Fisberg RM, Marchioni DML. A revised version of the Healthy Eating Index for the Brazilian population. Rev Saúde Pública 2011; 45:794-8. chose to include legumes in the “vegetables” and “meat, eggs, and legumes” components to develop the BHEI-R due to their significant contribution to Brazilian dietary habits, as they are an important source of protein, fibers, and minerals. Hence, the amount of energy provided by the food group of legumes (consisting of beans, peas, chickpeas, fava beans, and lentils), after having contributed to reach the maximum score for “meat, eggs, and legumes,” was computed along with “total vegetables” and “DGOVL” (dark green and orange vegetables and legumes), arousing the interest to evaluate the real contribution of this group to each component, according to the collection method.

Weight status

The weight profile was evaluated by using electronic scales with a capacity of up to 150kg, in increments of 100g (Plenna Sport, São Paulo, Brazil). Height was measured in duplicate by using a portable stadiometer, in increments of 0.1cm (Altura Exata, Belo Horizonte, Brazil), and assuming a maximum variation of 0.5cm between two measures and then using their mean value. To measure weight and height, adolescents were asked to wear light clothes, and to remain barefoot and in a standing position ³¹31. Gordon CC, Chumlea WC, Roche AF. Stature, recumbent, length, and weight. In: Lohman TG, Roche AF, Martorell R, editors. Anthropometric standardization reference manual. Champaign: Human Kinetics Books; 1988. p. 3-8.. Weight status was evaluated by calculating body mass index (BMI = weight/height²), classified according to cutoff points proposed by the World Health Organization ³²32. Onis M, Onyango AW, Borghi E, Siyam A, Nishida C, Siekmanna J. Development of a WHO growth reference for school-aged children and adolescents. Bull World Health Organ 2007; 85:660-7. and depending on sex and age.

Data analyses

Statistical analyses were performed by using the Statistical Package for Social Science software version 19.0 (IBM Corp., Armonk, USA). Continuous variables were analyzed for normal distributions by performing the Kolmogorov-Smirnov test. Differences in the distribution, observed according to age group and weight status between boys and girls, were evaluated by using the chi-square test. To assess differences (mean and standard deviation) in total energy and nutrient consumption, estimated by using the FFQ and multiple 24h-R, the paired Student t test was used.

Five methods were used to assess the relative agreement between dietary quality estimated with the FFQ in comparison with the use of multiple 24h-R. The total mean BHEI-R and its components of BHEI-R for the total population were compared according to the collection instrument, sex, and weight status by using the paired Wilcoxon test and paired Student t test. Another indicator, which refers to the total mean BHEI-R and its components, was the mean difference (FFQ - 24h-R) and its corresponding 95% confidence interval (95%CI), in which a difference was considered significant when the 95%CI did not include the zero.

The correlation between the scores attributed to the total BHEI-R and its components was evaluated according to the collection instrument, by using the command of partial correlations, adjusted for age.

The concordance between the scores referring to the total BHEI-R and its components, according to the collection instrument, was evaluated as follows: (1) by the intraclass correlation coefficient (ICC) and (2) the Bland-Altman method ¹⁷17. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 8:307-10.. The ICC and its respective 95%IC estimate of the total proportion can be attributable to the variability between individuals and is considered adequate when higher than 0.75³³33. Fleiss JL. Statistical methods for rates and proportions. 2nd Ed. New York: John Wiley & Sons; 1999..

The scores referring to the total BHEI-R and its components, obtained by using the two instruments, were transformed into natural logarithms and applied to the method proposed by Bland and Altman ¹⁷17. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 8:307-10., estimating the mean concordance and its respective 95% limits of agreement (LOA). To apply this method, the difference (FFQ - 24h-R) and the mean (FFQ + 24h-R/2) between the two instruments were calculated. LOA provides the range containing 95% of individual differences in scores referring to the total BHEI-R and its components between the two instruments, as well as the dissemination of the total sample concordance. Lower and higher LOAs were estimated as follows: [mean difference ± (1.96 x standard deviation of the differences between instruments)]. As data were analyzed upon their logarithmic transformation, the mean concordance and LOA were expressed in percentages, as the exponential of a difference is a ratio (x 100).

A mean concordance of 100% indicates a perfect agreement between the two instruments. A concordance of 110%, for example, indicates that on average, the FFQ overestimates the value obtained by multiple 24h-R by 1.1 times. Therefore, LOAs between 50% and 200% indicate that 95% of estimates for all the individuals were from 0.5 to 2.0 times the value estimated by using multiple 24h-R. In this study, LOAs between 50% and 200% were considered as adequate ³⁴34. Ambrosini GL, van Roosbroeck SAH, Mackerras D, Fritschi L, Klerk NH, Musk W. The reliability of ten-year dietary recall: implications for cancer research. J Nutr 2003; 133:2663-8.. Moreover, simple models of linear regression were developed between the measurement differences as the dependent variable and measurement mean as the independent variable ³⁵35. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8:135-60.. In this analysis, the β regression coefficient was expected to be close to zero and not statistically significant, demonstrating that the concordance between the two measurements is not influenced by their magnitude (H₀: β = 0.00 α = 0.05).

Results

Of the 120 adolescents invited to participate in the study, 110 (92%) provided informed consent. Of these adolescents, 106 (89%) participated in the FFQ validation with supporting photographs and were included in this study. The mean (SD) age of the subjects was 14.5 (1.75) years, and 57% of them were female. Regarding age distribution, 56% of the adolescents were between 12 and 14 years old. Regarding sex, no statistically significant difference was observed (p = 0.35). Excess weight was detected in 25% of the adolescents and was more common among the girls than the boys (32% vs. 15%, p = 0.05) (data not shown in tables).

Table 1 shows the lack of agreement for all the parameters evaluated in scores referring to “total fruits,” “whole fruits,” “whole cereal,” “sodium,” and “SoFAS” when the FFQ and multiple 24h-R were compared. No significant difference was observed in scores referring to “dark green and orange vegetables & legumes” (DGOVL) and “total grains,” although poor agreement (ICC = 0.29 and 0.08, respectively) and a low correlation coefficient (r = 0.16 and 0.05, respectively) were observed. Better agreement (ICC = 0.54) and correlation (r = 0.37, p < 0.01) were observed for “milk and derivatives,” however, a significant difference was observed between the scores estimated by using the two instruments (p < 0.01). Scores referring to the total BHEI-R, “total vegetables,” and “saturated fat” were statistically different (p < 0.01), with an ICC and a correlation coefficient showing poor agreement (ICC = 0.35, 0.39, and 0.15, respectively) and correlation (r = 0.21, 0.24, and 0.08). Finally, no differences were observed between the estimated scores for “meat, eggs, and legumes” and “oils, oilseeds, and fat fish content” [oils]) (p > 0.05). However, these were less discriminated with limitations to estimate ICC and the correlation coefficient; it was not possible to calculate the p value, as the estimates were the same for all the individuals.

According to the Bland-Altman method, the mean agreement estimated for the total BHEI-R and its components varied from 68% for “whole cereals” to 147% for “whole fruits,” as the estimated concordance between the two instruments varied according to the intake of almost all the components, except for the total BHEI-R, “total vegetables,” and “total grains” (p > 0.05). According to the LOA, for most FFQ components, they could underestimate or overestimate scores estimated by using multiple 24h-R by approximately 50%. For example, a 95% LOA for “total vegetables” was 58% and 152%, which means that for 95% of the adolescents evaluated in this study, the estimate of this component provided by the FFQ was underestimated by 42% and overestimated by 52% when compared with multiple 24h-R. Large LOAs were observed for “total fruits,” “whole fruits,” “DGOVL,” “whole cereals,” “milk and derivatives,” “saturated fat,” and “sodium” (Table 2).

Differences between scores referring to the total BHEI-R and its components, according to the collection instrument, presented a significant relationship with sex and weight status of adolescents, except for “DGOVL,” “meat, eggs, and legumes,” and “oils,” as these components showed no significant difference between the instruments in any of the groups evaluated (Table 3).

For data from the FFQ, the contribution of the legume group by “meat, eggs, and legumes” was 40.6kcal/1,000kcal and by “total vegetables” and “DGOVL” was 21.2kcal/1,000kcal. Meanwhile, data from multiple 24h-R revealed the contrary, that is, a contribution by “meat, eggs, and legumes” of 24.0kcal/1,000kcal, and by “total vegetables” and “DGOVL” of 36.9kcal/1,000kcal. The differences observed for the contribution from the legume group by “meat, eggs, and legumes,” “total vegetables,” and “DGOVL” were statistically significant (p < 0.01 between the two instruments; paired Wilcoxon test). To better understand these differences, the content (kcal/1,000kcal) of “meat, eggs, and legumes” (a component formed by all kinds of beef, pork, lamb, of hunting, poultry, fish, and eggs) was evaluated also before adding calories from the legume group. In this analysis, the contributions from this component were 134.8kcal/1,000kcal (95%CI: 126.4; 143.1) and 167.0kcal/1,000kcal (95%CI: 155.9; 178.2) for data obtained by the FFQ and multiple 24h-R, respectively, with a significant difference between the instruments (p < 0.01; paired Wilcoxon test). Therefore, the consumption of meat and eggs estimated by using multiple 24h-R was higher than that estimated by using the FFQ, reducing the contribution of legumes by “meat, eggs, and legumes” when BHEI-R was evaluated based on multiple 24h-R estimates. Subsequently, the contribution by “total vegetables” and “DGOVL” was higher. Hence, the score referring to “total vegetables” and “DGOVL” by multiple 24h-R was higher than that by the FFQ (data not shown in tables).

Discussion

The results obtained in this study indicate that the BHEI-R was strongly dependent on the instrument used to obtain the information on food consumption and showed poor agreement between the BHEI-R estimated by the FFQ or multiple 24h-R. The score referring to the total BHEI-R showed poor agreement. Greater agreement was observed for the components and, in particular, for specific items consumed with higher frequency, such as milk and dairy. For foods that are consumed occasionally, such as fruits, the concordance was poor, although it presented statistically different mean values. For highly consumed food such as oils and meats, the mean values were correlated, although the concordance could not be estimated owing to the distribution of the values. Finally, for poorly consumed foods such as whole grains, the mean difference was small.

Poor eating habits during adolescence, such as excessive fat (especially saturated), cholesterol, salt consumption, and poor fruit and vegetable intakes may predict the occurrence of obesity and cardiovascular diseases in both adolescence ³⁶36. Daniels SR. Diet and primordial prevention of cardiovascular disease in children and adolescents. Circulation 2007; 116:973-4.^,3737. Trovato GM. Behavior, nutrition and lifestyle in a comprehensive health and disease paradigm: skills and knowledge for a predictive, preventive and personalized medicine. EPMA J 2012; 22:8. and adulthood ³⁷37. Trovato GM. Behavior, nutrition and lifestyle in a comprehensive health and disease paradigm: skills and knowledge for a predictive, preventive and personalized medicine. EPMA J 2012; 22:8.^,3838. Li L, Law C, Power C. Body mass index throughout the life-course and blood pressure in mid-adult life: a birth cohort study. J Hypertens 2007; 25:1215-23.. Therefore, food intake evaluation is an essential tool for monitoring health status ³⁹39. Bertin RL, Karkle ENL, Ulbrich AZ, Neto AS, Bozza R, Araujo IQ, et al. The nutritional status and dietary intake of adolescents in public schools in the city of São Mateus do Sul, in the State of Paraná, Brazil. Rev Bras Saúde Matern Infant 2008; 8:435-43.. Hence, the use of food standards, including those previously defined as dietary quality indices, is considered an alternative approach to overcome limitations to assessing the diet-disease relationship ⁴4. Hu FB. Dietary pattern analysis: a new direction in nutritional epidemiology. Curr Opin Lipidol 2002; 13:3-9., as they aim to cover food consumption in general ¹1. Fransen HP, Ocké MC. Indices of diet quality. Curr Opin Clin Nutr Metab Care 2008; 11:559-65.^,22. Lazarou C, Newby PK. Use of dietary indices among children in developed countries. Adv Nutr 2011; 2:295-303.^,33. Wirfält E, Drake I, Wallström P. What do review papers conclude about food and dietary patterns? Food Nutr Res 2013; 57:20523..

Indices assessing dietary quality have received increasing attention for being methodological tools for simple collection and evaluation, besides allowing for estimating the adherence to dietary recommendations for healthy diets, monitoring changes in food consumption in the population studied, and investigating the associations with health risk factors ¹1. Fransen HP, Ocké MC. Indices of diet quality. Curr Opin Clin Nutr Metab Care 2008; 11:559-65.^,22. Lazarou C, Newby PK. Use of dietary indices among children in developed countries. Adv Nutr 2011; 2:295-303.^,4040. Waijers PMCM, Feskens EJM, Ocké MC. A critical review of predefined diet quality scores. Br J Nutr 2007; 97:219-31.. However, although the validity of these indices has been investigated for nutrient adequacy and risk of developing chronic diseases ¹1. Fransen HP, Ocké MC. Indices of diet quality. Curr Opin Clin Nutr Metab Care 2008; 11:559-65.^,22. Lazarou C, Newby PK. Use of dietary indices among children in developed countries. Adv Nutr 2011; 2:295-303.^,33. Wirfält E, Drake I, Wallström P. What do review papers conclude about food and dietary patterns? Food Nutr Res 2013; 57:20523.^,4040. Waijers PMCM, Feskens EJM, Ocké MC. A critical review of predefined diet quality scores. Br J Nutr 2007; 97:219-31.^,4141. Arvaniti F, Panagiotakos DB. Healthy indexes in public health practice and research: a review. Crit Rev Food Sci Nutr 2008; 48:317-27., few studies compared the results obtained by using different instruments such as the FFQ and multiple 24h-R ¹⁴14. Newby PK, Hu FB, Rimm EB, Smith-Warner SA, Feskanich D, Sampson L, et al. Reproducibility and validity of the Diet Quality Index Revised as assessed by use of a food-frequency questionnaire. Am J Clin Nutr 2003; 78:941-9.^,1515. Huybrechts I, Vereecken C, De Bacquer D, Vandevijvere S, Van Oyen H, Maes L, et al. Reproducibility and validity of a diet quality index for children assessed using a FFQ. Br J Nutr 2010; 104:135-44.^,4242. Ambrosini GL, O’Sullivan TA, Klerk NH, Mori TA, Beilin LJ, Oddy WH. Relative validity of adolescent dietary patterns: a comparison of a FFQ and 3d food record. Br J Nutr 2011; 105:625-33.. Therefore, to our knowledge, this is the first study of its kind to be conducted in a population of Brazilian adolescents.

The two studies that used a similar methodology to the one used in this study, estimating the correlation between a dietary quality index obtained by FFQ and multiple food recalls, found correlations higher than observed among Brazilian adolescents. In the Health Professionals Follow-up Study, Newby et al. ¹⁴14. Newby PK, Hu FB, Rimm EB, Smith-Warner SA, Feskanich D, Sampson L, et al. Reproducibility and validity of the Diet Quality Index Revised as assessed by use of a food-frequency questionnaire. Am J Clin Nutr 2003; 78:941-9. used a dietary quality index obtained by using two FFQs and compared it to the index obtained by using two weekly food records among men between 40 and 75 years old. They obtained a correlation of 0.66 for the first FFQ and 0.72 for the second FFQ. Moreover, the authors observed that the similarity of the index obtained by using the second FFQ and that obtained by using food records allowed for establishing a positive association with the intake of vitamins B-6 and C, fiber, folic acid, magnesium, calcium, and carotene; and an inverse correlation with total, saturated, and monounsaturated fats, and cholesterol (p < 0.05) ¹⁴14. Newby PK, Hu FB, Rimm EB, Smith-Warner SA, Feskanich D, Sampson L, et al. Reproducibility and validity of the Diet Quality Index Revised as assessed by use of a food-frequency questionnaire. Am J Clin Nutr 2003; 78:941-9.. Therefore, Huybrechts et al. ¹⁵15. Huybrechts I, Vereecken C, De Bacquer D, Vandevijvere S, Van Oyen H, Maes L, et al. Reproducibility and validity of a diet quality index for children assessed using a FFQ. Br J Nutr 2010; 104:135-44., estimated the quality of preschool diets for children between 2.5 and 6.5 years of age, comparing the index obtained by using one FFQ and that obtained by using three food records. They observed a correlation of 0.82 for the total score and, categorizing the individuals by tertiles, found that 62% of children belonged to the same category and only 3% were classified in the extremes ¹⁵15. Huybrechts I, Vereecken C, De Bacquer D, Vandevijvere S, Van Oyen H, Maes L, et al. Reproducibility and validity of a diet quality index for children assessed using a FFQ. Br J Nutr 2010; 104:135-44..

According to Ambrosini et al. ⁴²42. Ambrosini GL, O’Sullivan TA, Klerk NH, Mori TA, Beilin LJ, Oddy WH. Relative validity of adolescent dietary patterns: a comparison of a FFQ and 3d food record. Br J Nutr 2011; 105:625-33., the assessment of dietary indicators during childhood and adolescence is an important approach to understand how they can affect short and long-term health. However, no study was found evaluating the agreement between diet quality indices obtained from different dietary assessment methods assessing only adolescents. Waijers et al. ⁴⁰40. Waijers PMCM, Feskens EJM, Ocké MC. A critical review of predefined diet quality scores. Br J Nutr 2007; 97:219-31. conducted a literature review on indices commonly used to assess dietary quality and identified 20 indices. The authors found that arbitrary choices and correlations between dietary components were not properly evaluated. They also highlighted poor associations between indices and nutrient adequacy, and pointed out that the indices existing so far do not predict morbidity or mortality significantly better than the individual evaluation of dietary components ⁴⁰40. Waijers PMCM, Feskens EJM, Ocké MC. A critical review of predefined diet quality scores. Br J Nutr 2007; 97:219-31.. Arvaniti & Panagiotakos ⁴¹41. Arvaniti F, Panagiotakos DB. Healthy indexes in public health practice and research: a review. Crit Rev Food Sci Nutr 2008; 48:317-27. identified 23 indices used to assess dietary quality and found failures in identifying intercorrelations between its components and estimating specific health outcomes.

However, Román-Viñas et al. ⁴³43. Román-Viñas B, Barba LR, Ngo J, Martínez-González MA, Wijnhoven TMA, Serra-Majem L. Validity of dietary patterns to assess nutrient intake adequacy. Br J Nutr 2009; 101 Suppl 2:S12-S20.defined validity as the degree to which the index measures exactly what it intends to measure. The authors pointed out that the effectiveness of an instrument to evaluate dietary indices should consider the purpose for which it was developed; that is, if an index was not created with the aim of assessing nutrient adequacy, its ability to discriminate the consumption by the individuals is compromised. Therefore, it is important to note that the reproducibility and validity analysis of a dietary index is only the first step in developing a new method to evaluate diet¹⁵15. Huybrechts I, Vereecken C, De Bacquer D, Vandevijvere S, Van Oyen H, Maes L, et al. Reproducibility and validity of a diet quality index for children assessed using a FFQ. Br J Nutr 2010; 104:135-44..

Hence, this study contributes to identify the reliance of the BHEI-R and its components on the estimate, according to the instrument used to obtain information on food consumption among adolescents. This notion should be considered because dietary indices such as the BHEI-R are generally estimated to better represent the complexity of eating habits.

Therefore, further studies are required to optimize the performance of dietary quality indices and to minimize their flaws and gaps in order to predict conditions for which they play dominant roles and to provide a measurement summary of dietary quality that can be used in public policy and scientific research. Consequently, individual aspects should be considered in dietary evaluation since dietary indices may correlate with characteristics such as sex ⁴4. Hu FB. Dietary pattern analysis: a new direction in nutritional epidemiology. Curr Opin Lipidol 2002; 13:3-9.^,4444. Neutzling MB, Assunção MCF, Malcon MC, Hallal PC, Menezes AMB. Food habits of adolescent students from Pelotas, Brazil. Rev Nutr 2010; 23:379-88. and weight profiles ⁴⁵45. Andrade RG, Pereira RA, Sichieri R. Food intake in overweight and normal-weight adolescents in the city of Rio de Janeiro. Cad Saúde Pública 2003; 19:485-95.^,4646. Toral N, Slater B, Silva MV. Food consumption and overweight in adolescents from Piracicaba, São Paulo, Brazil. Rev Nutr 2007; 20:449-59., influencing the validity and comparison between dietary patterns.

In this study, different methodological approaches were used to assess the agreement between estimates obtained by using the two dietary assessment tools. In particular, the mean difference and 95%CI, ICC, and Bland-Altman method were applied to assess the validity of the FFQ in estimating dietary quality in adolescents. With the Bland-Altman method, it was possible to verify that a 95% LOA for most components of the BHEI-R violated the appropriate limits for underestimating and overestimating FFQ when compared with multiple 24h-R. In this analysis, the greatest differences in food consumption, that is, greater 95% LOA range, were observed for “whole fruits,” “DGOVL,” and “total fruits” and “sodium.”

This study presents several limitations that should be considered, especially those concerning the FFQ (e.g., depending on memory, food preferences, and availability of access to food that can compromise a retrospective consumption report) ⁴⁷47. Drewnovski A. Diet image: a new perspective on the food-frequency questionnaire. Nutr Rev 2001; 59:370-2.. However, this instrument also presents advantages for epidemiological studies, such that it is not subjected to the effect of intrapersonal variation, and is practical and cost-effective with regard to data collection and analysis ⁴⁸48. Fisberg RM, Marchioni DML, Colucci ACA. Assessment of food consumption and nutrient intake in clinical practice. Arq Bras Endocrinol Metab 2009; 53:617-24.. Moreover, although the main disadvantage of 24h-R is the inability to estimate usual food consumption from a specific day during data collection ¹³13. Cardoso MA. Desenvolvimento, validação e aplicações de questionários de frequência alimentar em estudos epidemiológicos. In: Kac G, Sichieri R, Gigante DP, organizadores. Epidemiologia nutricional. Rio de Janeiro: Editora Fiocruz/São Paulo: Atheneu; 2007. p. 201-12., in this study, multiple 24h-R were collected, followed by the deattenuation of data from these recalls to better control the effect of intra-individual variability and, therefore, to obtain estimates of the habitual consumption of adolescents ²⁷27. Nusser SM, Carriquiry AL, Dodd KW, Fuller WA. A semiparametric transformation approach to estimating usual intake distributions. J Am Stat Assoc 1996; 91:1440-9.^,2828. Nusser SM, Fuller WA, Guenther PM. Estimating usual dietary intake distributions: adjusting for measurement error and non-normality in 24-hour food intake data. In: Lyberg L, Biemer PP, Collins M, De Leeuw ED, Dippo C, Schwarz N, et al., editors. Survey measurement and process quality. New York: Wiley and Sons; 1997. p. 689-709.^,2929. Carriquiry AL. Estimation of usual intake distributions of nutrients and foods. J Nutr 2003; 133:601S-8S.. Alternatively, one possible explanation for the differences in underestimation/overestimation effect observed according to the instrument used can be attributed to characteristics of each instrument, i.e., the FFQ is composed by a fixed list of food, while the 24h-R is an open instrument. With the use of 24h-R, the omission of consumed food items may occur, while when applying the FFQ it is possible to observe reduced variability in the consumption of food items and overestimation of dietary intake ⁴⁹49. Pereira RA, Sichieri R. Métodos de avaliação do consumo alimentar. In: Kac G, Sichieri R, Gigante DP, organizadores. Epidemiologia nutricional. Rio de Janeiro: Editora Fiocruz/São Paulo: Atheneu; 2007. p. 181-200..

Despite this, one positive aspect of the BHEI-R is the grains segregation, that is, separately scoring refined grains (included in “total grains”) and whole cereals, as whole grains present a protective effect against cardiac diseases, whereas refined grains can increase their risk ⁵⁰50. Ye EQ, Chacko SA, Chou EL, Kugizaki M, Liu S. Greater whole-grain intake is associated with lower risk of type 2 diabetes, cardiovascular disease, and weight gain. J Nutr 2012; 142:1304-13.. However, reduced consumption of whole grains was observed among the adolescents evaluated in this study, which limited the discrimination and contribution of this component to the total BHEI-R score. In addition, Andrade et al. ¹¹11. Andrade SC, Previdelli AN, Marchioni DML, Fisberg RM. Evaluation of the reliability and validity of the Brazilian Healthy Eating Index Revised. Rev Saúde Pública 2013; 47:675-83. evaluating the validity and reliability of BHEI-R, according to the psychometric properties, observed that, despite some limitations, this index was reliable and structurally valid for assessing the quality of Brazilian diet. They also found that the content validity was supported since the BHEI-R includes the main recommendations of the 2006Brazilian Dietary Guidelines⁶6. Coordenação-Geral da Política de Alimentação e Nutrição, Secretaria de Atenção à Saúde, Ministério da Saúde. Guia alimentar para a população brasileira: promovendo a alimentação saudável. Brasília: Ministério da Saúde; 2006. (Série A. Normas e Manuais Técnicos). and made it possible to distinguish groups with known differences in diet quality ¹¹11. Andrade SC, Previdelli AN, Marchioni DML, Fisberg RM. Evaluation of the reliability and validity of the Brazilian Healthy Eating Index Revised. Rev Saúde Pública 2013; 47:675-83..

Finally, another aspect that should be considered to evaluate previously defined dietary patterns is the theoretical framework on which the creation of each index was based. According to Newby et al. ¹⁴14. Newby PK, Hu FB, Rimm EB, Smith-Warner SA, Feskanich D, Sampson L, et al. Reproducibility and validity of the Diet Quality Index Revised as assessed by use of a food-frequency questionnaire. Am J Clin Nutr 2003; 78:941-9., the creation of dietary quality indices can limit their use when based on nutrition policies, such as the BHEI-R being based on the dietary guide ⁶6. Coordenação-Geral da Política de Alimentação e Nutrição, Secretaria de Atenção à Saúde, Ministério da Saúde. Guia alimentar para a população brasileira: promovendo a alimentação saudável. Brasília: Ministério da Saúde; 2006. (Série A. Normas e Manuais Técnicos).. It is noteworthy that, as previously mentioned, the index evaluated in this study was developed based on the recommendations of the Brazilian Dietary Guidelines published in 2006. However, a new version of these guidelines was recently published ⁵¹51. Departamento de Atenção Básica, Secretaria de Atenção à Saúde, Ministério da Saúde. Guia alimentar para a população brasileira. 2a Ed. Brasília: Ministério da Saúde; 2014.. In this new guidelines, dietary recommendations were proposed in a different way to those proposed by the 2006Brazilian Dietary Guidelines. With the advancement of nutritional science, these indices may become outdated and as good as the components on which they are based, thus needing to be revised as science and nutrition policies evolve.

Conclusion

This study showed that the BHEI-R was strongly dependent on the instrument used to obtain information on food consumption among adolescents. Thus, researchers interested in using the BHEI-R to assess the quality of the diet of Brazilian adolescents, should take into consideration the effect of underestimation and overestimation that may occur in BHEI-R and its components according to the dietary assessment method used in collecting food consumption data. Moreover, these results emphasize the need for further evaluations to estimate the usefulness of the BHEI-R in predicting health risk factors and its validity to diagnose nutritional inadequacies.

Acknowledgments

This study was funded by the Brazilian National Research Council – CNPq (process 506336/2004-2). However, the CNPq had no role in the study design, analysis or writing of this manuscript and the authors have no conflicts of interest to report.

References

Publication Dates

History