Validity and reproducibility of a food frequency questionnaire to assess food group intake in adolescents

The objective of this study was to assess the validity and reproducibility of a food frequency questionnaire to assess intake of the food groups included in the food guide pyramid for adolescents (FFQ-FP). The final version of the FFQ-FP consisted of 50 food items. The study was carried out with a sample of 109 adolescents over a period of four months. A 24hr recall (24hr) was conducted four times and the FFQ-FP was conducted twice. Validity was determined by comparing the second FFQ-FP and the mean of the four 24hrs, while reproducibility was verified by comparing the results of the two FFQ-FPs. Statistical analysis was carried out using medians, standard deviations, Pearson and intraclass correlations and Kappa statistics to assess concordance. Best results were achieved for the rice (including bread, grains and starches), meats and sugars groups. Weakest correlation was observed for the variable vitamin C. The validity and reproducibility of the FFQ-FP was satisfactory for most variables. Eating; Adolescent; Reproducibility of Results Resumo O objetivo foi verificar a validade e reprodutibilidade de um questionário de frequência alimentar baseado nos grupos alimentares da pirâmide alimentar para adolescentes (QFA-PA). Após a identificação dos alimentos a serem incluídos no QFA-PA, a versão final consistiu em 50 itens. O estudo foi realizado com 109 adolescentes durante um período de quatro meses. Foram aplicados quatro recordatórios de 24h (R24h) e dois QFA-PA. A validade foi obtida pela comparação entre o segundo QFA-PA e a média dos R24h, enquanto que a reprodutibilidade foi mensurada por meio da comparação dos dois QFA-PA. As análises estatísticas foram realizadas utilizando-se médias, desvios-padrão, correlações de Pearson, e intraclasse e concordância. As variáveis melhor avaliadas foram os grupos do arroz, pão, massa, batata e mandioca, das carnes e de açúcares e doces. Os piores resultados foram observados para a vitamina C. O QFA-PA apresentou validade satisfatória e reprodutibilidade aceitável para a maioria das variáveis. Ingestão de Alimentos; Adolescente; Reprodutibilidade dos Testes 1795 ARTIGO ARTICLE http://dx.doi.org/10.1590/0102-311X00055512 Martinez MF et al. 1796 Cad. Saúde Pública, Rio de Janeiro, 29(9):1795-1804, set, 2013 Introduction Food frequency questionnaires (FFQ) are widely used in epidemiological studies to evaluate the relationship between diet and disease. Apart from having a high degree of validity and reproducibility and being practical and easy to use, this cost-effective method simplifies data analysis, does not influence food intake patterns and minimizes intrapersonal variation. FFQs include a list of foods (which may vary according to the study objectives or the foods commonly consumed by the study population) and options of frequency of consumption. The development of efficient validated tools to assess food consumption among adolescents is fundamental to discovering the association between food intake and noncommunicable chronic diseases 1,2. In light of the above, the objective of this study was to assess the validity and reproducibility of a food frequency questionnaire to assess intake of the food groups included in the food guide pyramid adapted for adolescents (FFQ-FP). Materials and methods This cross-sectional study was a part of a larger project called Eating Attitudes and their Determinants among Adolescents in the City of São Paulo. The study population consisted of 109 adolescents from three randomly selected classes in the Parque da Juventude School, which was randomly selected from 12 technical schools participating in the above project. Only individuals aged between 15 years and 19 years and 11 months, who had completed at least three 24hour recalls (24hR) and one FFQ, and who had signed an informed consent form were included in the study. Development of the FFQ-FP A database that provides information about food consumption by the adolescents 3,4,5 was created to develop the FFQ. The reported foods were grouped according to the physical characteristics and nutrient profile of relatively homogeneous food items. Eating habits were characterized by compiling a list of frequently reported and absent items included in the FFQ-FP (alcoholic beverage, soy beverage, flavored milk in cartons, fermented milk beverages, candy, bubblegum, popsicles, cereal, instant noodles, chicken nuggets, cream cheese, yogurt and sweets) that made up at least 90% of energy and macronutrient and micronutrient intake (cholesterol, calcium and iron) and a list of frequently reported items that do not provide additional energy input (water, tea/ coffee, greens and vegetables). Consumption of the foods that qualified for inclusion in the FFQFP was classified into the following frequency categories previously adopted by Colcucci 6: at least twice a day; once a day; twice to four times a week; once a week; once to three times a month; less than once a month; and never. Portion sizes were not assessed to ensure a more realistic characterization of eating habits 1. The FFQ-FP adopted the standard units of measurement of the food groups used in the food guide pyramid adapted for adolescents 4. The food items were organized in the pyramid in the following respective food groups: rice (rice, bread, crackers, pasta and cake without filling), greens (greens and vegetables), fruit (fruit), milk (milk, cheeses, cream cheese and yogurt), meat (beef, chicken, fish, pork, egg), beans (beans), sugar (sugar, milk powder, sweets) and oils (margarine or butter, mayonnaise, olive oil/oil). The reference period for reporting habitual food intake was the 12 months prior to filling out the questionnaire. Reference method and data collection The reference method used by this study to assess the validity and reproducibility of the FFQ-FP was the self-completed 24hR 7. Data was collected between March and June 2010 using four 24hR and two FFQ-FPs (FFQ-FP1 and FFQ-FP2) conducted at 30-day and 90-day intervals, respectively. Validity was assessed by comparing the results of the FFQ-FP2 with the mean of the four 24hR, while reproducibility was evaluated by comparing the results of the two FFQ-FPs.

The objective of this study was to assess the validity and reproducibility of a food frequency questionnaire to assess intake of the food groups included in the food guide pyramid for adolescents (FFQ-FP).The final version of the FFQ-FP consisted of 50 food items.The study was carried out with a sample of 109 adolescents over a period of four months.A 24hr recall (24hr) was conducted four times and the FFQ-FP was conducted twice.Validity was determined by comparing the second FFQ-FP and the mean of the four 24hrs, while reproducibility was verified by comparing the results of the two FFQ-FPs.Statistical analysis was carried out using medians, standard deviations, Pearson and intraclass correlations and Kappa statistics to assess concordance.Best results were achieved for the rice (including bread, grains and starches), meats and sugars groups.Weakest correlation was observed for the variable vitamin C. The validity and reproducibility of the FFQ-FP was satisfactory for most variables.

Introduction
Food frequency questionnaires (FFQ) are widely used in epidemiological studies to evaluate the relationship between diet and disease.Apart from having a high degree of validity and reproducibility and being practical and easy to use, this cost-effective method simplifies data analysis, does not influence food intake patterns and minimizes intrapersonal variation.FFQs include a list of foods (which may vary according to the study objectives or the foods commonly consumed by the study population) and options of frequency of consumption.The development of efficient validated tools to assess food consumption among adolescents is fundamental to discovering the association between food intake and noncommunicable chronic diseases 1,2 .
In light of the above, the objective of this study was to assess the validity and reproducibility of a food frequency questionnaire to assess intake of the food groups included in the food guide pyramid adapted for adolescents (FFQ-FP).

Materials and methods
This cross-sectional study was a part of a larger project called Eating Attitudes and their Determinants among Adolescents in the City of São Paulo.The study population consisted of 109 adolescents from three randomly selected classes in the Parque da Juventude School, which was randomly selected from 12 technical schools participating in the above project.Only individuals aged between 15 years and 19 years and 11 months, who had completed at least three 24hour recalls (24hR) and one FFQ, and who had signed an informed consent form were included in the study.

Development of the FFQ-FP
A database that provides information about food consumption by the adolescents 3,4,5 was created to develop the FFQ.The reported foods were grouped according to the physical characteristics and nutrient profile of relatively homogeneous food items.
Eating habits were characterized by compiling a list of frequently reported and absent items included in the FFQ-FP (alcoholic beverage, soy beverage, flavored milk in cartons, fermented milk beverages, candy, bubblegum, popsicles, cereal, instant noodles, chicken nuggets, cream cheese, yogurt and sweets) that made up at least 90% of energy and macronutrient and micronutrient intake (cholesterol, calcium and iron) and a list of frequently reported items that do not provide additional energy input (water, tea/ coffee, greens and vegetables).Consumption of the foods that qualified for inclusion in the FFQ-FP was classified into the following frequency categories previously adopted by Colcucci 6 : at least twice a day; once a day; twice to four times a week; once a week; once to three times a month; less than once a month; and never.Portion sizes were not assessed to ensure a more realistic characterization of eating habits 1 .
The FFQ-FP adopted the standard units of measurement of the food groups used in the food guide pyramid adapted for adolescents 4 .The food items were organized in the pyramid in the following respective food groups: rice (rice, bread, crackers, pasta and cake without filling), greens (greens and vegetables), fruit (fruit), milk (milk, cheeses, cream cheese and yogurt), meat (beef, chicken, fish, pork, egg), beans (beans), sugar (sugar, milk powder, sweets) and oils (margarine or butter, mayonnaise, olive oil/oil).
The reference period for reporting habitual food intake was the 12 months prior to filling out the questionnaire.

Reference method and data collection
The reference method used by this study to assess the validity and reproducibility of the FFQ-FP was the self-completed 24hR 7 .Data was collected between March and June 2010 using four 24hR and two FFQ-FPs (FFQ-FP1 and FFQ-FP2) conducted at 30-day and 90-day intervals, respectively.
Validity was assessed by comparing the results of the FFQ-FP2 with the mean of the four 24hR, while reproducibility was evaluated by comparing the results of the two FFQ-FPs.

Data processing
Consumption of the food items included in the 24hR was converted into the units of measurement used in the Virtual Nutri Plus software (University of São Paulo, São Paulo, Brazil).Frequency of consumption based on the abovementioned categories was converted into daily equivalent frequency of consumption.
Daily intake by pyramid food group was calculated by dividing the number of servings consumed (based on frequency of intake) by the number of servings recommended by the food guide pyramid adapted for adolescents 4,5 and multiplying the result by the daily frequency of consumption.
Total serving intake by food group was the sum of the total number of servings consumed of each food item.

Statistical analysis
Mean and standard deviation (SD) of energy and nutrient consumption and serving intake were calculated using Microsoft Excel 2007 (Microsoft Corp., USA) and SPSS v.12.0 (SPSS Inc., Chicago, USA).The significance level was set at 5% (p ≤ 0.05).
The Kolmogorov-Smirnov test was used to test for normal distribution.Variables with asymmetric distributions were transformed into their natural logarithms.A variance ratio test was carried out to correct for intrapersonal and interpersonal variation using the following equation: where r v = true correlation, r o = observed correlation between the FFQ-FP2 and 24hR average, λ = ratio of intrapersonal and interpersonal variation from the 24hR, n = number of replicates (in this case four recalls).
The variables were energy-adjusted using the method of residues described by Willet & Stampfer 8 in order to control for confounding factors caused by total consumption of energy and remove external variations 1 .

Validation and reproducibility
Differences between means were analyzed using a paired Student's t-test.Crude, deattenuated and energy-adjusted deattenuated Pearson correlation coefficients (PCC) were used to evaluate whether the FFQ-FP was effective for classifying individuals by intake levels.Crude, deattenuated and energy-adjusted deattenuated intraclass correlation coefficients (ICC) were used to test for the degree of agreement between the tools.Acceptable correlation values varied between 0.40 and 0.70 1 .
Consumption was categorized into tertiles to observe the proportion of individuals classified in same and opposite tertiles.Kappa statistics were used to evaluate concordance between surveys.

Ethical issues
This study was approved by the Research Ethics Committee of the University of São Paulo (number 133/2009).

Results
The study was carried out with a sample of 109 adolescents, of which 60 (55%) were boys.The mean age of the sample was 16 years (SD = 0.99).

Validity
Compared with the 24hR method, the FFQ-FP2 underestimated the intake of energy, water, saturated fat, protein, cholesterol, vitamins A, C and D, phosphorus and iron, and consumption in the rice (bread, pasta, potato, manioc, etc.) and meat (including eggs) groups (Table 1).
Energy-adjusted deattenuated PCCs were high for the rice (0.88), fruit (0.77), meat (0.82) and sugars (0.82) groups, and acceptable for water, the other pyramid food groups, and all macro and micronutrients (0.41 to 0.7), except polyunsaturated fat, total fiber and vitamins C, D and E (0.13-0.37).There was an increase in the coefficient values of the following groups and items after adjustment for energy intake: rice, fruits, beans, sugars, fats (except for saturated fats), cholesterol, vitamins D and E, phosphorus and iron (Table 2).
ICC values were similar to PCC values for crude data (except for vitamin A and iron), deattenuated data (except for vitamin A and iron) and energy-adjusted deattenuated data (except for water, the fruit and beans groups, total fat and vitamin D) (Table 2).
The proportion of individuals classified in the same tertile by the 24hR and FFQ-FP2 varied from 43.5% (polyunsaturated fat and vitamin D) to 71.3% (sugars).The degree of concordance between the two tools was 54.5%, and 13% of the adolescents were classified in opposite tertiles.Kappa values varied between 0.15 (polyunsaturated fat) and 0.57 (sugars) (Table 2).

Reproducibility
The results for mean energy, nutrient and servings intake from the two FFQ-FP tests are presented in Table 1; most of the values for nutrients were lower in the retest.Statistically significant differences between the FFQ-FP1 and the FFQ-FP2 were observed for carbohydrates.
Energy intake had an intermethod PCC of 0.55, while the mean value for all variables was 0.50.After adjustment for energy intake, values increased in the sugars group (0.74) (Table 3).Energy intake had an ICC of 0.53, while the mean values for crude and adjusted data were 0.55 and 0.50, respectively.Values for crude data were high in the rice, sugars (0.71) and meat groups (0.77).After adjustment for energy intake, values increased slightly in the sugars group (0.72) (Table 3).
The proportion of individuals classified in the same tertile by the two FFQ-FPs varied between 41.6% (monounsaturated fat intake) and 65.7% (sugars group).Mean concordance was 51.4% and 14% of the adolescents were classified in opposite tertiles.Kappa values varied between 0.13 (monounsaturated fat intake) and 0.51 (sugars group) (Table 3).

Discussion
Currently several options exist for assessing food intake and one the most commonly used tools are FFQs.However, there are only a limited num-ber of options of validated tools for assessing food intake in adolescents 1,9,10 .
The ideal tool should be brief, easy to administer, low-cost and provide useful information.However, specific characteristics of adolescents should be taken into account when designing FFQs, including variations in cognitive abilities, such as memory and their limited knowledge of foods and food preparation, and difficulties in estimating the size of servings 10 .Despite this,

Structure of the FFQ-FP
The number of food items included in the FFQ can vary widely and Cade et al. 12 identified the use of between five and 350 foods items.However, Egashira et al. 13 suggest that extensive lists may not evaluate food consumption accurately because they lead to fatigue and tedium when filling out the questionnaire.
The FFQ-FP used in this study included 50 food items and this may have exerted enough influence to lower the correlation coefficients.However, it should be noted that the selection of food items was based on previously tested methodologies 1,14 , with emphasis on high energy foods in the case of the 24hR.Although the formulation of the FFQ-FP included the design and use of the food groups of the food guide pyramid 15 , fruit, leafy greens and vegetables were not specified due to the high degree of variety and variability in consumption.

Validity
FFQs have the potential to play an important role in the monitoring and evaluation of food intake behavior 16,17 .In Brazil, however, studies that assess the validity of FFQs designed to assess food intake among adolescents are scarce.
The selection of an appropriate reference method is an essential element of the effective assessment of FFQs and research in this area is often hampered by the lack of a gold standard for dietary intake 18 .Furthermore, error detection methods should be as independent as possible 19,20 .The main sources of error in FFQs are participant memory, the interpretation of questions, perception of serving sizes and the limitations imposed by a fixed list of food items.Open-ended diet records do not depend on memory and are likely to result in less correlated errors 1,18 .Nevertheless, keeping a diet record is a substantial burden on participants and often influences the respondent's diet.Thus, multiple 24hr dietary recalls have been shown to be the most appropriate reference method for use in epidemiological studies designed to validate food intake 21,22 and a review by Cade et al. 12 showed that 75% of studies assessed FFQ against 24hR.
Due to the high degree of intrapersonal variability associated with this method, it is common to repeat the 24hR or use larger samples.Adjustments were therefore made to account for intrapersonal variability to minimize error due to correction of the consumption distribution.
Adjustment for energy intake led to lower correlation coefficients for most nutrients included in this study, corroborating the findings of Willett 1 .Increases in correlation coefficients after adjustment may be due to error removal due to the effect of total energy and nutrient consumption 17 .Molag et al. 2 observed that energy-adjusted correlation coefficients were 0.02-0.08higher for all nutrients, except vitamin C (0.05).In the present study, intermethod correlations were higher after adjustment (0.01-0.16).
Although there was a significant difference in the means of most dietary variables between the different methods, it is important note that correlation values of 23 (82%) out of the 28 groups analyzed by this study were above 0.40.These findings are similar to those of Voci et al. 17 and Willet 1 , who stated that correlations must be between 0.4 and 0.7 to be acceptable.Correlations less than 0.30 were observed for polyunsaturated fats, fiber and vitamins C, D and E. Similar results were observed by Marks et al. 21and Malekshah et al. 22 .
FFQ-FP correlation coefficients for fruit, greens and milk groups observed by Millen et al. 23 were higher than those observed by Spencer et al. 24 (0.17 and 0.58, respectively).The results obtained in the present study contradict those from a review of FFQ validation studies carried out by Cade et al. 12 in which mean correlations were generally lower for leafy greens and vegetables due to a number of reasons, including duplicated item counting and social desirability bias.
The results regarding fat intake using the FFQ-FP were unsatisfactory, even after energy adjustment, corroborating the findings of the European Prospective Investigation into Cancer and Nutrition (EPIC) study 25 .
The kappa values observed by this study were similar to those found by Hong et al. 26 , who reported values ranging from 0.36 to 0.54 for energy, protein and fat, and 0.14, 0.19 and 0.2 for vitamin C, calcium and fiber, respectively, indicating moderate and fair reliability.The values of the present study were lower than those observed by Assis et al. 27 in a study assessing the validity of FFQ for students (kappa values of 0.76 for fruit and 0.71 for leafy greens).In the present study, a mean of 54.5% of individuals were classified in the same tertile, corroborating the findings of Masson et al. 28 .According to the classification used by this study all variables, except fats, vitamins E, D and niacin, presented a concordance rate of over 50% for individuals classified in the same tertiles.

Reproducibility
Another important element of FFQ is the reference period for reporting food intake.The replication interval should neither be too short (because subjects may remember their previous responses and simply repeat the answers) nor too long (so as not to influence changes in diet) 29 .This study adopted a three-month interval for the FFQ-FP, which is considered a reasonable period of time.It should also be noted that the time interval can also influence changes in food intake due to seasonality and it is possible that this may have happened in this study, thus leading to lower true correlations, especially for micronutrients 30 .
Cullen et al. 10 and Marchioni et al. 29 found lower retest values, different values between two FFQ applications and significant differences for carbohydrates, which is consistent with the findings of the current study.
The PCC value observed by the present study agrees with the findings of Rockett et al. 9 who observed values between 0.26 (protein and iron) and 0.58 (calcium).
The crude ICC values of energy, vitamin C and the meats group were 0.53, 0.35 and 0.77, respectively (mean of 0.55), while adjusted values of vitamin C and sugars were 0.29 and 0.72, respectively (mean of 0.50).The values observed by the present study are similar to those found by Speck et al. 31 .Marchioni et al. 29 observed lower adjusted values (0.25 fat and 0.58 vitamin C).
The concordance rate between the two tools was 51.4% and 14% of the adolescents were classified in opposite tertiles.Similar results were found by Marchioni et al. 29 .Considering the variability of adolescent food intake, Vereecken et al. 32 determined that this rate represents a reasonable level of reproducibility of FFQ for adolescents.
Kappa values above 0.40 were observed for the variables energy, fiber and the rice, greens, fruits and sugars groups.The value for protein was similar to that found by Marchioni et al. 29 (0.33).
It is important to note that this is the first accurate FFQ that evaluates food intake based on the food pyramid developed and validated for adolescents in Brazil.The FFQ-FP represents a new approach to the assessment of dietary intake which is more visually appealing and quicker and easier to fill out and its structure facilitates the classification of individuals according intake levels.
A possible limitation of this study is the fact that the number of 24hR was insufficient to assess the intake of micronutrients, fiber and fats.Furthermore, certain variables, particularly vitamins, are subject seasonal variations.
Another limitation is that the food list used by this study was based on a study developed in a public school in Ilha Bela, State of São Paulo by Leal 3 .However, this study found mean en-ergy values for boys and girls of 2,600kcal and 2,200kcal, respectively, which is similar to the findings of Albano & Souza 33 in a study carried out in the city of São Paulo.Similar results were also found with respect to macronutrients, micronutrients and food groups for the two cities.Carbohydrate intake was lower than the recommended intake for adolescents, while protein and lipid intake was higher.The mean intake in the milk group (including cheese and yogurt) was 50% lower than the recommended intake for this age group 33,34 .Intake in the greens and fruit groups was lower than recommended in the food pyramid.This finding is similar to the findings of a study in São Paulo carried out by Toral et al. 35 .
Since this is a pilot study, the differences in eating behavior are not particularly marked.It is important to highlight that the correlations found by the present study are consistent with the literature, indicating that food intake in Ilha Bela can be used as a basis to compose the FFQ-FP list for the metropolitan region of São Paulo.

Conclusions
The validity of the FFQ-FP used in this study was satisfactory for all variables except polyunsaturated fats, vitamins C, D and E. Reproducibility was also satisfactory for all variables except water, vitamins A and C and the greens group.It is hoped that the results of this study contribute to the development of guidelines for nutrition intervention, public policies and further epidemiological studies.

Table 1
Mean (SD) of the variables energy, water, food groups and nutrients of the four 24-hour recalls (24hR) and two food frequency questionnaires (FFQ-AP) based on food groups included in the food guide pyramid adapted for adolescents (energy-adjusted and deattenuated).Parque da Juventude Technical School, São Paulo, Brazil, 2011.

Table 2
Correlation coefficients, percentage of adolescents classified in tertiles, weighted kappa, energy-adjusted deattenuated concordance of validity.Parque da Juventude Technical School, São Paulo, Brazil, 2011.FFQ-AP: food frequency questionnaire based on food groups included in the food guide pyramid adapted for adolescents.

Table 3
Correlation coefficients, percentage of adolescents classified in tertiles, weighted kappa, energy-adjusted deattenuated concordance of reproducibility.Parque da Juventude Technical School, São Paulo, Brazil, 2011.AP: food frequency questionnaire based on food groups included in the food guide pyramid adapted for adolescents.