Abstract:

Objective:

To assess the validity of a 140-item semiquantitative food frequency questionnaire (SFFQ), in Mexican adolescents and adults.

Materials and methods:

Dietary intakes using a SFFQ and two 24-hour dietary recalls (24DRs), in nonconsecutive days during the same week were measured from 178 adolescents and 230 adults participating in the Mexican National Health and Nutrition Survey-2012.Validity was evaluated using correlation coefficients (CC),deattenuated CC, linear regression models, cross-classification analysis, and the Bland-Altman method.

Results:

In adults, deattenuated correlation coefficients between the SFFQ and the 24DRs ranged from 0.30 for folate to 0.61 for saturated fat. In addition, 63% adults and 62% adolescents were classified in the same and adjacent quartile of nutrient intake when comparing data from SFFQ and 24DRs.

Conclusions:

The SFFQ had moderate validity for energy, macronutrients and micronutrients. It also had good validity to rank individuals according to their dietary intake of different nutrients.

Keywords:
food frequency questionnaire; validity; adult; adolescents; survey; Mexico

Resumen:

Objetivo:

Evaluar la validez de un cuestionario de frecuencia de consumo de alimentos semicuantitativo (CFA), en adolescentes y adultos mexicanos.

Material y métodos:

Se evaluó la ingesta de alimentos de 178 adolescentes y 230 adultos que participaron en la Encuesta Nacional de Salud y Nutrición-2012, de México, mediante un CFA y el promedio de dos recordatorios de 24-horas (24DRs). La validez se evaluó utilizando coeficientes de correlación, coeficientes de correlación corregidos (CCC) y análisis de clasificación.

Resultados:

En adultos, los CCC entre el CFA y el promedio de los 24DRs estuvieron entre 0.30 para folatos y 0.61 para grasas saturadas. Adicionalmente, 63% de los adultos y 62% de los adolescentes fueron clasificados en el mismo o adyacente cuartil cuando fueron comparados los datos del CFA y el 24DR.

Conclusión:

El CFA mostró una validez moderada para energía, macronutrimentos y micronutrimentos. Además, mostró buena validez para clasificar a los individuos de acuerdo con su ingesta dietética.

Palabras clave:
cuestionario de frecuencia de consumo; validez; adultos; adolescentes; encuestas; México

Dietary assessment represents a continuing challenge, principally in large-scale studies. The development of valid, efficient, and cost-effective instruments for measuring dietary intakes for specific populations is still a key research prioritiy.¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141.^,22. Pietinen P, Hartman AM, Haapa E, Rasanen L, Haapakoski J, Palmgren J, et al. Reproducibility and validity of dietary assessment instruments. II. A quantitative food frequency questionnaire. Am J Epidemiol 1988;128(3):667-676. Different dietary methods, such as food records (FRs), 24-hour dietary recalls (24DRs),¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141. and food frequency questionnaires (FFQ), can be used for assessing habitual dietary intake. Although FRs and 24DRs have been successfully employed and shown to be accurate methods; the time, respondents' motivation, participant's literacy, and economic constraints of these dietary instruments make them unsuitable for most studies.³3. Buzzard M. 24-hour dietary recall and food record methods. In: Willett WC, edit. Nutritional Epidemiology, 2nd edition. Oxford: Oxford University Press, 1998. On the other hand, among the various instruments planned to assess nutrient and food intake, the FFQ are used to estimate individual perception of standard food intake over a reference period of time such as months or years. In contrast with other dietary assessment methods, the FFQ are easier to administer. They are relatively less expensive, less difficult to achieve, and they also offer a rapid estimate of usual food intake;¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141.^,44. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilization of food frequency questionnaires a review. Public Health Nutr 2002;5(4):567-587. http://doi.org/dnffmv
http://doi.org/dnffmv... making them more feasible for epidemiological studies. The FFQ have been commonly used as a practical and efficient method to assess habitual diet over periods of time in dietary surveys.¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141. They have also provided reasonable measurements of energy, macronutrients, and micronutrients in different countries.⁵5. Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain C, Witschi J, et al. Reproducibility and validity of a quantitative food frequency questionnaire. Am J Epidemiol 1985;122(1):51-65.

6. Prentice RL. Dietary assessment and the reliability of nutritional epidemiology report. Lancet 2003;362(9379):182-183. http://doi.org/dbnsq6
http://doi.org/dbnsq6...

7. Hernandez-Avila M, Romieu I, Parra S, Hernandez-Avila J, Madrigal H, Willett WC. Validity and reproducibility of a food frequency questionnaire to assess dietary intake of women living in Mexico city. Salud Publica Mex 1998;40(2):133-140. http://doi.org/cqwzwz
http://doi.org/cqwzwz... ^-88. Bingham SA. Dietary assessment in the European prospective study of diet and cancer (EPIC). Eur J Cancer Prev 1997;6(2):118-124. However, all FFQ should be adapted to the target population due to the great differences in food supply, as well as the broad range of dietary habits from one population to another.¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141. In addition, it is well acknowledged that nutritional values reported from FFQ data can be subject to random and systematic measurement errors, which can profoundly influence the design, analysis, and interpretation of nutritional epidemiological studies.⁹9. Liu K. Statistical issiues related to semiquantitative food-frequency questionnaire. Am J Clin Nutr 1994;59(1 Suppl):262S-265S.^,1010. Carroll RJ, Pee D, Freedman LS, Brown CC. Statistical design of calibration studies. Am J Clin Nutr 1997;65(4 Suppl):1187S-1189S. Therefore, to correctly interpret the results of epidemiological studies that use FFQ, there is a need to know the relationship between reported intakes from the FFQ and true usual intakes.¹¹11. Jain MG, Rohan TE, Soskolne CL, Kreiger N. Calibration of the dietary questionnaire for the Canadian study of diet, lifestyle and health cohort. Public Health Nutr 2003;6(1):79-86. http://doi.org/bs6hn9
http://doi.org/bs6hn9... Until now, different methods have been employed for the validation of dietary intakes. However, limitations like cost and degree of invasiveness of the biomarkers,¹²12. Thompson FE, Subar AF, Loria CM, Reedy JL, Baranowski T. Need for technological innovation in dietary assessment. J Am Diet Assoc 2010;110(1):48-51. http://doi.org/d2p5nw
http://doi.org/d2p5nw... and others related to the FR like reduction of the response rate, and even a change in subjects' diet, are important factors limiting their use in validation studies.¹³13. Kristal AR, Feng Z, Coates RJ, Oberman A, George V. Associations of race/ethnicity, education and dietary intervention with the validity and reliability of a food frequency questionnaire: TheWomen's Health trial feasibility study in minority populations. Am J Epidemiol 1997;146(10):856-869. http://doi.org/brf6
http://doi.org/brf6... For that reason, collection of multiple 24DRs has been suggested as another comparison standard method for the validation of dietary intakes. Actually, approximately 75% of the validation studies of the FFQ are validated against multiple 24DRs, which have been found to be suitable for comparison with questionnaires to establish validity.⁴4. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilization of food frequency questionnaires a review. Public Health Nutr 2002;5(4):567-587. http://doi.org/dnffmv
http://doi.org/dnffmv...

Since 2006, as part of the Mexican National Health and Nutrition Survey 2012 (Ensanut 2012, by its Spanish acronym), a semi-quantitative food frequency questionnaire (SFFQ) was used for assessing the dietary intake of Mexican population. However, the created SFFQ has not yet been properly validated. Therefore, the main objective of the present study was to evaluate whether this SFFQ is valid in Mexican adolescent and adult populations by comparison with the average of two 24DRs.

Materials and methods

The present analysis was done with data from the Ensanut 2012, that is a probabilistic population-based survey with multi-stage stratified sampling, designed to be representative of the nation, its three main regions (North, Central and Mexico City, and South), and rural and urban areas. The survey design and sampling procedures have been previously described in detail.¹⁴14. Romero-Martínez M, Shamah-Levy T, Franco-Nuñez A, Villalpando S, Cuevas-Nasu L, Gutiérrez JP, et al. National Health and Nutrition Survey 2012: design and coverage. Salud Publica Mex 2013;55(Suppl 2):S332-S340. http://dx.doi.org/10.21149/spm.v55s2.5132
https://doi.org/10.21149/spm.v55s2.5132... Briefly, the Ensanut 2012 aims to monitor health and nutrition conditions, health program coverage, and access to health services. The survey obtained information from 50 528 households, with a response rate of 87%; 46 303 interviews with adolescents and adults were conducted.

Design and study population

For the present study, data was collected in three phases. First, identification of selected housing field; second, application of household and individual questionnaires; and third, the application of the dietary questionnaires, and anthropometric measurements. The SFFQ and the first 24DR were electronically collected after the first and second phase. Finally, the second 24DR was collected 2 or 3 days apart from the first 24DR.

Using a simple random sampling, a subsample of 186 adolescents and 278 adults (≥ 20 years) were included for a validity study. Among them, for the present analysis, 178 adolescents and 230 adults, who had complete information of one SFFQ and two 24DRs, and with no outlier energy (values between 600 and 6 000 kcal) and nutrients intake values,¹⁵15. Rosner B. Percentage points for a generalized ESD many-outlier procedure. Technometrics 1983;25(2):165-172. http://doi.org/brbv
http://doi.org/brbv... were included.

This study was conducted according to the Declaration of Helsinki guidelines, and written informed consent was obtained from either the participants or their parents (for participants below 18 years old) before they were enrolled in the study. Research Ethics Committee at the National Institute of Public Health (INSP, for its acronym in Spanish) revised and approved the informed consent forms and the study protocol.

Data collection

Dietary assessment

The semi-quantitative food frequency questionnaire

The electronic SFFQ used in the Ensanut 2012 is an adapted version of the questionnaire employed in the Ensanut 2006.¹⁶16. Rodríguez-Ramírez S, Mundo-Rosas V, Jiménez-Aguilar A, Shamah-Levy T. Methodology for the analysis of dietary data from the Mexican National Health and Nutrition Survey 2006. Salud Publica Mex 2009;51(Suppl 4):S523-S529. http://doi.org/fwfwpw
http://doi.org/fwfwpw... Briefly, the food list (101 food items) included in the questionnaire used in the Ensanut 2006, as well as the portion size, and mean weight of portion for each population group were estimated by a group of INSP researchers from the analysis of the most-consumed-foods data obtained from the National Nutrition Survey 1999. The new version of the final SFFQ (used in the Ensanut 2012) includes consumption of 140 foods during seven days prior to the date of the interview. Frequency of food items could be characterized by set categories ranging from never to six times a day. Participants also designated the portion size of the ingested food items, using predefined categories. These data were first converted to times per day, and then portion size per day was computed. To calculate the consumption of energy (kcal/day) and daily nutrients intakes, the daily frequency of consumption (portions/day) of each food was multiplied by the amount of energy and nutrients in a standard serving or portion size of that food (using the food composition tables compiled by the INSP), and the contributions of all foods were totaled using Microsoft Visual FoxPro 7.0. The SFFQ was collected by personnel trained in standardized data collection and entry procedures.

24-hour dietary recall

Participants completed two electronic 24DRs distributed over all days of the week; the first 24DR was obtained at the same time of the SFFQ. Personnel trained in standardized methods collected all the required information through face-to-face interviews, using 24DR automated multiple pass method (24DR-AMPM software version 1.0) developed and adapted to the Mexican population by a group of INSP researchers.^**López-Olmedo N, Carriquiry AL, Rodríguez-Ramírez S, RamírezSilva I, Espinosa-Montero J, Hernández-Barrera L, Campirano-Núñez F, Martínez-Tapia B, Rivera JA. Usual intake of energy and macronutrients in Mexican population. J Nutr 2016;146(Suppl), In press. For each 24DR, participants were asked in detail about their food consumption during the previous day. In brief, at the beginning of the interview, participants listed the food items they had consumed during the previous day (since the moment they woke up until they went to sleep), with no prompts from the interviewer about different possible eating occasions. A list of foods that are often forgotten was also used to elicit recall, in this case the interviewer got back to the beginning of the preliminary food list and helped the interviewer remember foods frequently omitted. Subsequently, the food list obtained in the previous steps was organized according to the time each food was consumed and the context each meal was done (place and activities). Then, detailed information on each food item was collected, for example: 1) the food was single, dish or recipe; 2) the amount of single food consumed or ingredients used in a recipe (in household measures or food weight); 3) the type of food weight (net or gross); 4) the food preparation method (raw, cooked, steamed or oven); 5) the total amount of the food served; 6) the amount of food that was not consumed; 7) how processed foods were consumed (frozen and/or packaging). Lastly, a review of the final food list was made in order to obtain additional information or to correct any specific data that had been improperly recorded. We calculated the daily mean intakes of energy, macro and micronutrients, estimated by the two 24DRs. The average 24DRs data were used as the standard to measure the relative validity of the SFFQ.

Other participant characteristics

Sociodemographic characteristics (age, sex and socioeconomic status) were obtained with predefined questionnaires. Localities with less than 2 500 residents were considered rural, and areas with 2 500 or more residents were considered urban. A household wealth index (HWI) was created using principal components analysis with household characteristics and family assets. This index was divided into tertiles, with 1 being the lowest category.

Statistical analysis

We performed descriptive analysis of the main characteristics of interest; means were computed for continuous variables; and frequencies and percentages were calculated for categorical variables.

Means and medians were computed for energy and nutrient intakes assessed by the SFFQ and the 24DRs. For the purpose of this study, the following nutrient intakes were compared: energy (kcal), protein (g), total fat (g), saturated fat (g), monounsaturated fat (g), polyunsaturated fat (g), cholesterol (mg), carbohydrates (g), dietary fiber (g), calcium (mg), magnesium (g), iron (g), zinc (g), vitamin A (ER), vitamin D (mcg), vitamin B12 (mcg), vitamin C (mg), and folate (mcg). We log-transformed the dietary data to improve the normal distribution and reduce skewness. The residual method was employed to eliminate the possibility of variation due to energy intake.⁵5. Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain C, Witschi J, et al. Reproducibility and validity of a quantitative food frequency questionnaire. Am J Epidemiol 1985;122(1):51-65.^,1717. Willett WC, Stampfer MJ. Total energy intake: implications for epidemiologic analyses. Am J Epidemiol 1986;124(1):17-27.

Pearson correlation coefficients (PCC) were computed to evaluate the relationship between the energy and nutrients from the SFFQ, log-transformed and energy adjusted, and the average of two 24DRs. We also calculated deattenuated correlation coefficients, using the approach described by Rosner, to take into account within-person variations caused by day-to-day fluctuations.¹⁸18. Rosner B, Willett WC. Interval estimates for correlation coefficients corrected for within-person variation: implications for study design and hypothesis testing. Am J Epidemiol 1988;127(2):377-386. Additionally, to assess the validity of nutrient values, derived from the SFFQ, we also used regression analysis and compared them with the mean of the two 24DRs.

Kappa statistic¹⁹19. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174. http://doi.org/dtzfj3
http://doi.org/dtzfj3... ^,2020. Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L, et al. Statistical approaches for assessing the relative validity of a food frequency questionnaire: use of correlation coefficients and the kappa statistic. Public Health Nutr 2003;6(3):313-321. http://doi.org/b38ssz
http://doi.org/b38ssz... was used to evaluate how well the SFFQ could categorize individuals into quartiles of the nutrient distributions when compared with the average 24DRs categorization. For the present analysis, subjects were classified into quartiles based on the energy-adjusted nutrient intakes. The degree of misclassification was calculated by examining the proportion of subjects classified by the reference method that fell into the same, adjacent, or extreme quartile when classified by the SFFQ.

Furthermore, Bland and Altman²¹21. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1(8476):307-310. http://doi.org/bf8tkx
http://doi.org/bf8tkx... plots were used to graphically examine the agreement for energy, carbohydrate, fat, and protein intake between the two methods.

All statistical analyses were performed with STATA statistical software version 13.0. A value of p<0.05 was considered to be statistically significant.

Results

Overall, 186 adolescents and 278 adults (≥20 y) participated in the validation study. Of these, six adolescents and 28 adults were removed from the present analysis as their estimated daily energy intake was less than 600 kcal/day or >6 000 kcal/day. Additionally, participants who left 10 percent or more food items blank on the questionnaire were also excluded (two adolescents and 20 adults). For the present analysis, a total of 178 adolescents and 230 adults who completed two 24DRs and the SFFQ were included.

Among the adolescents who participated in this study, 50% were females; and the average age was 15.5 ± 2.3 y. Of the adult participants eligible for this analysis, 58.3% were females and 41.7% were males; the mean age was 41.4 ± 12.2 y; and 62.6% were from urban areas (Table I).

In adults, values for energy and nutrients estimated by the SFFQ were slightly higher than those obtained with the 24DRs, except for protein, polyunsaturated fatty acids, fiber, and magnesium. In this age group, the SFFQ overestimated by 265 kcal the energy intake, and overestimated nutrient were particularly high for carbohydrate, calcium, vitamin A, and folate. In adolescents, all the mean values of the 24DRs were lower than those of the SFFQ, except cholesterol, vitamin D, and vitamin B12. The analysis of nutrient intakes showed significant differences in energy, saturated fat, carbohydrates, calcium, zinc, vitamin C, and folate between SFFQ and 24 DRs (Table II).

Overall, in adults, PCC of crude unadjusted data varied from 0.25 (iron) to 0.49 (saturated fat). Whereas in adolescents' coefficients for energy and crude nutrient intakes ranged from 0.21 for vitamin D to 0.51 for carbohydrates. In subjects older than 20 y, correlation coefficients (CC) above 0.40 or higher were voted for total energy intake, total fat, saturated fat, monounsaturated fat, cholesterol, carbohydrates, and calcium. In both age groups, the adjustment of total energy slightly improves the PCC in some nutrients (e.g. saturated fat, vitamin C), but decreases the values in others (e.g. polyunsaturated fat, protein). Except for that of zinc and polyunsaturated fat in adolescents, all correlations were statistically significant with p<0.05. Adjustment for within-person variability (deattenuated coefficients) increased all values of the CC, ranging from 0.30 for folate to 0.61 for saturated fat in adults and 0.19 to 0.61 for zinc and calcium in adolescents, respectively (Table III).

In the cross-classified analysis (Table IV), a subject would be correctly classified if their energy or nutrient intakes were ranked into the same or an adjacent quartile. In adults, for most of the nutrients, the proportion of subjects classified into the exact same quartile varied from 30.0% (vitamin D) to 44.8% (monounsaturated fat), whereas, the percentage of misclassification (extreme or opposite quartile) ranged from 10.8 for total fat to 19.1 for vitamin A. On the other hand, for adolescents, classification of subjects into one quartile (in the same or adjacent categories) ranged from 55.6% for polyun-saturated fat to 69.3% for carbohydrates. For this age group, extreme misclassification into opposite quartile was weak with values between 8.7% (carbohydrates) and 17.6% (folate).

According to Bland and Altman plot analysis, we found that for energy and all macronutrients, both methods showed good level of concordance. For example, for energy, approximately five adolescents and ten adults fell outside the limit of agreements. Similar results were observed for proteins, carbohydrates and lipids (Figures 1 and 2).

Discussion

The present study evaluated the relative validity of a 140-item SFFQ in a Mexican adolescent and adult population. Comparing daily nutrient intake, obtained from the SFFQ, with those derived from the average of two 24hour dietary recalls, our results indicated that the SFFQ had moderate to good relative validity (varied from 0.30 to 0.61 in adults and from 0.19 to 0.61 in adolescents) for most macroand micronutrients. Additionally, we observed a good agreement between the two methods when exploring interquartile categorization, as more than 63% adults and 62% adolescents were correctly classified into the same or adjacent quartiles for energy and most macro and micronutrients.

The absolute estimated energy, macroand micronutrient intakes derived from the SFFQ were in general higher than those derived from the reference method (24DRs), which is very often reported in previous validation studies.²⁰20. Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L, et al. Statistical approaches for assessing the relative validity of a food frequency questionnaire: use of correlation coefficients and the kappa statistic. Public Health Nutr 2003;6(3):313-321. http://doi.org/b38ssz
http://doi.org/b38ssz... ^,2222. Jaceldo-Siegl K, Knutsen SF, Sabate J, Beeson WL, Chan J, Herring RP, et al. Validation of nutrient intake using an FFQ and repeated 24 h recalls in black and white subjects of the Adventist Health Study-2 (AHS-2). Public Health Nutr 2010;13(6):812-819. http://doi.org/dj6fqz
http://doi.org/dj6fqz...

23. Zhuang M, Yuan Z, Lin L, Hu B, Wang X, Yang Y, et al. Reproducibility and relative validity of a food frequency questionnaire developed for adults in Taizhou, China. PLoS One 2012;7(11):e48341. http://doi.org/f25ftj
http://doi.org/f25ftj...

24. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7...

25. Dehghan M, Martinez S, Zhang X, Seron P, Lanas F, Islam S, et al. Relative validity of an FFQ to estimate daily food and nutrient intakes for Chilean adults. Public Health Nutr 2013;16(10):1782-1788. http://doi.org/brf8
http://doi.org/brf8...

26. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009;63(2):282-391. http://doi.org/d2m7dq
http://doi.org/d2m7dq...

27. Araujo MC, Yokoo EM, Pereira RA. Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc 2010;110(8):1170-1177. http://doi.org/dz7jg4
http://doi.org/dz7jg4... ^-2828. Xia W, Sun C, Zhang L, Zhang X, Wang J, Wang H, et al. Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS One 2011;6(5):e19656. http://doi.org/fbsgj8
http://doi.org/fbsgj8... This possibly results when people are asked to recall the frequency of a large number of foods consumed in an FFQ and thus, they might tend to overestimate their actual intake. Also, difficulty in conceptualizing the assigned portion sizes and difficulties in reporting the frequencies of usual intake could be related.¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141. For example, in a study conducted by Kristal and colleagues²⁹29. Kristal AR, Vizenor NC, Patterson RE, Neuhouser ML, Shattuck AL, McLerran D. Precision and bias of food frequency-based measures of fruit and vegetable intakes. Cancer Epidemiol Biomarkers Prev 2000;9(9):939-944. that evaluated bias of FFQ when measuring fruit and vegetable consumption, they found that estimated intakes were lower when comparing a brief questionnaire (FFQ with only seven items) with dietary recalls; whereas, when they compared an FFQ with thirty or more items with a dietary recall or food records higher intakes were observed.

Usually, the association between FFQ and reference method have been assessed by correlation coefficients,¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141. and due to the numerous measurement errors for each dietary assessment instrument, the observed correlation coefficients are measures of relative validity.²⁵25. Dehghan M, Martinez S, Zhang X, Seron P, Lanas F, Islam S, et al. Relative validity of an FFQ to estimate daily food and nutrient intakes for Chilean adults. Public Health Nutr 2013;16(10):1782-1788. http://doi.org/brf8
http://doi.org/brf8... ^,3030. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative food frequency questionnaire for Argentinean adults. PLoS One 2012;7(5):e37958. http://doi.org/brf9
http://doi.org/brf9... In general, the CC computed for each nutrient in the present study were similar or slightly lower than the ones reported in other validation studies carried out among adolescent²⁶26. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009;63(2):282-391. http://doi.org/d2m7dq
http://doi.org/d2m7dq...

27. Araujo MC, Yokoo EM, Pereira RA. Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc 2010;110(8):1170-1177. http://doi.org/dz7jg4
http://doi.org/dz7jg4... ^-2828. Xia W, Sun C, Zhang L, Zhang X, Wang J, Wang H, et al. Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS One 2011;6(5):e19656. http://doi.org/fbsgj8
http://doi.org/fbsgj8... and adult population.²²22. Jaceldo-Siegl K, Knutsen SF, Sabate J, Beeson WL, Chan J, Herring RP, et al. Validation of nutrient intake using an FFQ and repeated 24 h recalls in black and white subjects of the Adventist Health Study-2 (AHS-2). Public Health Nutr 2010;13(6):812-819. http://doi.org/dj6fqz
http://doi.org/dj6fqz...

23. Zhuang M, Yuan Z, Lin L, Hu B, Wang X, Yang Y, et al. Reproducibility and relative validity of a food frequency questionnaire developed for adults in Taizhou, China. PLoS One 2012;7(11):e48341. http://doi.org/f25ftj
http://doi.org/f25ftj...

24. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7... ^-2525. Dehghan M, Martinez S, Zhang X, Seron P, Lanas F, Islam S, et al. Relative validity of an FFQ to estimate daily food and nutrient intakes for Chilean adults. Public Health Nutr 2013;16(10):1782-1788. http://doi.org/brf8
http://doi.org/brf8... ^,3030. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative food frequency questionnaire for Argentinean adults. PLoS One 2012;7(5):e37958. http://doi.org/brf9
http://doi.org/brf9... In our study, the average PCC, between SFFQ and two 24DRs, for absolute nutrient intake was 0.365 for adults and 0.374 for adolescents.

For both adults and adolescents, corrections by energy adjustment increased the observed CC for some nutrients, but decreased the CC for the majority. This may indicate, according to Willet and Stampfer,¹⁹19. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174. http://doi.org/dtzfj3
http://doi.org/dtzfj3... that energy adjustment increases CC when the variability of nutrient consumption is related to energy intake, but decreases CC when the variability depends on systematic errors of overestimation and underestimation. In our study, the lower correlations' values found in some nutrients may indicate that the SFFQ, to some extent, systematically over/underestimated intake of those nutrients. However, error in over/underestimation is expected in the SFFQ. Similarly, Liu and colleagues,²⁴24. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7... Dehghan and colleagues,³⁰30. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative food frequency questionnaire for Argentinean adults. PLoS One 2012;7(5):e37958. http://doi.org/brf9
http://doi.org/brf9... Deschamps and colleagues,²⁶26. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009;63(2):282-391. http://doi.org/d2m7dq
http://doi.org/d2m7dq... Slater and colleagues,³¹31. Slater B, Philippi ST, Fisberg RM, Latorre MR. Validation of a semiquantitative adolescent food frequency questionnaire applied at a public school in Sao Paulo, Brazil. Eur J of Clin Nutr 2003;57(5):629-635. http://doi.org/bzgpjb
http://doi.org/bzgpjb... and Xia and colleagues²⁸28. Xia W, Sun C, Zhang L, Zhang X, Wang J, Wang H, et al. Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS One 2011;6(5):e19656. http://doi.org/fbsgj8
http://doi.org/fbsgj8... found that energy adjustment did not improve the crude correlations in their studies.

Regardless of the differences between 24DR and SFFQ in the present study, due to the correction for day-to-day variation intakes, deattenuated correlations coefficients were usually higher than crude correlations. Our results, in both adults and adolescents, showed that for more than 60% of nutrient, deattenuation increased the correlations and they were ≥ 0.40, which is considered as valid. Our results are lower than some reported by previous validation studies,¹¹11. Jain MG, Rohan TE, Soskolne CL, Kreiger N. Calibration of the dietary questionnaire for the Canadian study of diet, lifestyle and health cohort. Public Health Nutr 2003;6(1):79-86. http://doi.org/bs6hn9
http://doi.org/bs6hn9... ^,3232. Kroke A, Klipstein-Grobusch K,Voss S, Moseneder J,Thielecke F, Noack R, et al. Validation of a self-administered food-frequency questionnaire administered in the European Prospective Investigation into Cancer and Nutrition (EPIC) Study: comparison of energy, protein, and macronutrient intakes estimated with the doubly labeled water, urinary nitrogen, and repeated 24-h dietary recall methods. Am J Clin Nutr 1999;70(4):439-447. but comparable to others.²⁴24. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7... ^,3030. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative food frequency questionnaire for Argentinean adults. PLoS One 2012;7(5):e37958. http://doi.org/brf9
http://doi.org/brf9... ^,3131. Slater B, Philippi ST, Fisberg RM, Latorre MR. Validation of a semiquantitative adolescent food frequency questionnaire applied at a public school in Sao Paulo, Brazil. Eur J of Clin Nutr 2003;57(5):629-635. http://doi.org/bzgpjb
http://doi.org/bzgpjb... ^,3333. Thompson FE, Kipnis V, Midthune D, Freedman LS, Carroll RJ, Subar AF, et al. Performance of a food-frequency questionnaire in the US NIH-AARP (national institutes of health-American association of retired persons) diet and health study. Public Health Nutr 2008;11(2):183-195. http://doi.org/b3dzdf
http://doi.org/b3dzdf... For example, with respect to energy, lower coefficients have been informed in the Willet FFQ (approximately 0.17 when both genders were combined), and the Block FFQ (0.39 when both genders were combined)³⁴34. Subar AF, Thompson FE, Kipnis V, Midthune D, Hurwitz P, McNutt S, et al. Comparative validation of the Block, Willett, and National Cancer Institute food frequency questionnaires: the Eating at America's Table Study. Am J Epidemiol 2001;154(12):1089-1099. http://doi.org/bx7db4
http://doi.org/bx7db4... as compared with 0.53 (both genders) derived from adult population in our study. In terms of macronutrients, it was particularly evident that our correlations for protein were lower than the coefficients observed in other macronutrients (0.33 for adults and 0.34 for adolescents). However, our findings were similar to those obtained in previous validation studies⁷7. Hernandez-Avila M, Romieu I, Parra S, Hernandez-Avila J, Madrigal H, Willett WC. Validity and reproducibility of a food frequency questionnaire to assess dietary intake of women living in Mexico city. Salud Publica Mex 1998;40(2):133-140. http://doi.org/cqwzwz
http://doi.org/cqwzwz... ^,2626. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009;63(2):282-391. http://doi.org/d2m7dq
http://doi.org/d2m7dq... conducted in Mexico in adult women (0.29 to 0.32) and other study carried out on French adolescents (0.33). Additionally, we observed low correlations for polyunsaturated fat (0.36 for adults and 0.21 for adolescents), which are similar to the coefficients observed in other studies.²²22. Jaceldo-Siegl K, Knutsen SF, Sabate J, Beeson WL, Chan J, Herring RP, et al. Validation of nutrient intake using an FFQ and repeated 24 h recalls in black and white subjects of the Adventist Health Study-2 (AHS-2). Public Health Nutr 2010;13(6):812-819. http://doi.org/dj6fqz
http://doi.org/dj6fqz...

23. Zhuang M, Yuan Z, Lin L, Hu B, Wang X, Yang Y, et al. Reproducibility and relative validity of a food frequency questionnaire developed for adults in Taizhou, China. PLoS One 2012;7(11):e48341. http://doi.org/f25ftj
http://doi.org/f25ftj...

24. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7...

25. Dehghan M, Martinez S, Zhang X, Seron P, Lanas F, Islam S, et al. Relative validity of an FFQ to estimate daily food and nutrient intakes for Chilean adults. Public Health Nutr 2013;16(10):1782-1788. http://doi.org/brf8
http://doi.org/brf8...

26. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009;63(2):282-391. http://doi.org/d2m7dq
http://doi.org/d2m7dq...

27. Araujo MC, Yokoo EM, Pereira RA. Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc 2010;110(8):1170-1177. http://doi.org/dz7jg4
http://doi.org/dz7jg4... ^-2828. Xia W, Sun C, Zhang L, Zhang X, Wang J, Wang H, et al. Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS One 2011;6(5):e19656. http://doi.org/fbsgj8
http://doi.org/fbsgj8... This result could be a consequence of the SFFQ development, which mainly focused on food items that substantially contributed to energy intake. In this respect, the assessment of nutrients that contribute little to the energy intake such as polyunsaturated fat, could be difficult. Certainly, polyunsaturated fat consumption comes mainly from intake of fatty fish, nuts, and some oils; in our SFFQ questions on fish intake were based on preparation (canned fish in oil, water or tomato; dried or salted fish; fresh fish) which influences energy intake rather than on the kind of fish, which would be important for a more accurate estimation of polyunsaturated fat intake. Despite our deattenuated CC for some micronutrients, specially zinc (0.31 and 0.19) and folate (0.30 and 0.30) in both adults and adolescents, respectively, were low, they were significant with p values <0.05, suggesting reasonable agreement between the two dietary methods. Other studies have also reported poor correlation for micronutrients.¹¹11. Jain MG, Rohan TE, Soskolne CL, Kreiger N. Calibration of the dietary questionnaire for the Canadian study of diet, lifestyle and health cohort. Public Health Nutr 2003;6(1):79-86. http://doi.org/bs6hn9
http://doi.org/bs6hn9... ^,2424. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7... ^,2626. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009;63(2):282-391. http://doi.org/d2m7dq
http://doi.org/d2m7dq...

The CC is usually accepted for measuring the strength of association between new methods against the comparison of the "gold standard" method. However, the use of correlation analysis for assessing validity has frequently been questioned on the basis that it does not measure agreement between the two methods. Generally, FFQ are used for ranking individuals according to food or nutrient intake rather than for estimating absolute levels of intake. In this respect, we used cross-classification into quartiles of intake and weighted kappa to assess the agreement and showed that the two dietary instruments were comparable. Despite some differences in the estimation of nutrients, for the majority of nutrients, approximately more than 60% of adults and adolescents were classified in the same or an adjacent category, which is consistent with other studies.²³23. Zhuang M, Yuan Z, Lin L, Hu B, Wang X, Yang Y, et al. Reproducibility and relative validity of a food frequency questionnaire developed for adults in Taizhou, China. PLoS One 2012;7(11):e48341. http://doi.org/f25ftj
http://doi.org/f25ftj...

24. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7...

25. Dehghan M, Martinez S, Zhang X, Seron P, Lanas F, Islam S, et al. Relative validity of an FFQ to estimate daily food and nutrient intakes for Chilean adults. Public Health Nutr 2013;16(10):1782-1788. http://doi.org/brf8
http://doi.org/brf8...

26. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys JM, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009;63(2):282-391. http://doi.org/d2m7dq
http://doi.org/d2m7dq...

27. Araujo MC, Yokoo EM, Pereira RA. Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc 2010;110(8):1170-1177. http://doi.org/dz7jg4
http://doi.org/dz7jg4...

28. Xia W, Sun C, Zhang L, Zhang X, Wang J, Wang H, et al. Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS One 2011;6(5):e19656. http://doi.org/fbsgj8
http://doi.org/fbsgj8...

29. Kristal AR, Vizenor NC, Patterson RE, Neuhouser ML, Shattuck AL, McLerran D. Precision and bias of food frequency-based measures of fruit and vegetable intakes. Cancer Epidemiol Biomarkers Prev 2000;9(9):939-944.

30. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative food frequency questionnaire for Argentinean adults. PLoS One 2012;7(5):e37958. http://doi.org/brf9
http://doi.org/brf9... ^-3131. Slater B, Philippi ST, Fisberg RM, Latorre MR. Validation of a semiquantitative adolescent food frequency questionnaire applied at a public school in Sao Paulo, Brazil. Eur J of Clin Nutr 2003;57(5):629-635. http://doi.org/bzgpjb
http://doi.org/bzgpjb... This result demonstrated reasonable good agreement between methods and may reflect good sensitivity for our SFFQ. Finally, an acceptable agreement (kappa 0.20 to 0.45) was obtained for energy and most nutrients for both adolescents and adults.

In the present study there are some limitations that must be considered. Biomarkers have often been suggested as the "gold standard" for energy and some nutrients, but due to the viability and financial limitations, like most validation studies,⁴4. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilization of food frequency questionnaires a review. Public Health Nutr 2002;5(4):567-587. http://doi.org/dnffmv
http://doi.org/dnffmv... we selected the 24-hour dietary recall as the reference method. Despite other dietary methods, like diet records, have been suggested as a reference method, in a review on the validations of FFQs, Cade and collegues⁴4. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilization of food frequency questionnaires a review. Public Health Nutr 2002;5(4):567-587. http://doi.org/dnffmv
http://doi.org/dnffmv... found that 75% of the studies validated FFQs against repeated 24DRs. Even though we extensively trained interviewers, we recognize that both 24DR and SFFQ have some error sources, such as the reliance on memory and the perception of portion size.¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141. However, the SFFQ stresses usually long-term memory, whereas 24DR relies on short-term memory. Additionally, the 24DR instrument was interviewer-based using open-ended questions, while the SFFQ was administered with close-ended questions. Such differences let us assume that errors between methods are sufficiently independent and that the 24DR is a satisfactory method for this target instrument.²⁴24. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J 2013;12:49. http://doi.org/brf7
http://doi.org/brf7... ^,3535. Haftenberger M, Heuer T, Heidemann C, Kube F, Krems C, Mensink GB. Relative validation of a food frequency questionnaire for National Health and Nutrition Monitoring. Nutr J 2010;9:36. http://doi.org/dmzppt
http://doi.org/dmzppt... To estimate long-term usual intake, collection of three-to-four days of recalls in each season of the year has been considered as ideal.¹1. Willett WC. Food frequency methods. In: Willett WC, edit. Nutritional Epidemiology, 3rd edition. Oxford: Oxford University Press, 2013:96-141. In our study, two 24-hour dietary recalls were used as comparison method; therefore, the estimate of intake may have been closer to true intake if we had collected more days of food consumption. Regarding micronutrients, it has been proposed that the number of days which must be examined to allow a true estimation of average daily intake is greater for micronutrients than for macronutrients and surpasses the two days evaluated in this study.³⁶36. Boyle AM. Community nutrition in action: an entrepreneurial approach, 6th edition. Belmont, OH: Cengage Learning, 2012. However, other studies enhanced the possibility of using only a small number of replicate measures for the reference method combined with a statistical adjustment to remove the effects of within-person variation.²⁸28. Xia W, Sun C, Zhang L, Zhang X, Wang J, Wang H, et al. Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS One 2011;6(5):e19656. http://doi.org/fbsgj8
http://doi.org/fbsgj8... In this respect, some authors have suggested that the greatest statistical efficiency is obtained with only two, and at the most five, duplicates per subject.¹⁰10. Carroll RJ, Pee D, Freedman LS, Brown CC. Statistical design of calibration studies. Am J Clin Nutr 1997;65(4 Suppl):1187S-1189S.^,2020. Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L, et al. Statistical approaches for assessing the relative validity of a food frequency questionnaire: use of correlation coefficients and the kappa statistic. Public Health Nutr 2003;6(3):313-321. http://doi.org/b38ssz
http://doi.org/b38ssz... In this context, to validate our SFFQ against two 24DRs seems to be adequate. Another limitation of our study is that we did not administer the SFFQ twice, thus we cannot evaluate the reproducibility of the instrument. For this reason, future effort needs to be done to assess the reproducibility of this SFFQ.

There is no single perfect way of developing an SFFQ or measuring its validity. However, in general, the methods implemented in the present study, including sample size; selection of population; SFFQ structure, like number of food items; and statistical methods, were consistent with commonly accepted practices.

In conclusion, the SFFQ used in the Ensanut 2012 was found to be a suitable tool for the assessment of energy and dietary intake of nutrients by Mexican adolescents and adults, especially for ranking them according to their nutrient intake. The present SFFQ might be a useful instrument in large epidemiological studies with a focus on diet as a risk factor for various health outcomes. Finally, further efforts should be made to evaluate the reproducibility of the SFFQ.

References

Publication Dates

History