ABSTRACT

OBJECTIVE

To develop and assess the reliability of an instrument that enables auditing information on consumer food environment indicators, such as availability, price, promotional and advertising strategies, and quantity of brands available, using the food recommendations adopted by the Dietary Guidelines for the Brazilian Population as a theoretical basis.

METHODS

This is a methodological study in two phases: 1. development of the audit instrument and 2. assessment of its reliability and reproducibility . The Content Validity Index was estimated for each instrument item (>0.80 satisfactory). Inter-rater and test-retest reliability were assessed by percentage agreement and Kappa coefficients. Pearson’s correlation coefficient and Scatter-plots were used to measure the degree of linear correlation between two quantitative variables.

RESULTS

The Content Validity Index was 0.91. Inter-rater and test-retest reliability were mostly high (Kappa> 0.80), for food availability indicators. Among the items that measure advertising, Kappa values for inter-rater reliability ranged from 0.57 to 1.00 and for the test-retest ranged from 0.18 to 0.90. Prices and quantity of brands showed a positive linear correlation between measurements performed by researcher 1 and 2 and between visits 1 and 2.

CONCLUSIONS

AUDITNOVA is reliable for measuring aspects such as availability, price, quantity of brands, and advertising of foods available in the consumer food environment.

INTRODUCTION

The food environment, in its multiple dimensions¹1. Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330-3, ii. https://doi.org/10.4278/0890-1171-19.5.330
https://doi.org/10.4278/0890-1171-19.5.3... , influences food consumption and formation of eating habits²2. Glanz K, Johnson L, Yaroch AL, Phillips M, Ayala GX, Davis EL. Measures of retail food store environments and sales: review and implications for healthy eating initiatives. J Nutr Educ Behav. 2016;48(4):280-288.e1. https://doi.org/10.1016/j.jneb.2016.02.003
https://doi.org/10.1016/j.jneb.2016.02.0... , ³3. Rose D, Hutchinson PL, Bodor JN, Swalm CM, Farley TA, Cohen DA, et al. Neighborhood food environments and body mass index. Am J Prev Med. 2009;37(3):214-9. https://doi.org/10.1016/j.amepre.2009.04.024
https://doi.org/10.1016/j.amepre.2009.04... . Strong evidence relates it to the chronic noncommunicable disease (CNCD) epidemic, especially obesity, in developed⁴4. Egger G, Swinburn B. An “ecological” approach to the obesity pandemic. BMJ. 1997;315(7106):477-80. https://doi.org/10.1136/bmj.315.7106.477
https://doi.org/10.1136/bmj.315.7106.477... , ⁵5. Swinburn B, Egger G. Preventive strategies against weight gain and obesity. Obes Rev. 2002;3(4):289-301. https://doi.org/10.1046/j.1467-789X.2002.00082.x
https://doi.org/10.1046/j.1467-789X.2002... and developing countries⁶6. Jaime PC, Duran AC, Sarti FM, Lock K. Investigating environmental determinants of diet, physical activity, and overweight among adults in Sao Paulo, Brazil. J Urban Health. 2011;88(3):567-81. https://doi.org/10.1007/s11524-010-9537-2
https://doi.org/10.1007/s11524-010-9537-... . The food environment is also connected to increased body mass index³3. Rose D, Hutchinson PL, Bodor JN, Swalm CM, Farley TA, Cohen DA, et al. Neighborhood food environments and body mass index. Am J Prev Med. 2009;37(3):214-9. https://doi.org/10.1016/j.amepre.2009.04.024
https://doi.org/10.1016/j.amepre.2009.04... and to two important dimensions that ensure food and nutritional security, healthy food access, and availability⁷7. Ministério da Saúde (BR), Secretaria de Vigilância em Saúde, Departamento de Análise de Situação de Saúde. Plano de Ações Estratégicas para o Enfrentamento das Doenças Crônicas Não Transmissíveis (DCNT) no Brasil 2011-2022. Brasília, DF; 2011 [cited 2019 Mar 22]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/plano_acoes_enfrent_dcnt_2011.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... . Using a socioecological behavior approach, Glanz et al.¹1. Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330-3, ii. https://doi.org/10.4278/0890-1171-19.5.330
https://doi.org/10.4278/0890-1171-19.5.3... suggest a conceptual model that divides the food environment into four main domains: community food environment, organizational food environment, information food environment, and consumer food environment. The latter refers to what consumers find inside and around retail food establishments, for example, healthy food availability, variety, price, promotions, shelf position, nutritional information, and advertising, determining factors in decision-making processes of food acquisition and consumption by the population⁸8. Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330-3, ii. https://doi.org/10.4278/0890-1171-19.5.330
https://doi.org/10.4278/0890-1171-19.5.3... , ⁹9. Almeida LB, Scagliusi FB, Duran AC, Jaime PC. Barriers to and facilitators of ultra-processed food consumption: perceptions of Brazilian adults. Public Health Nutr. 2018;21(1):68-76. https://doi.org/10.1017/S1368980017001665
https://doi.org/10.1017/S136898001700166... . For example, a study in the United States found only 7% of the total food information in the weekly circulated leaflet from a supermarket chain was about fruit, 10% about vegetables, 10% about milk and dairy products, and 18% about cereals and grains; moreover, it showed information in these leaflets often influence consumers when making their food purchases¹⁰10. Ravensbergen EAH, Waterlander WE, Kroeze W, Steenhuis IHM. Healthy or unhealthy on sale? A cross-sectional study on the proportion of healthy and unhealthy foods promoted through flyer advertising by supermarkets in the Netherlands. BMC Public Health. 2015;15:470. https://doi.org/10.1186/s12889-015-1748-8
https://doi.org/10.1186/s12889-015-1748-... .

The Dietary Guidelines for the Brazilian Population (DGBP) also recognizes the role of the food environment in promoting healthy eating and indicates six obstacles that hamper the adherence to current nutritional recommendations: information, advertising, time, cooking skills, cost, and food availability. Four of them are directly related to the food environment (information, advertising, cost, and availability), according to the model by Glanz et al.¹1. Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330-3, ii. https://doi.org/10.4278/0890-1171-19.5.330
https://doi.org/10.4278/0890-1171-19.5.3... . Food environment aspects contributing to the health and quality of life of populations have been widely discussed in many countries and motivated the creation of indicator monitoring networks, such as the International Network for Food and Obesity/CNCD Research, Monitoring and Action Support (INFORMAS), a global network of public-interest organizations and researchers from 30 countries that aims to monitor, benchmark and support public and private sector actions to increase healthy food environments and reduce obesity, CNCD and health-related inequalities¹¹11. INFORMAS. Country Matrix updated March 2019. Aukland (NZL): University of Aukland; 2019 [cited 2019 Mar 22]. Available from: https://www.informas.org/country-matrix-2/
https://www.informas.org/country-matrix-... . INFORMAS proposes food environment monitoring indicators such as food nutritional composition, labeling, food advertising, supply chain, quantity and types of retail food establishments, prices, and investments in the food production chain¹¹11. INFORMAS. Country Matrix updated March 2019. Aukland (NZL): University of Aukland; 2019 [cited 2019 Mar 22]. Available from: https://www.informas.org/country-matrix-2/
https://www.informas.org/country-matrix-... .

As food environment contributes to obesity and CNCD, as shown in Brazilian technical documents⁷7. Ministério da Saúde (BR), Secretaria de Vigilância em Saúde, Departamento de Análise de Situação de Saúde. Plano de Ações Estratégicas para o Enfrentamento das Doenças Crônicas Não Transmissíveis (DCNT) no Brasil 2011-2022. Brasília, DF; 2011 [cited 2019 Mar 22]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/plano_acoes_enfrent_dcnt_2011.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... , ¹²12. Ministério da Saúde (BR), Secretaria de Atenção à Saúde, Departamento de Atenção Básica. Guia alimentar para a população brasileira. 2 ed. Brasília, DF; 2014 [cited 2019 Mar 22]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/guia_alimentar_populacao_brasileira_2ed.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... , ¹³13. Ministério da Saúde. Política Nacional de Alimentação e Nutrição (PNAN). Brasília, DF; 2013 [cited 2019 Mar 22]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/politica_nacional_alimentacao_nutricao.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... and international networks¹¹11. INFORMAS. Country Matrix updated March 2019. Aukland (NZL): University of Aukland; 2019 [cited 2019 Mar 22]. Available from: https://www.informas.org/country-matrix-2/
https://www.informas.org/country-matrix-... , valid and reliable measures are necessary to study food environment¹⁴14. Glanz K, Sallis JF, Saelens BE, Frank LD. Nutrition Environment Measures Survey in stores (NEMS-S): development and evaluation. Am J Prev Med. 2007;32(4):282-9. https://doi.org/10.1016/j.amepre.2006.12.019
https://doi.org/10.1016/j.amepre.2006.12... . In the Brazilian scenario, research on this topic is recent⁶6. Jaime PC, Duran AC, Sarti FM, Lock K. Investigating environmental determinants of diet, physical activity, and overweight among adults in Sao Paulo, Brazil. J Urban Health. 2011;88(3):567-81. https://doi.org/10.1007/s11524-010-9537-2
https://doi.org/10.1007/s11524-010-9537-... , ¹⁵15. Martins PA, Cremm EC, Leite FHM, Maron LR, Scagliusi FB, Oliveira MA. Validation of an adapted version of the nutrition environment measurement tool for stores (NEMS-S) in an urban area of Brazil. J Nutr Educ Behav. 2013;45(6):785-92. https://doi.org/10.1016/j.jneb.2013.02.010
https://doi.org/10.1016/j.jneb.2013.02.0... , ¹⁶16. Duran AC, Lock K, Latorre MRDO, Jaime PC. Evaluating the use of in-store measures in retail food stores and restaurants in Brazil. Rev Saude Publica. 2015;49:80. https://doi.org/10.1590/S0034-8910.2015049005420
https://doi.org/10.1590/S0034-8910.20150... , and studies need to expand production and collection of indicators, especially on the consumer’s food environment, which influences the behavior of food purchase and consumption¹1. Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330-3, ii. https://doi.org/10.4278/0890-1171-19.5.330
https://doi.org/10.4278/0890-1171-19.5.3... .

None of the audit instruments already validated for the Brazilian context¹⁷17. Monteiro CA, Cannon G, Moubarac JC, Levy RB, Louzada MLC, Jaime PC. The UN Decade of Nutrition, the NOVA food classification and the trouble with ultra-processing. Public Health Nutr. 2018;21(1):5-17. https://doi.org/10.1017/S1368980017000234
https://doi.org/10.1017/S136898001700023... , ¹⁸18. Monteiro C, Cannon G, Levy R, Moubarac J, Jaime P, Martins A, et al. NOVA. The star shines bright. Food classification. Public Health. World Nutr. 2016;7(1-3):28-38. assess food availability and advertising according to the DGBP recommendations. Martins et al.¹⁵15. Martins PA, Cremm EC, Leite FHM, Maron LR, Scagliusi FB, Oliveira MA. Validation of an adapted version of the nutrition environment measurement tool for stores (NEMS-S) in an urban area of Brazil. J Nutr Educ Behav. 2013;45(6):785-92. https://doi.org/10.1016/j.jneb.2013.02.010
https://doi.org/10.1016/j.jneb.2013.02.0... , validating the Brazilian food environment audit instrument based on the adaptation of the study by Glanz et al.¹⁴14. Glanz K, Sallis JF, Saelens BE, Frank LD. Nutrition Environment Measures Survey in stores (NEMS-S): development and evaluation. Am J Prev Med. 2007;32(4):282-9. https://doi.org/10.1016/j.amepre.2006.12.019
https://doi.org/10.1016/j.amepre.2006.12... , approximate the food groups recommended by the DGBP, but explore only three indicators (availability, price, and quality) and disregard the latest version of the NOVA food classification, which divides them into four groups: in natura /minimally processed, culinary ingredients, processed foods, and ultra-processed foods¹⁷17. Monteiro CA, Cannon G, Moubarac JC, Levy RB, Louzada MLC, Jaime PC. The UN Decade of Nutrition, the NOVA food classification and the trouble with ultra-processing. Public Health Nutr. 2018;21(1):5-17. https://doi.org/10.1017/S1368980017000234
https://doi.org/10.1017/S136898001700023... . Brazilian instruments lack indicators that dialogue with national recommendations. Regarding this, identifying the density of commercial establishments that primarily sell ultra-processed foods, the food advertisements according to food type in line with NOVA, the sales of ultra-processed foods in places such as checkout aisles or the sales of in natura products at the establishment entrance and the promotional prices of healthy and unhealthy foods may contribute to the understanding of the relation between food consumption and environment in the face of the new Brazilian nutritional and epidemiological scenario¹²12. Ministério da Saúde (BR), Secretaria de Atenção à Saúde, Departamento de Atenção Básica. Guia alimentar para a população brasileira. 2 ed. Brasília, DF; 2014 [cited 2019 Mar 22]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/guia_alimentar_populacao_brasileira_2ed.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... .

The low reliability of the methods and instruments that propose to analyze the food environment and the current lack of objective criteria in the generation of indicators are barriers to understanding the mechanism of association between food environment, obesity, and food consumption²⁰20. Ohri-Vachaspati P, Leviton LC. Measuring food environments: a guide to available instruments. Am J Health Promot. 2010;24(6):410-26. https://doi.org/10.4278/ajhp.080909-LIT-190
https://doi.org/10.4278/ajhp.080909-LIT-... . Thus, this study aims to develop and evaluate the reliability of a food environment audit instrument that enables capturing information of consumer food environment indicators such as availability, price, promotional and advertising strategies, and quantity of brands available, using the dietary recommendations adopted by the DGBP as theoretical basis.

METHODS

This is a methodological study in two phases: 1. development of the audit instrument and 2. assessment of its reliability and reproducibility . The Ethics Committee of Faculdade de Saúde Pública of Universidade de São Paulo approved this study under CAAE number 69045917.5.0000.5421. All commercial establishments were aware of the informed consent form and voluntarily participated in the study.

Audit Instrument Development

We designed the NOVA-based food environment audit instrument named AUDITNOVA in stages that involved systematic meetings of the research group; detailed analysis of Brazilian and international audit instruments and detailed analysis of the NOVA food classification proposed by Monteiro et al.¹⁷17. Monteiro CA, Cannon G, Moubarac JC, Levy RB, Louzada MLC, Jaime PC. The UN Decade of Nutrition, the NOVA food classification and the trouble with ultra-processing. Public Health Nutr. 2018;21(1):5-17. https://doi.org/10.1017/S1368980017000234
https://doi.org/10.1017/S136898001700023... . The processing of food identified by NOVA involves physical, biological and chemical processes applied after food is separated from nature and before it is eaten or prepared as dishes and meals. NOVA classifies all foods into four major groups:

• group 1: in natura or minimally processed foods. In natura foods are defined as edible parts of plants or animals after leaving nature and minimally processed foods are in natura foods altered by processes that include removal of inedible or unwanted parts and drying, crushing, grinding, fractionating, filtering, roasting, boiling, non-alcoholic fermentation, pasteurization, refrigeration, freezing, and vacuum packaging;

• group 2: culinary ingredients. Substances derived from group 1 or from nature by processes that include pressing, refining, grinding, and drying to make durable products used for cooking and preparing food at home or in restaurants;

• group 3: processed foods. They are essentially made by adding salt, oil, sugar, or other substances from group 2 to group 1;

• group 4: ultra-processed foods. Formulations made mainly or entirely from food-derived substances and additives, with little or no food from group 1¹⁷17. Monteiro CA, Cannon G, Moubarac JC, Levy RB, Louzada MLC, Jaime PC. The UN Decade of Nutrition, the NOVA food classification and the trouble with ultra-processing. Public Health Nutr. 2018;21(1):5-17. https://doi.org/10.1017/S1368980017000234
  https://doi.org/10.1017/S136898001700023... .

We selected 66 foods for the AUDITNOVA with the highest frequency of acquisition by the Brazilian population according to data from the 2008-2009 Family Budget Survey (FBS)²¹21. Instituto Brasileiro de Geografia e Estatística. Pesquisa de Orçamentos Familiares, 2008-2009: análise do consumo alimentar pessoal no Brasil. Rio de Janeiro: IBGE; 2011. . The first version of the instrument had two blocks, one on food availability, prices, variety, and quality and another on advertising.

The first version of the instrument underwent the content validation process by a panel of judges with the participation of nine experts distributed among the following areas: food research, food advertising, and consumer protection. DGBP was the theoretical framework adopted during the panel of judges, especially Chapter 2 (“Choosing Food”) and Chapter 5 (“Understanding and Overcoming Obstacles”). The judges reviewed each AUDITNOVA item and assigned scores based on the 4-point Likert scale (1: disagree, 2: major revisions are necessary, 3: minor revisions are necessary, and 4: agree) for the clarity, relevance, pertinence, and representativeness attributes. In addition, the judges had a field for writing suggestions if the score was 3 or less. We estimated the content validity index (clear, relevant, pertinent, and representative) for each item and for each instrument block and considered it satisfactory when it reached 0.80 agreement among the judges. We discarded items below this percentage in the final version of the instrument²²22. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003;27(2):94-104. https://doi.org/10.1093/swr/27.2.94
https://doi.org/10.1093/swr/27.2.94... .

The final version of AUDITNOVA, after analysis by the panel of judges, contains 14 blocks of questions divided into: block 1 — general information (municipality identification; evaluator identification; commercial establishment identification (sequential number assigned by the researcher); business name and address, date, collection start time and end time); block 2 — establishment type and products sold (type of commercial establishment, food groups sold according to NOVA, food groups sold primarily according to NOVA); block 3 — establishment entrance (fruit and vegetable section at the entrance of the store, food and advertisement availability in checkout aisles); block 4 — fruit and vegetable section (availability, unit of measure, current price in reais/kilo, and price type — normal or promotional); block 5 — meat, chicken, and fish section (availability, most expensive price in reais/kilo, cheapest price in reais/kilo, and price type); block 6 — dairy section (availability, quantity of brands available, most expensive price, cheapest price, and price type); block 7 — grocery section (availability, quantity of brands available, most expensive price, cheapest price, and price type); block 8 — canned food section (availability, quantity of brands available, most expensive price, cheapest price, and type of price); block 9 — bakery section (availability, quantity of brands available, most expensive price, cheapest price, and type of price); block 10 — frozen food section (availability, quantity of brands available, most expensive price, cheapest price, and price type); block 11 — beverage section (availability, quantity of brands available, most expensive price, cheapest price, and price type); block 12 — chocolate and snack section (availability, quantity of brands available, most expensive price, cheapest price, and price type); and finally, blocks 13 and 14 — advertisements inside and outside the establishment, respectively (Supplementary Archive).

Reproducibility and Reliability of the Audit Instrument

We performed the AUDITNOVA reliability assessment study on a convenience sample in the metropolitan region of the city of São Paulo (SP), easily accessible by public transportation from Faculdade de Saúde Pública. We designed the neighborhood selection to maximize the ability to contrast supermarkets in neighborhoods with different income levels. To guarantee socioeconomic differences between them, we chose sites with different human development indexes (HDI): Pinheiros (HDI = 0.98), Higienópolis (HDI = 0.93), Belém (HDI = 0.91), and Sacomã (HDI) = 0.84). In each neighborhood, we selected 20 commercial establishments, among supermarkets, hypermarkets, and markets, totaling a sample of 80 establishments. Only seven refused to participate in the survey. Supermarkets, hypermarkets and markets have various food products available to consumers²³23. Durán AC, Almeida SL, Latorre MRDO, Jaime PC. The role of the local retail food environment in fruit, vegetable and sugar-sweetened beverage consumption in Brazil. Public Health Nutr. 2016;19(6):1093-102. https://10.1017/S1368980015001524
https://10.1017/S1368980015001524... , ²⁴24. Costa JC, Claro RM, Martins APB, Levy RB. Food purchasing sites. Repercussions for healthy eating. Appetite. 2013;70:99-103. https://doi.org/10.1016/j.appet.2013.06.094
https://doi.org/10.1016/j.appet.2013.06.... and are important equipment for measuring the reliability and reproducibility of a food environment audit instrument, as they enable the researcher to apply the instrument integrally.

To assess inter-rater reliability, two trained researchers independently visited the 73 commercial establishments in the region chosen for the study. Inter-rater reliability is used to assess the consistency of a measurement by different evaluators. To assess test-retest reliability, the same researchers revisited 41 sites 32 days after the initial observations. Test-retest reliability is used to assess the consistency of a measurement between two distinct moments.

For categorical variables, percentage agreement and Kappa coefficients assessed inter-rater and test-retest reliability. We quantified Kappa values using the following scale: 0.01 to 0.20, slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; 0.81 to 0.99, high agreement²⁵25. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74. . We interpreted negative agreement values as a sign that evaluators agreed less on one item than expected due to chance — for example, a systematic disagreement among observers because diverse food items were available in the supermarkets. To assess the reliability of food availability according to NOVA, we grouped the 66 food items in the instrument into: group 1 — sum of all in natura /minimally processed foods; group 2 — sum of all culinary ingredients; group 3 — sum of all processed foods, and group 4 — sum of all ultra-processed foods.

For quantitative variables such as price and quantity of brands available, we first performed an exploratory analysis using scatter plots illustrating the linear fit and the quadratic fit. Scatter plots allow identifying patterns in data distribution and possible systematic and random errors depending on how dots are distributed along the line²⁶26. Rousson V, Gasser T, Seifert B. Assessing intrarater, interrater and test–retest reliability of continuous measurements. Stat Med. 2002;21(22):3431-46. https://doi.org/10.1002/sim.1253
https://doi.org/10.1002/sim.1253... . Subsequently, we estimated Pearson’s correlation coefficient (r), which measures the degree of linear correlation between two quantitative variables. It is a dimensionless index with values between -1.0 and 1.0, which reflects the intensity of a linear relationship between two data sets. We estimated Pearson’s correlation coefficient between the pairs of variables collected by researcher 1 and researcher 2 and between the variables collected on the first and second visits. We estimated price and brand averages for each of the four food groups analyzed. We performed statistics on the statistical package Stata 14.

RESULTS

At the content validation stage conducted by nine judges, the experts provided the necessary information to review the audit instrument and improve its content. The content validity index (CVI), which represents the average score for the clarity, relevance, pertinence, and representativeness attributes, was 0.91 for the entire instrument. Although the CVI was greater than 0.80 for most items, the suggestions provided by the experts were incorporated into the instrument because they are totally suitable according to the researcher’s assessment ( Table 1 ).

During the audit process, both trained researchers’ first visits to the 73 establishments occurred in an average of 41 days (standard deviation = 11.8 days). The average application time of AUDITNOVA was 90 minutes (standard deviation = 7.0 minutes). The researchers’ second visit occurred between 32 and 47 days after the first collection, with an average of 39.5 days (standard deviation = 4.8 days).

Table 2 shows the inter-rater and test-retest reliability results for the 66 audited foods and for the four NOVA food groups. Most foods had Kappa values higher than 0.80 (substantial agreement) for both inter-rater and test-retest. Analyzing the NOVA groups, we found in natura /minimally processed foods showed moderate Kappa values (0.41–0.60) for both inter-rater and test-retest. The other three food groups (culinary ingredients, processed foods, and ultra-processed foods) had Kappa values above 0.70 for inter-rater reliability, ranging from 0.57 to 0.64 for test-retest reliability, which indicates moderate agreement between visits.

Table 3 shows the inter-rater and test-retest reliability results for the sold and primarily sold foods according to NOVA indicators, fruit and vegetable section at the entrance of the establishment and price type (normal or promotional) for foods grouped according to NOVA. Despite the high agreements observed for all indicators, Kappa values in both inter-rater and test-retest reliability were reasonable (0.21–0.40) and moderate (0.41–0.60), showing the low capacity of agreement of some, such as priority sale of processed foods and price type of ultra-processed foods.

Table 4 shows the inter-rater and test-retest reliability results for the advertising variables in AUDITNOVA. The inter-rater reliability obtained a higher number of Kappa coefficients above 0.80 than the test-retest. For inter-rater reliability, Kappa values ranged from 0.57 to 1.00, and for the test-retest, from 0.18 to 0.90 — the highest disagreement.

Figure 1 shows the scatter plots illustrating the relationship between food price variables according to NOVA collected by researchers 1 and 2 and collected on the first and second visit. The inclinations of the lines show a positive correlation in all cases.

Figure 2 shows the scatter plots illustrating the relation between the variables quantity of food brands according to NOVA, collected by researchers 1 and 2 and collected at the first and second visit. The inclinations of the lines show a positive correlation in all cases.

Table 5 shows the average values of price and quantity of brands found by researchers 1 and 2 at each visit, as well as the Pearson correlation (r) values of the pairs of variables analyzed. Although all correlation values were positive, the quantity of minimally processed food brands had the lowest r values both among evaluators and between visits.

DISCUSSION

The food environment audit instrument developed in this study, AUDITNOVA, had high inter-rater and test-retest reliability, which ensures that it is a reliable instrument for studies aimed at working with food environment indicators based on the NOVA food classification proposed by Monteiro et al.¹⁷17. Monteiro CA, Cannon G, Moubarac JC, Levy RB, Louzada MLC, Jaime PC. The UN Decade of Nutrition, the NOVA food classification and the trouble with ultra-processing. Public Health Nutr. 2018;21(1):5-17. https://doi.org/10.1017/S1368980017000234
https://doi.org/10.1017/S136898001700023... . We carefully selected the indicator foods of the four groups proposed in NOVA because Brazilians frequently purchase them, according to national surveys, and DGBP recommends them. These foods included in AUDITNOVA may assess retail establishments regarding their availability of healthy and unhealthy foods according to Brazilian guidelines¹²12. Ministério da Saúde (BR), Secretaria de Atenção à Saúde, Departamento de Atenção Básica. Guia alimentar para a população brasileira. 2 ed. Brasília, DF; 2014 [cited 2019 Mar 22]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/guia_alimentar_populacao_brasileira_2ed.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... . In addition, information about price, quantity of brands and advertising will enable assessing the consumer’s food environment in detail, observing the barriers and conveniences that consumers face when choosing their food¹1. Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330-3, ii. https://doi.org/10.4278/0890-1171-19.5.330
https://doi.org/10.4278/0890-1171-19.5.3... . Most of the indicators in this instrument are appropriate for the planning of policy programs aimed at modifying the environment, assessing intervention needs and population needs when faced with food availability, and serving as evaluation, surveillance and advocacy indicators for other actions based on the consumer’s food environment²⁰20. Ohri-Vachaspati P, Leviton LC. Measuring food environments: a guide to available instruments. Am J Health Promot. 2010;24(6):410-26. https://doi.org/10.4278/ajhp.080909-LIT-190
https://doi.org/10.4278/ajhp.080909-LIT-... .

High inter-rater reliability shows the definitions and instructions in the measurement manual and training methods were sufficient to prepare observers to collect high-quality data. The high test-retest reliability in most of the indicators suggests only minor changes in food availability, price, quantity of brands, and advertising strategies occur over the data collection period. Thus, the measures collected with AUDITNOVA generated a stable estimate of the consumer’s food environment. However, the availability and price of in natura /minimally processed foods often change over the seasons; therefore, whenever the instrument is reapplied, repeated observations should be considered to assess or control seasonal effects¹⁴14. Glanz K, Sallis JF, Saelens BE, Frank LD. Nutrition Environment Measures Survey in stores (NEMS-S): development and evaluation. Am J Prev Med. 2007;32(4):282-9. https://doi.org/10.1016/j.amepre.2006.12.019
https://doi.org/10.1016/j.amepre.2006.12... .

In Brazil, studies on food environment are recent, Martins et al.¹⁵15. Martins PA, Cremm EC, Leite FHM, Maron LR, Scagliusi FB, Oliveira MA. Validation of an adapted version of the nutrition environment measurement tool for stores (NEMS-S) in an urban area of Brazil. J Nutr Educ Behav. 2013;45(6):785-92. https://doi.org/10.1016/j.jneb.2013.02.010
https://doi.org/10.1016/j.jneb.2013.02.0... and Duran et al.¹⁶16. Duran AC, Lock K, Latorre MRDO, Jaime PC. Evaluating the use of in-store measures in retail food stores and restaurants in Brazil. Rev Saude Publica. 2015;49:80. https://doi.org/10.1590/S0034-8910.2015049005420
https://doi.org/10.1590/S0034-8910.20150... developed and validated pioneering instruments for auditing the community food environment and the consumer food environment. The audit instrument validated by Martins et al.¹⁵15. Martins PA, Cremm EC, Leite FHM, Maron LR, Scagliusi FB, Oliveira MA. Validation of an adapted version of the nutrition environment measurement tool for stores (NEMS-S) in an urban area of Brazil. J Nutr Educ Behav. 2013;45(6):785-92. https://doi.org/10.1016/j.jneb.2013.02.010
https://doi.org/10.1016/j.jneb.2013.02.0... is an adaptation of the one developed by Glanz et al.¹⁴14. Glanz K, Sallis JF, Saelens BE, Frank LD. Nutrition Environment Measures Survey in stores (NEMS-S): development and evaluation. Am J Prev Med. 2007;32(4):282-9. https://doi.org/10.1016/j.amepre.2006.12.019
https://doi.org/10.1016/j.amepre.2006.12... to measure the consumer food environment, specifically retail food establishments, and to assess aspects such as food availability, price, and quality with a food list guided by the food pyramid and the degree of processing. However, it disregards the full version of the NOVA classification. Duran et al.¹⁶16. Duran AC, Lock K, Latorre MRDO, Jaime PC. Evaluating the use of in-store measures in retail food stores and restaurants in Brazil. Rev Saude Publica. 2015;49:80. https://doi.org/10.1590/S0034-8910.2015049005420
https://doi.org/10.1590/S0034-8910.20150... proposed an audit instrument designed to audit retail food establishments and restaurants and to measure aspects such as availability, variety, quality, price, and advertising of healthy food indicators such as fruits and vegetables and of unhealthy food indicators such as ultra-processed foods. The main differences of the AUDITNOVA developed and validated in this study comparing with the other two Brazilian instruments were the full use of the NOVA classification in the food item selection, the expansion of advertising and promotional strategies by food groups, the availability of 66 food items (including culinary ingredients and processed foods), the inclusion of strategic aspects of the consumer food environment (such as checkout aisles, endcaps, and islands), and the collection of information on normal or promotional prices, determining factors in the food acquisition by the population¹1. Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330-3, ii. https://doi.org/10.4278/0890-1171-19.5.330
https://doi.org/10.4278/0890-1171-19.5.3... , ⁹9. Almeida LB, Scagliusi FB, Duran AC, Jaime PC. Barriers to and facilitators of ultra-processed food consumption: perceptions of Brazilian adults. Public Health Nutr. 2018;21(1):68-76. https://doi.org/10.1017/S1368980017001665
https://doi.org/10.1017/S136898001700166... , ¹⁴14. Glanz K, Sallis JF, Saelens BE, Frank LD. Nutrition Environment Measures Survey in stores (NEMS-S): development and evaluation. Am J Prev Med. 2007;32(4):282-9. https://doi.org/10.1016/j.amepre.2006.12.019
https://doi.org/10.1016/j.amepre.2006.12... , ²⁰20. Ohri-Vachaspati P, Leviton LC. Measuring food environments: a guide to available instruments. Am J Health Promot. 2010;24(6):410-26. https://doi.org/10.4278/ajhp.080909-LIT-190
https://doi.org/10.4278/ajhp.080909-LIT-... .

The main indicators proposed in this instrument showed substantial and high Kappa values. Kappa values were moderate for the indicator of availability of in natura /minimally processed foods, especially in the test-retest, but Kappa values were substantial when evaluating food items in isolation. However, the seasonality and the low variety of in natura/ minimally processed foods in supermarkets and markets compared with street markets, big retail markets and farmer’s markets may have influenced the indicator reliability²⁴24. Costa JC, Claro RM, Martins APB, Levy RB. Food purchasing sites. Repercussions for healthy eating. Appetite. 2013;70:99-103. https://doi.org/10.1016/j.appet.2013.06.094
https://doi.org/10.1016/j.appet.2013.06.... , ²⁷27. Duran AC, Almeida SL, Latorre MRDO, Jaime PC. The role of the local retail food environment in fruit, vegetable and sugar-sweetened beverage consumption in Brazil. Public Health Nutr. 2016;19(6):1093-102. https://doi.org/10.1017/S1368980015001524
https://doi.org/10.1017/S136898001500152... .

AUDITNOVA enables measuring the different food information sources available in the consumer food environment in detail, dividing the types of advertising according to the NOVA’s four food groups. The DGBP recognizes that the publicity and information available in the consumer food environment can become an obstacle for the population to reach the food recommendations¹²12. Ministério da Saúde (BR), Secretaria de Atenção à Saúde, Departamento de Atenção Básica. Guia alimentar para a população brasileira. 2 ed. Brasília, DF; 2014 [cited 2019 Mar 22]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/guia_alimentar_populacao_brasileira_2ed.pdf
http://bvsms.saude.gov.br/bvs/publicacoe... , because large food industries, especially the ultra-processed food ones, use the advertisements to sell more products, not to educate consumers²⁸28. World Health Organization. Marketing food to children: changes in the global regulatory environment, 2004-2006. Geneva: WHO; 2007 [cited 2018 Oct 10]. Available from: http://www.who.int/dietphysicalactivity/publications/regulatory_environment_2007/en/
http://www.who.int/dietphysicalactivity/... .

The World Health Organization also recognizes that the massive advertising campaigns adopted by the food industries, especially those aimed at children and with different appeals (health, fitness, convenience, releases, children’s characters, among others), affect these individuals’ health. Thus, countries should review the regulatory processes regarding the propagation of these advertisements on packaging and in the mainstream media²⁸28. World Health Organization. Marketing food to children: changes in the global regulatory environment, 2004-2006. Geneva: WHO; 2007 [cited 2018 Oct 10]. Available from: http://www.who.int/dietphysicalactivity/publications/regulatory_environment_2007/en/
http://www.who.int/dietphysicalactivity/... . Concerning this, the development of audit instruments that provide an overview of these advertising practices in the consumer food environment and corroborate the DGBP will be essential for the advancement of public policies and regulation. Dietary environment indicators that enable producing more evidence about their influence are part of the strategy to face obesity and CNCD⁵5. Swinburn B, Egger G. Preventive strategies against weight gain and obesity. Obes Rev. 2002;3(4):289-301. https://doi.org/10.1046/j.1467-789X.2002.00082.x
https://doi.org/10.1046/j.1467-789X.2002... .

The advertising variables measured by AUDITNOVA showed higher inter-rater reliability than in the test-retest, including many values that could not be computed due to the low availability of advertisements in the establishments. Duran et al. also observed this fact in their study,¹⁶16. Duran AC, Lock K, Latorre MRDO, Jaime PC. Evaluating the use of in-store measures in retail food stores and restaurants in Brazil. Rev Saude Publica. 2015;49:80. https://doi.org/10.1590/S0034-8910.2015049005420
https://doi.org/10.1590/S0034-8910.20150... , and it may indicate the researcher’s difficulty in identifying the different advertising strategies available in the retail establishment and in knowing how to distinguish, in particular, the types of appeals that these advertisements bring. Advertisements with Kappa values lower than 0.40 in the test-retest were: tabloids with in natura /minimally processed food advertisements, promotional islands with ultra-processed foods, appeal to the convenience of ultra-processed foods, ultra-processed food launches, and advertisements of culinary ingredients and processed foods, in general. One hypothesis to improve the reliability of this indicator would be to conduct more than one field researcher training throughout the audit process to reaffirm the different types of appeal and approaches of food advertising in retail and/or to expand the sample of audited establishments to increase the prevalence of these types of advertising. However, researchers can still use the instrument in the field. As the instrument is built in independent blocks, they will be free to select the indicators that best fit their research goals.

The variables price and quantity of brands showed positive correlations between the measurements made by researchers 1 and 2 and visits 1 and 2. Both price and quantity of brands influence consumers when buying food²⁹29. Nau C, Kumanyika S, Gittelsohn J, Adam A, Wong MS, Mui Y, et al. Identifying financially sustainable pricing interventions to promote healthier beverage purchases in small neighborhood stores. Prev Chronic Dis. 2018;15:E12. https://doi.org/10.5888/pcd15.160611
https://doi.org/10.5888/pcd15.160611... . Measuring these aspects reliably, even over a certain time range, is essential for the use of the instrument in the monitoring and mapping of these indicators in different commercial establishments and in different social realities.

Although this study disregarded the food environment throughout the year, at certain times (for example, Christmas, Easter, Father’s Day, and Mother’s Day), price, availability, and especially advertising indicators^aaCriança e Consumo. Páscoa para quê? Responda prá gente saber! [cited 2019 Mar 27]. Available from: http://criancaeconsumo.org.br/noticias/pascoaparaque/ may vary beyond expected due to advertising campaigns and new products available on those dates. Therefore, the researcher must assess the necessity of applying the instrument in these periods.

Some of the strengths of this study are the content validation process prepared by a panel of judges specialized in food environment and food advertising, and the use of the NOVA food classification as a theoretical and analytical framework. In addition, the use of Brazilian databases, such as that of Pesquisa de Orçamentos Familiares (POF – Brazilian Family Budget Survey), provided subsidies for selecting foods that are frequently purchased by the Brazilian population. Another strength of the study is the presence of foods in greater variety in relation to Brazilian instruments, enabling the grouping according to NOVA, as well as the inclusion of more complete information on advertising, prices and quantity of brands, which may provide a more detailed overview of food environment for researchers who will use the instrument.

One of the limitations of this study is the convenience sample of only one Brazilian city and the low variety of audited business types (supermarkets, hypermarkets, and markets only). This sample does not represent the municipality and the country; however, neighborhoods have significant socioeconomic variations that may impact the food availability audited. Another limitation is the lack of evaluation of seasonal differences during the year. The instrument evaluated only retail establishments used by the population for food purchase and not for immediate consumption, such as bars and restaurants. As many individuals eat out in Brazil²¹21. Instituto Brasileiro de Geografia e Estatística. Pesquisa de Orçamentos Familiares, 2008-2009: análise do consumo alimentar pessoal no Brasil. Rio de Janeiro: IBGE; 2011. , developing and validating appropriate instruments to audit these places according to the new national food recommendations is necessary. This study did not use a quality indicator of retail food establishments based on possible scores generated by the instrument, a fact recognized as important, which will be considered for future studies.

The instrument developed, AUDITNOVA, proved to be reliable for audits in the food environment, especially in the consumer food environment, as it enables an overview of types of retail equipment in the territory and a broad analysis of the main determinants that contribute to supporting the population to choose healthier food. AUDITNOVA is reliable for measuring aspects such as availability, price, quantity of brands, and food advertising. Associations between food environment, food consumption, and obesity are becoming more frequent; however, reliable data collection instruments are needed to reach these results. The development and validation of a food environment audit instrument based on the recommendations presented in the DGBP dialogues with other Brazilian policies and supports the development of evidence that allows us to rethink the role of the food environment in availability, access, and consequently, food and nutritional security of the Brazilian population. We published the data collection training manual developed in this investigation and the AUDITNOVA instrument, which are available for download at: http://colecoes.sibi.usp.br/fsp/items/show/3364#?c=0&m=0&s =0&cv=0.

Research supported by the International Development Research Center (IDRC) and the Brazilian Institute of Consumer Protection (Instituto Brasileiro de Defesa do Consumidor – IDEC) for data collection and São Paulo Research Foundation (Process 2016/12766-6)

REFERENCES

Publication Dates

History