Abstract

The study aimed to develop a Quantitative Health Inspection Instrument (IQIS) large-sized Brazilian food and nutrition services. The inspection technology based on the Potential Risk Assessment Model (MARP) and the Brazilian Health Legislation was used. Twelve dimensions, 41 modules, and 57 risk control (critical/non-critical) indicators were structured on a scale of 0-5, totalling 1,512 indices with closed-ended response coding. The IQIS was validated with the Kappa Coefficient, with excellent agreement for the attributes of clarity and relevance (k = 0.82 and k = 0.92) and good agreement for applicability (k = 0.78). The Kruskal-Wallis test showed no statistically significant difference between the assessments (p = 0.423), the Intraclass Correlation Coefficient was satisfactory (ICC = 0.53), and Cronbach’s Alpha (α = 0.71) was acceptable. The final result made it possible to classify the service as having an unacceptable health risk. IQIS is considered to have validated content, be reliable and reproducible to assess the hygienic-sanitary conditions, being a technological innovation for food and nutrition services and sanitary, allowing a detailed and rigorous inspection.

Key words
Sanitary Inspection; Sanitary Risk; Food Services; Validation Studies

Introduction

Foodborne diseases (FBD) are common and underreported in Brazil. The World Health Organization (WHO) estimates that one-third of the population will suffer from FBD annually, but only a small part will make the notification¹1 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Departamento das Doenças Transmissíveis. Vigilância Epidemiológica das Doenças Transmitidas por Alimentos - VE - DTA. Brasília: MS; 2014.

2 Oliveira ABA, Paula CMD, Capalonga R, Cardoso MRI, Tondo EC. Doenças transmitidas por alimentos, principais agentes etiológicos e aspectos gerais: uma revisão. Rev HCPA 2010; 30(3):279-285.^-33 Gould LH, Walsh KA, Vieira AR, Herman K, Williams IT, Hall AJ, Cole D .Surveillance for foodborne disease outbreaks - United States, 1998-2008. MMWR Surveill Summ 2013; 62(2):1-34.. According to the Centers for Disease Control and Prevention (CDC), 48 million people fall ill each year from FBD in the U.S. In 2014, the CDC reported 864 outbreaks, resulting in 13,246 diseases, 712 hospitalizations and 21 deaths⁴4 Centers for Disease Control and Prevention (CDC). Surveillance for Foodborne Disease Outbreaks, United States, 2014: Annual Report. Atlanta: US Department of Health and Human Services, CDC; 2016.. In Brazil, 11,241 outbreaks were recorded in the period 2000-2015. Of these, 218,507 people became ill, and 2,121,110 were exposed. Fifteen percent of the outbreaks were related to food consumed in restaurants and bakeries, and 8.2% with housing and work⁵5 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Vigilância Epidemiológica das Doenças Transmitidas por Alimentos. São Paulo: MS; 2015.. Food contamination can occur throughout the production chain, and risk management is a significant health control measure, which is essential to avoid it⁶6 Souza EL, Silva CA, Sousa CP. Bacteriocins: molecules of fundamental impact on the microbial ecology and potential food biopreservatives. Braz Arch Biol Technol 2005; 48(4):559-566.^,77 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Guia de vigilância epidemiológica. Brasília: MS; 2010. (Série A. Normas e Manuais Técnicos).. A considerable proportion of food outbreaks arise from the association between the consumption of food contaminated by improper handling and conservation or distribution⁸8 Greig J, Ravel A. Analysis of foodborne outbreak data reported internationally for source attribution. Int J Food Microbiol 2009; 130(2):77-87.^,99 Scallan E, Griffin P, Angulo F, Tauxe R, Hoekstra R. Foodborne Illness Acquired in the United States - Unspecified Agents. Emerg Infect Dis 2011; 17(1):16-22..

Regulation of health risk in food services

The oldest technical health control regulation is the Codex Alimentarius, established by the United Nations Food and Agriculture Organization and the WHO¹⁰10 Codex Alimentarius Commission (CAC). Hazard analysis and critical control point (HACCP) system and guidelines for its application [anexo na Internet]. Rev. 4. CAC/RCP 1-1969; 2003. [acessado 2018 Jun 6]. Disponível em: https://www.loex.de/files/downloads/lebensmittel/Codex%20Alimentarius%20(EN).pdf
https://www.loex.de/files/downloads/lebe... . In 1993, the Food Hygiene Committee of the Codex published a guide for the application of the Hazard Analysis and Critical Control Point (HACCP) system, whose application in Brazil is based on Ordinance Nº 1.428/1993¹¹11 Virgínia MSA, Ricardo SDP. Programa Boas Práticas de Fabricação. Recife: EDUFRPE; 2012.^,1212 Brasil. Ministério da Saúde (MS). Portaria nº 1.428, de 26 de novembro de 1993. Aprova o regulamento técnico para inspeção sanitária de alimentos, as diretrizes para o estabelecimento de boas práticas de produção e de prestação de serviços na área de alimentos. Diário Oficial da União 1993; 2 dez. of the Ministry of Health. In Brazil, the publication of RDC Nº 216/2004 is a health control milestone, establishing minimal conditions for food and nutrition services, but without an inspection roadmap¹³13 Brasil. Resolução RDC n°216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set.. The Health Surveillance Center of the State Health Secretariat of São Paulo published Ordinance Nº CVS 5/2013¹⁴14 São Paulo. Portaria CVS nº 5, de 09 de abril de 2013. Aprova o Regulamento Técnico sobre Boas Práticas para Estabelecimentos comerciais de alimentos e para serviços de alimentação, e o roteiro de inspeção. Diário Oficial do Estado de São Paulo 2013; 9 abr., complementing the procedures of RDC N^o 216/2004, defining an inspection roadmap with statewide coverage.

The health inspection technology is highlighted as an instrument of risk management, assessing compliance with health legislation⁷7 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Guia de vigilância epidemiológica. Brasília: MS; 2010. (Série A. Normas e Manuais Técnicos). throughout the food chain. The usual health inspection routines evaluate the services using dichotomous variables with compliant and non-compliant indicators, which show the level of compliance with the legislation, without judging the criticality of the analyzed items, such as the Roadmap for the Assessment of Hygienic-Sanitary Conditions in Food Services¹⁵15 Mello A, Carmo C, Leite S, Miguel M, Colares L. Elaboração, validação de conteúdo e da confiabilidade do instrumento para avaliação higiênico-sanitária de serviços de alimentação. Vig Sanit Debate 2014; 2(3):86-93.. The Integrated Handbook for Prevention and Control of Foodborne Diseases⁷7 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Guia de vigilância epidemiológica. Brasília: MS; 2010. (Série A. Normas e Manuais Técnicos)., published by the Ministry of Health, provides for an inspection roadmap that defines a criterion for classifying indicators based on the relevance of individual risk regarding product quantity and safety and workers’ safety. The School Feeding Good Practices Checklist¹⁶16 Stedefeldt E, Cunha D, Silva Júnior Ê, Silva S, Oliveira A. Instrumento de avaliação das Boas Práticas em Unidades de Alimentação e Nutrição Escolar: da concepção à validação. Cien Saude Colet 2013; 18(4):947-953. considers the consequence potential of each indicator. On May 10, 2013, the Ministry of Health approved Ordinance Nº 817/2013, with the national guidelines for the elaboration and implementation of the Food Service Categorization Pilot Project¹⁷17 Brasil. Portaria nº 817, de 10 de maio de 2013. Aprova as diretrizes nacionais para a elaboração e execução do projeto-piloto de categorização dos serviços de alimentação para a Copa do Mundo FIFA 2014. Diário Oficial da União 2013; 10 maio., implemented in the host cities of the 2014 FIFA World Cup, providing for the classification of the criticality of each indicator, health and safety consequences, level of association concerning outbreak-associated flaws, and weighting to establish values¹⁸18 Brasil. Agência Nacional de Vigilância Sanitária (Anvisa). Categorização dos Serviços de Alimentação: Elaboração e Validação da Lista de Avaliação. São Paulo: MS; 2013.

Technological innovation for large food services

Health surveillance has three sets of practices, with varying risk notions depending on the strategy. Health promotion actions aimed at group education aim at increasing the quality of health of the population and are unrelated to a specific risk factor. Risk or harm prevention actions act on specific factors, based on epidemiological risk, to reduce or eliminate new occurrences. Health protection actions seek to strengthen defenses. They address risk as possible harmful events to health¹⁹19 Leite DLH, Navarro M. Risco potencial: um conceito de risco operativo para vigilância sanitária. In: Costa EA. Vigilância Sanitária: temas para debate. Salvador: EDUFBA; 2009. p. 61-82.. Considering its dynamic nature, it is necessary to search for new technologies that consider the complex and cross-cutting nature of the processes and manage health risks²⁰20 Leite DLH. Vigilância sanitária em serviços de saúde: risco e proteção da saúde em serviços de hemodiálise [tese]. Salvador: Universidade Federal da Bahia; 2007.. All the health inspection guidelines found in the literature¹⁵15 Mello A, Carmo C, Leite S, Miguel M, Colares L. Elaboração, validação de conteúdo e da confiabilidade do instrumento para avaliação higiênico-sanitária de serviços de alimentação. Vig Sanit Debate 2014; 2(3):86-93.^,1616 Stedefeldt E, Cunha D, Silva Júnior Ê, Silva S, Oliveira A. Instrumento de avaliação das Boas Práticas em Unidades de Alimentação e Nutrição Escolar: da concepção à validação. Cien Saude Colet 2013; 18(4):947-953.^,1818 Brasil. Agência Nacional de Vigilância Sanitária (Anvisa). Categorização dos Serviços de Alimentação: Elaboração e Validação da Lista de Avaliação. São Paulo: MS; 2013^,2121 Spexoto AA. Aplicação do sistema de análise de perigos e pontos críticos de controle (APPCC) em propriedades leiteiras [tese]. São Paulo: Universidade de São Paulo; 2003. are based on the probabilistic risk concept. The notion of risk concept, of great relevance in the area of health surveillance, was thus proposed²²22 Navarro VTNM. Risco, Radiodiagnóstico e Vigilância Sanitária. Salvador: EDUFBA; 2009. to cover the complexity of a risk concept.

The potential risk has two essential characteristics that differentiate it from the concept of classic risk, relating to the possibility and not to the probability of occurrence of the unexpected. A classical risk evaluation is based on events that have occurred, while the potential risk builds on those that are occurring and the effects that may or may not occur. The potential risk can be quantified and classified into levels of acceptability, and its operationalization enables the monitoring and comparison of several objects under health surveillance control. In this context, the Potential Risk Assessment Model (MARP) was developed to gauge risk and its classification in the space of acceptability¹⁹19 Leite DLH, Navarro M. Risco potencial: um conceito de risco operativo para vigilância sanitária. In: Costa EA. Vigilância Sanitária: temas para debate. Salvador: EDUFBA; 2009. p. 61-82.. It classifies the potential risk into acceptable, tolerable and unacceptable through mathematical formalism. MARP’s application is particularized from the risk control indicators, based on a defined acceptability scale classified into critical and non-critical²²22 Navarro VTNM. Risco, Radiodiagnóstico e Vigilância Sanitária. Salvador: EDUFBA; 2009. categories.

This study aimed at developing the IQIS for large food services based on the MARP and Brazilian legislation. The use of the potential risk concept evidenced advances in health risk management in related areas such as hemodialysis²⁰20 Leite DLH. Vigilância sanitária em serviços de saúde: risco e proteção da saúde em serviços de hemodiálise [tese]. Salvador: Universidade Federal da Bahia; 2007. and radiodiagnosis²²22 Navarro VTNM. Risco, Radiodiagnóstico e Vigilância Sanitária. Salvador: EDUFBA; 2009. services. Its application in food and nutrition services is a technological innovation, enabling risk anticipation and health protection.

Methods

This study was developed from October 2015 to May 2016, with the following steps: i) Elaboration of the instrument by sector health professionals; ii) IQIS evaluation concerning content validity through submission to the Expert Committee, as well as external validity and reliability for the performance of Pretest and Test, applied in large food and nutrition service in São Sebastião do Passé, Bahia, Brazil, which prepares 2,541 daily meals for workers: 276 breakfasts, 1,393 lunches, 243 dinners and 629 snacks. The Research Ethics Committee of the Bahia School of Medicine and Public Health approved the research. Figure 1 summarizes the IQIS development and validation process.

Elaboration of the IQIS

This stage counted on a group of seven professionals, of which two are doctors in public health and risk management researcher, two are ergonomists and managers in workers’ health, two are nutritionist experts in public health and experience in collective nutrition, and one is a health surveillance expert working in the area of risk management. The literature review allowed the analysis of the existing instruments and methods, as well as the theoretical basis for the decision and elaboration of a new instrument. Following the identification of primary sources, review of secondary sources, critical reading and summary of the literature was performed using keywords or descriptors: risk, health risk, epidemiological risk, potential risk, health surveillance, health inspection, food health inspection, health inspection roadmap, health inspection in restaurants roadmap, industrial kitchen health inspection roadmap, health inspection validation, questionnaires, evaluation instruments, and elaboration and validation of instrument²¹21 Spexoto AA. Aplicação do sistema de análise de perigos e pontos críticos de controle (APPCC) em propriedades leiteiras [tese]. São Paulo: Universidade de São Paulo; 2003.^,2323 Tobias W, Ponsano E, Pinto M. Elaboração e implantação do sistema de análise de perigos e pontos críticos de controle no processamento de leite pasteurizado tipo A. Cien Rural 2014; 44(9):1608-1614.

24 Belik W. Perspectivas para segurança alimentar e nutricional no Brasil. Saude Soc 2003; 12(1):12-20.

25 Cavalli S, Salay E. Gestão de pessoas em unidades produtoras de refeições comerciais e a segurança alimentar. Rev Nutr 2007; 20(6):657-667.

26 Figueiredo A, Miranda M. Análise de Risco aplicada aos alimentos no Brasil: perspectivas e desafios. Cien Saude Colet 2011; 16(4):2251-2262.

27 Kepple A, Segall-Corrêa A. Conceituando e medindo segurança alimentar e nutricional. Cien Saude Colet 2011; 16(1):187-199.^-2828 Maluf SR, Menezes F, Valente LF. Contribuição ao Tema da Segurança Alimentar no Brasil. Rev Cadernos Debate 1996; IV:66-88..

The IQIS is based on the MARP, whose mathematical formalism is detailed by Navarro²²22 Navarro VTNM. Risco, Radiodiagnóstico e Vigilância Sanitária. Salvador: EDUFBA; 2009., applied to food and nutrition services. This model proposes underlying processes according to the natural flow of the inspector. The main activities that are a potential risk for each of these processes are defined, as are the risk control indicators for each of these activities. Finally, each indicator is classified as critical (I_C) and non-critical (I_NC) and associated with an interval scale from zero to 5, where zero evidences nonexistent or inadequate risk control and five is excellent risk control, with the following classification: 0 - nonexistent or inadequate; 1 - insufficient; 2 - reasonable; 3 - good; 4 - very good and 5 - excellent.

The coding of closed-ended answers for each index of the scale was developed to reduce the subjectivity of the evaluator, with six possibilities for each indicator. The food production flow was used as a criterion to define the realms, which were broken down into activities, areas or equipment that generate potential risk, called modules. These developed into risk control indicators and were associated with indices with quantitative variables, coded in closed-ended responses. Fourteen eight-hour meetings were held with professionals, who employed RDC N^o 216/2004. The requirements to meet the resolution were considered in index 3 of each indicator, and the lower and upper grades were defined from the experience and literature researched.

A panel was developed with the professionals to classify indicators into critical and non-critical; these professionals used their knowledge for this task and, later, the table of the association of risk factors for the occurrence of outbreaks was elaborated by National Health Surveillance Agency specialists¹⁸18 Brasil. Agência Nacional de Vigilância Sanitária (Anvisa). Categorização dos Serviços de Alimentação: Elaboração e Validação da Lista de Avaliação. São Paulo: MS; 2013. The panel was developed from three 8-hour meetings, in which each member expressed his opinion on the indicator in question, achieving the group’s consensus.

IQIS validation

Many validity and reliability methods are mentioned in the research literature²⁹29 Hernández SR, Fernández CC, Baptista LP. Metodología de la investigación. 4ª ed. México: McGraw-Hill; 2007.

30 D'Ancona MAC. Metodología cuantitativa: estrategias y técnicas de investigación social. Madrid: Síntesis Sociología; 1999.

31 Polit D, Hungler B, Palacios MR, Féher de la TG. Investigación científica en Ciencias de la Salud. México: McGraw-Hill Interamericana; 2007.^-3232 Coluci M, Alexandre N, Milani D. Construção de instrumentos de medida na área da saúde. Cien Saude Colet 2015; 20(3):925-936. and, concerning methodological conduction, we chose the definitions that establish validity as how much a test measures what we wish to gauge and what reliability is related to the accuracy and precision of the measurement procedure³³33 Cooper D, Schindler P. Métodos de Pesquisa em Administração. 12ª ed. São Paulo: McGraw Hill; 2016..

A Judges’ Committee was established to validate the content and consisted of three nutritionists, with average professional experience in the areas of food safety and health surveillance of 17 years; one worked as a teacher and two at the Regional Council of Nutritionists. One had a master’s degree, and two had specialization in health surveillance. The first version of the IQIS was submitted to the Judges’ Committee, along with the electronic spreadsheet for content evaluation, regarding the attributes of applicability, clarity, and relevance. Applicability was evaluated on a scale with four alternatives: 1 - fully agree, 2 - partially agree, 3 - no opinion, and 4 - disagree. Clarity was evaluated on a scale with three alternatives: 1 - high, 2 - fair and 3 - low, to verify whether the indicators were written intelligibly. Relevance was assessed on a scale with three alternatives: 1 - relevant, 2 - not relevant, and 3 - not applicable, noting if the items reflected the concepts involved and if they were adequate to achieve the proposed objectives. An eight-character code was created for each set of realm, module, indicator and index to organize the database generated in this stage. Experts gathered for two days in a quiet and uninterrupted environment, answered the 1,344 evaluations and were instructed not to exchange information with each other. The Kappa coefficient was used to analyze the results, considering k > 0.80-1.00 as excellent concordance; k > 0.60-0.79, good concordance; k > 0.40-0.59, moderate concordance; k > 0.20-0.39, poor concordance and k > 0-0.19, no concordance³⁴34 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33(1):159-174.. All results with a Kappa Coefficient above 0.60 were accepted, i.e. showing good concordance.

The Pretest checked whether all items were understandable to members of the target population. To that end, we invited five nutritionists, with a mean age of 42 years and an average of 14 years of experience in this area, all with specialization in health surveillance or related areas. The IQIS was applied on the same day, by direct observation for all the risk control indicators, except for the documentary ones, answered by interviews with the sector’s managers, which was conducted over eight hours during the food service’s administrative hours. The evaluators received the printed instrument and were instructed to assess the modules at the same time, as well as not to exchange information among themselves, to avoid influencing the answers. We used interobserver frequency statistical tests in the pre-test analysis, discussing the results in an interactive process between researchers and nutritionists to clarify controversial points. All items with a percentage of concordance less than 80% were analyzed to eliminate or adjust ambiguous indicators or indices or that carried value judgments. We assessed whether the concepts were drafted in a way that was comprehensible to what was expected to be measured and adequate for the proposed objectives. The documents and their modification proposals were analyzed and accepted, generating the second version of IQIS.

The test mainly aimed to evaluate the psychometric characteristics of the instrument. To this end, we invited seven nutritionists with a mean age of 38 years and an average of 11 years’ food production experience. All of them had specialization in health surveillance or related areas, and applied the second version of IQIS in the same food and nutrition service of the pre-test, following the same previous conditions. The data were collected with the printed forms imported to the SPSS software version 24.0 to perform the analysis³⁵35 IBM Corp. IBM SPSS Statistics [Windows]. Versão 21.0. New York: IBM Corp; 2012..

Statistical analysis

Data statistical treatment was performed through descriptive and exploratory analyses to investigate the accuracy of entries, the distribution of missing cases and the distribution of frequencies. The non-parametric analysis of variance was used to compare the percentages of the adequacy of hygienic-sanitary conditions of the food and nutrition service, employing the Kruskal-Wallis test and considering the significance level of 5% in all analyses³⁶36 Marôco J. Análise Estatística com Utilização do SPSS. 3ª ed. Lisboa: Edições Sílabo; 2007.. The Intraclass Correlation Coefficient was applied to verify the reliability and usability of the instrument in other contexts using the same Test data, considering that in the case of ICC ≥ 0.75, reliability should be considered excellent; an ICC of 0.40-0.75 shows satisfactory reliability; and an ICC < 0.40³⁷37 Wynd C, Schmidt B, Schaefer M. Two Quantitative Approaches for Estimating Content Validity. West J Nurs Res 2003; 25(5):508-518. reflects poor reliability. Finally, the Cronbach Alpha coefficient was calculated for each realm to verify the internal reliability, where α ≥ 0.70 values were considered acceptable and α ≥ 0.80³⁸38 Streiner D, Norman G. Health measurement scales. Oxford: Oxford University Press; 2015.^,3939 Hair Jr. JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª ed. São Paulo: Bookman; 2009. were highly reliable values.

Results

The IQIS was defined in its first version based on the requirements of RDC N^o 216/2004¹³13 Brasil. Resolução RDC n°216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set. and the production process flow. It contained 12 realms and 41 modules, spread into 76 risk control indicators and 1,344 indices associated with quantitative variables. Also, indicators and indices were associated with each realm and module to assess the respective health risk, resulting in 224 different verification items. Each realm-module-indicator-index combination features an IQIS item. Box 1 shows an example of an IQIS Verification Item.

The panel of experts initially ranked 42% of the 76 indicators as critical. After the Pre-test and evaluation of the concordance rate, 57 indicators were redefined and evaluated in light of the risk factors table for the event of outbreak¹⁸18 Brasil. Agência Nacional de Vigilância Sanitária (Anvisa). Categorização dos Serviços de Alimentação: Elaboração e Validação da Lista de Avaliação. São Paulo: MS; 2013. The Kappa Coefficient was used to validate the content. The IQIS items were evaluated for applicability, clarity and relevance, and results showed that the instrument has a high interobserver concordance, as per Table 1.

The applicability attribute obtained 78% of the option I fully agree in the response of the judges. Clarity was evaluated as high, obtaining 85% of the answers. The relevance attribute achieved the option relevant with 91% of the answers. The IQIS was considered validated with an excellent agreement for the attributes of clarity and relevance, with k = 0.82 and k = 0.92, respectively, and good agreement for the applicability attribute, with k = 0.78. Suggestions to change the index texts (30%) were analyzed and accepted and gave rise to the second version of IQIS, adjusted to the opinion of the Committee. The results of the application were analyzed as to the rate of agreement between them. Of the 224 responses of each evaluator, 115 had an agreement of less than 80%, which were reevaluated by the researchers and adjusted, giving rise to the second version. Following content validation, the IQIS was restructured with 12 realms, 41 modules, 57 indicators (30 critical and 27 non-critical), 252 items and 1,512 indices.

The Kruskal-Wallis test was applied to the data resulting from the evaluation of the seven nutritionists in the Test and showed no statistically significant difference (p = 0,423) between the evaluations obtained for both the instrument as a whole and its realms. The items were evaluated according to the risk for the maintenance of hygienic-sanitary quality, classified in the 12 realms, as shown in Table 2.

When calculating the proportion of adequate hygienic-sanitary conditions of the food service and of each realm, we observed that the IQIS instrument allowed us to identify the items that required correction in order to comply with the legal requirements, since there was agreement on the answers indicated. There was no statistically significant difference between the evaluations of nutritionists by the Kruskal-Wallis test, showing that the instrument is reliable and reproducible. Using the same database, the statistical analyses of the Cronbach Alpha and the Intraclass Correlation Coefficient by realm were performed to verify IQIS’ reliability, which evidenced acceptable reliability. The Kruskal-Wallis test showed that there was no statistically significant difference between the evaluations (p = 0.423), the Intraclass Correlation Coefficient was satisfactory (ICC = 0.53), and the Cronbach’s Alpha was considered acceptable (α = 0.71).

The Test’s database was used, and the mathematical formalism of the MARP was applied to each realm of the IQIS to evaluate the potential risk range of the food and nutrition service. The rates of compliance were analyzed to evaluate its effectiveness, in which the responses indicated in indices 3, 4 and 5 were “compliant”. Table 3 shows the classification of each realm vis-à-vis the potential risk variation range, as acceptable, tolerable or unacceptable. The results of the application of the IQIS showed that the food and nutrition service evaluated had a mean compliance rate of 35% in the evaluation of the risk control indicators and the potential risk classified in the range of risk variation was deemed unacceptable.

Discussion

In the current Brazilian social and economic context, health control measures require risk management of hygienic-sanitary conditions of food and nutrition services as an essential factor to reduce the incidence of FBDs. Thus, the preparation and validation of a quantitative instrument for inspection of large food and nutrition services, based on the MARP, support prevention and control measures. The IQIS was developed and tested for its validity to corroborate this, with consistent results that allow the immediate application. Its elaboration considered the RDC Nº 216/2004¹³13 Brasil. Resolução RDC n°216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set. in full, which contributes to standardized terminology, besides ensuring that all national legal requirements be considered. Similar studies, such as the Food Service Categorization Assessment List¹⁸18 Brasil. Agência Nacional de Vigilância Sanitária (Anvisa). Categorização dos Serviços de Alimentação: Elaboração e Validação da Lista de Avaliação. São Paulo: MS; 2013 and the Best Practices in School Food (BPAE)¹⁶16 Stedefeldt E, Cunha D, Silva Júnior Ê, Silva S, Oliveira A. Instrumento de avaliação das Boas Práticas em Unidades de Alimentação e Nutrição Escolar: da concepção à validação. Cien Saude Colet 2013; 18(4):947-953., used methods to prioritize items in RDC Nº 216/2004¹³13 Brasil. Resolução RDC n°216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set., which were more relevant to health risk control, as well as developed their respective risk classification methods at the end of the application of the lists. Despite the different methodologies for classification of risk control indicators as critical and non-critical, a similarity between the IQIS critical indicators and the Food Service Categorization Checklist is observed¹⁸18 Brasil. Agência Nacional de Vigilância Sanitária (Anvisa). Categorização dos Serviços de Alimentação: Elaboração e Validação da Lista de Avaliação. São Paulo: MS; 2013. The MARP applied to food and nutrition services resembles HACCP in its purpose of ensuring food security, but evidences different methodological aspects, since it is used to analyze potential hazards of operations, based on the concept of probabilistic risk. On the other hand, the MARP works with the potential risk, regarding a possible health problem, without necessarily describing the problem and its probability. It is a concept that expresses the value judgment about the potential exposure to a possible risk¹⁹19 Leite DLH, Navarro M. Risco potencial: um conceito de risco operativo para vigilância sanitária. In: Costa EA. Vigilância Sanitária: temas para debate. Salvador: EDUFBA; 2009. p. 61-82., and this is an advance in food safety assurance.

Although it is usual to use the health inspection instrument to collect nonconformities in food and nutrition services, it is necessary to use methods to classify the risk control indicators, defining their respective criticalities, for the adequate health risk management. Table 3 data analysis showed that the proportion of nonconformities in each realm is not always equivalent to their level of criticality. We observed that, concerning realms VII. Raw materials, Ingredients, and packaging; VIII. Food preparation; and X. Exposure to the consumption of the prepared food, while more than 60% of their items were evaluated with appropriate hygienic-sanitary conditions, they were classified with a potentially unacceptable risk and jeopardized the whole system. The MARP facilitates the comparison of the potential risks evaluated, guiding crucial stages of risk management, as well as implementing corrective measures in cases of failure. A similar study evaluates the risk classification of food and nutrition services as a reliable strategy for risk communication and food security promotion, significantly contributing to reduced foodborne diseases⁴⁰40 Cunha DT, Saccol ALF, Tondo EC, Oliveira ABA, Ginani VC, Araújo CV, Lima TAS, Castro AKF, Stedefeldt E. Inspection score and grading system for food services in Brazil: the results of a food safety strategy to reduce the risk of foodborne diseases during the 2014 FIFA World Cup. Front Microbiol 2016; 7:614..

The validation of quantitative instruments, with a view of risk management for food safety control in collective food and nutrition, is useful and gains substantial significance by supporting various professionals in the field of research and practice, enabling an accurate view of the most critical issues, without losing sight of the other items required by law. IQIS has been shown to be a viable application instrument, with a reduced level of interference of the evaluator due to the objective response options, appropriate for a detailed and thorough inspection, which takes time to complete in large food and nutrition services. It should be noted that this time difference to implement the IQIS and dichotomous instruments is not significant, which reinforces its applicability by private nutritionists, as well as by regulatory bodies linked to health surveillance.

The lack of validation studies involving quantitative evaluation tools for food and nutrition services - which include all the items required in RDC N^o 216/2004¹³13 Brasil. Resolução RDC n°216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set. - and of a gold standard appear as hindrances to the design of this study, which led to the combination of several methodological strategies to ensure the validation of the IQIS. Among the main limitations of this research is that sample size was restricted by the need of the group of evaluators to apply the IQIS concurrently, since the reality of food production is dynamic and changes its hygienic-sanitary conditions in the respective sectors as per the production flow throughout the day.

Conclusions

The results of the IQIS evidence content validity and it can be used with good reliability and reproducibility by nutritionists to evaluate the hygienic-sanitary conditions of the large food and nutrition services and to manage the priority risks, supporting the adoption of best practices of food handling and favoring the prevention of diseases. The contributions evidenced the relevance of developing the IQIS, taking into account the need for nutritionists to have a specific technological innovation for the segment of collective feeding that meets the requirements of RDC Nº 216/2004¹³13 Brasil. Resolução RDC n°216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set. and allows health risk management, covering all its complexity. Similar work will generate knowledge and tools to ensure health risk management in food and nutrition services, thus contributing to the provision of safer food with a lower probability of foodborne diseases. An essential benefit of the work is the appropriate management of resources in the hierarchy of improvements, fundamental in the current socioeconomic context of crisis and scant resources. We recommend that the technical application of IQIS be extended to other food and nutrition services, as well as its possible be considered for use by regulatory agencies such as Health Surveillance. As a perspective for future studies, we suggest the preparation of instructional material for a better understanding of the instrument by professionals employing the IQIS and a more objective and appropriate version for application in realities with reduced availability of resources or health emergency situations.

References

Publication Dates

History