Abstract

This work aimed to draft and validate a checklist of the production processes of pediatric diets for use in hospital lactaries. A bibliographic search was carried out in order to draft the instrument. Content validation was performed with 5 experts, using the Delphi technique and the 5-point Likert scale. Appearance validation was performed by 3 nutritionists, in the same environment, day, and time. The instrument included a header, a guide to completion instructions, and 225 assessment items that obtained validated content. The minimum and maximum values for the content validity index (CVI) were 0.88 and 0.96, respectively. The intraclass correlation coefficient (ICC) and Cronbach’s alpha (α) was >0.80 in more than 90% of the items evaluated. The minimum and maximum values for the Appearance Validity Index (AVI) were, respectively, 0.73 and 0.93. The Kruskal Wallis test showed no significant difference during the evaluations (p-value>0.05) for the blocks of the routine version and the management version. The instrument versions presented in more than 80% of the α and ICC blocks > 0.80. Therefore, the instrument presented validated content and appearance, presenting reproducibility and reliability in terms of the feasibility of use in a nutritionist’s practice.

Key words:
Delphi technique; Infant Formulas; Checklist; Lactary; Validation

Introduction

Foodborne diseases (FBD) are defined as a syndrome that generally includes anorexia, nausea, vomiting, and/or diarrhea, and is attributed to the intake of food or water that has been contaminated by bacteria, viruses, parasites, toxins, pesticides, chemical products, and heavy metals¹1 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Coordenação Geral de Desenvolvimento da Epidemiologia em Serviços. Guia de Vigilância em Saúde. Brasília: MS; 2019.^,22 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Departamento de Vigilância das Doenças Transmissíveis. Guia de Vigilância em Saúde. Brasília: MS; 2018..

FBDs can be subdivided into three categories: (1) intoxication, caused by the intake of foods containing pre-formed microbial toxins during the intense proliferation of pathogenic microorganisms in foods; (2) infection, caused by the intake of foods containing cells viable to pathogenic microorganisms that can migrate to other tissues; (3) toxic infections caused by the contamination of the foods after the intake of these products, which can lodge in the intestinal mucosa, where the process of multiplication, sporulation, or lysis caused by enterocytes occurs³3 Sirtoli DB, Comarella L. O papel da vigilância sanitária na prevenção das doenças transmitidas por alimentos (DTA). R Saude Desenvol 2018; 12(10):197-209.^,44 Lombardi EC, Bonnas DS, Jardim FBB, Oliveira KA, Silva RT. Atuação dos profissionais de saúde na investigação de suspeitas de surtos de DTA nos hospitais de Uberlândia, Minas Gerais. Rev Segur Aliment Nutr 2020; 27:e020005..

In Brazil, an epidemiological survey was conducted by region by the Brazilian Ministry of Health, through the Health Surveillance Secretariat (SVS, in Portuguese) and the National Disease Notification System (SINAN, in Portuguese) between 2000 and 2017. It was verified that the preparations manipulated in hospital lactaries, the milk, and its derivatives are involved in outbreaks of FBD, as are food products for special nutritional uses, such as enteral diets, nutrition modules, among others²2 Brasil. Ministério da Saúde (MS). Secretaria de Vigilância em Saúde. Departamento de Vigilância das Doenças Transmissíveis. Guia de Vigilância em Saúde. Brasília: MS; 2018..

FBDs in the context of a hospital environment are characterized as infections referent to healthcare-related infections (IRAS, in Portuguese), primarily involving newborns of gestational age and adequate birth weight, as well as preterm births that present a low birth weight, that is, a weight under 2,500 grams⁵5 Accioly E, Sauders C, Lacerda EMA. Nutrição em Obstetrícia e Pediatria. 2º ed. Rio de Janeiro: Guanabara Koogan e Cultura Médica; 2012.. This population presents immature intestinal and immunological systems and are thus more prone to risks of infant morbidity and mortality⁵5 Accioly E, Sauders C, Lacerda EMA. Nutrição em Obstetrícia e Pediatria. 2º ed. Rio de Janeiro: Guanabara Koogan e Cultura Médica; 2012.

6 Rossi P. Avaliação de perigos microbiológicos no preparo de formulações infantis em lactário hospitalar [dissertação]. São Paulo: Universidade Estadual de Campinas; 2007.^-77 Carvalho ML, Araújo TRN, Santos CFB, Sousa AFL, Moura MEB. Infecções hospitalares em unidade de terapia intensiva neonatal. R Interd 2014; 7(4):189-198..

IRAS can be characterized as early, when they occur within the first 48 hours of life, and as late, when they occur after 48 hours of life. As it is considered an emerging problem in public health, it is estimated that in Brazil approximately 15% of the hospitalized patients contract some type of IRAS, considering that the higher indices (18.4%) can be found in patients hospitalized in public hospitals and represent the main cause of hospital morbidity and mortality, increasing patients’ time of hospitalization, thereby increasing hospital costs and reducing the turnover of hospital beds⁶6 Rossi P. Avaliação de perigos microbiológicos no preparo de formulações infantis em lactário hospitalar [dissertação]. São Paulo: Universidade Estadual de Campinas; 2007.^,88 Organização Pan-Americana da Saude (OPAS). Centro Latino-Americano de Perinatologia, Saúde da Mulher e Reprodutiva. Prevenção de infecções relacionadas a assistência à saúde em neonatologia. Montevideu: CLAP/SMR-OPS/OMS; 2016.

9 Castilhos APM, Souza TP, Almeida CPB. Prevenção de infecção hospitalar em unidades de internação pediátrica: Uma revisão da literatura. RSC 2016; 12(3):656-665.^-1010 Brasil. Ministério da Agricultura, Pecuária e do Abastecimento (MAPA). Portaria nº 46, de 10 de fevereiro de 1998. Institui o Sistema de Análise de Perigos e Pontos Críticos de Controle - APPCC a ser implantado, gradativamente, nas indústrias de produtos de origem animal sob o regime do Serviço de Inspeção Federal - SIF, de acordo com o Manual Genérico de Procedimentos; Diário Oficial da União 1998; 10 fev..

The contamination of infant formulas can occur due to the unpreparedness of the food handlers, which can lead to poor hygiene habits and to flaws throughout the production process. It is also possible to consider that the inadequacy of the time/temperature binomial during the preparation and distribution of the food ready for consumption, together with the non-conformities during storage, can contribute to the multiplication of microorganisms and make the food improper for consumption¹¹11 Cavalli SB, Salay E. Gestão de pessoas em unidades produtoras de refeições comerciais e a segurança alimentar. Rev Nutr 2007; 20(6):657-667.^,1212 Piovacari SMF, Figueira VACR, Potenza ALS. Segurança alimentar: lactário. Educ Contin Saude Einstein 2009; 7(4):216-218..

To minimize the risk of FBD in hospital units, it is important to implement best practice (BP) measures for the handling of food, which can be monitored by a checklist.

The checklist is an easily applied instrument and provides fast results that enable the identification of non-conformities to guide the adoption of corrective measures. Nonetheless, it is important for the instrument to have a valid content so that it can be comprised of essential items that provide a reliable evaluation of the phenomenon to be studied.

Due to the non-existence of a checklist developed by an inspection agency that is standardized, specific, and official for use in hospital lactaries, there are currently studies using the application of this type of instrument in these locations, basing themselves on Brazilian laws and/or support manuals or programs from the public or private institutions¹³13 Rossi P, Kabuki DY, Kuaye AY. Avaliação microbiológica do preparo de fórmula láctea infantil em lactário hospitalar. RIAL 2010; 69(4):503-509.

14 Linhares IW. Avaliação das condições higiênico-sanitárias no preparo de fórmulas infantis em lactário hospitalar: sanitárias no preparo de fórmulas infantis em e lactário hospitalar [dissertação]. Minas Gerais: Universidade Federal de Minas Gerais; 2012.

15 Reginato A, Penna FL, Trento FKS, Giordano LCRS, Kinchoku H, Antunes EC. Qualidade microbiológica de fórmulas infantis administradas em hospital público do município de Campinas, São Paulo. Rev Segur Aliment Nutr 2014; 21(1):387-394.

16 Siqueira DR. Gestão da qualidade no preparo de dietas enterais e fórmulas infantis no lactário de um hospital universitário [dissertação]. Rio de Janeiro: Universidade Federal do Rio de Janeiro; 2016.^-1717 Zancanaro F, Mendes MA, Lemos MP, Schmeling TB. Condições higienicossanitárias das instalações e dos procedimentos de elaboração e distribuição de fórmulas infantis em lactário de hospital de Itajaí, SC. Hig Aliment 2017; 31(272/273):56-61..

In this light, the present study sought to draft and to validate the content and appearance of a checklist of production processes infant formula, pediatric enteral diets, and expressed human milk to aid in the monitoring of the adoption of BP to be used in Food and Nutrition Units (UAN, in Portuguese) of hospitals that have lactaries.

Methods

This study was approved by the Research Ethics Committee of the Clementino Fraga Filho University Hospital (HUCFF), logged under CAAE number09414119.0.0000.5257, and presented the following stages: I) Drafting of the instrument; II) Validation of the instrument’s content by specialists; III) Validation of the appearance of the instrument by the target public. Figure 1 summarizes the drafting of the process and the validation of the instrument.

Drafting of the instrument

For this stage, a bibliographic survey was conducted in the database of the Brazilian Health Regulatory Agency (ANVISA) website, as well as in the databases of the Latin American and Caribbean Health Sciences (LILACS), Medical Literature Analysis and Retrieval System Online (MEDLINE), and Scientific Electronic Library Online (SCIELO).

In the first stage, the following health sciences DeCS/MeSHterms were used: “Checklist or Delphi Technique and Delphi Technique”; “Técnica Delfos” OR “Surveillance of Services” OR “Food Quality”; “Unidade de alimentação e nutrição hospitalar AND (collection:(“06-national/BR” OR “05-specialized”) OR db:(“LILACS” OR “MEDLINE”)) AND ( collection:(“06-national”))”.

The second stage filtered studies conducted as of 2006 (considering this year, since there were technical manuals drafted by the World Health Organization (WHO) for infant formulas), in Portuguese, English, and Spanish. The study presented the following descriptors: (1) in Portuguese: lactário, fórmulas lácteas infantis, fórmulas infantis, dieta enteral, terapia enteral, controle higienicossanitário de unidades de alimentação e nutrição, serviços de alimentação e nutrição em unidades hospitalares, controle microbiológico de fórmulas infantis (lactary, infant milk formulas, infant formulas, enteral diet, enteral therapy, hygiene sanitary control of the UAN, food and nutrition services in hospital units, microbiological control of infant formulas); (2) in English: food quality, food safety service, powdered infant formula, powdered milk, breastfeeding, human milk bank, surveillance of services; (3) in Spanish: fórmulas lácteas artificiales, lactancia maternal, fórmulas lácteas infantiles, lactantes (artificial milk formulas, breast milk, infant milk formulas, breastfeeding).Our work selected the studies that evaluated and/or validated and applied instruments of hygiene sanitary inspection and/or microbiological analysis and/or the analysis of dangers and critical points of control in food and nutrition services of the hospital sectors included in UAN and/or lactaries, human milk banks, commercial sectors (for example, restaurants, delicatessens, among others), and school sectors.

The checklist of production processes infant formula, pediatric enteral diets, and expressed human milk was drafted and referenced considering the bibliographic survey conducted. To structure the instrument, the ANVISA Decrees RDC nº 216/2004¹⁸18 Brasil. Resolução RDC n° 216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set.; RDC nº 63/2000¹⁹19 Brasil. Resolução RDC nº 63, de 6 de julho de 2000. Dispõe sobre Regulamento Técnico para Terapia de Nutrição Enteral. Diário Oficial da União 2000; 6 jul.; and RDC nº 275/2002²⁰20 Brasil. Resolução RDC nº 275, de 21 de outubro de 2002. Dispõe sobre o Regulamento Técnico de Procedimentos Operacionais Padronizados aplicados aos Estabelecimentos Produtores/Industrializadores de Alimentos e a Lista de Verificação das Boas Práticas de Fabricação em Estabelecimentos Produtores/Industrializadores de Alimentos. Diário Oficial da União 2002; 21 out. were used, which treat the hygiene sanitary control and the best food handling practices in UAN¹⁸18 Brasil. Resolução RDC n° 216, de 15 de setembro de 2004. Dispõe sobre Regulamento Técnico de Boas Práticas para Serviços de Alimentação. Diário Oficial da União 2004; 15 set., enteral nutritional therapy¹⁹19 Brasil. Resolução RDC nº 63, de 6 de julho de 2000. Dispõe sobre Regulamento Técnico para Terapia de Nutrição Enteral. Diário Oficial da União 2000; 6 jul. and industrial food processing²⁰20 Brasil. Resolução RDC nº 275, de 21 de outubro de 2002. Dispõe sobre o Regulamento Técnico de Procedimentos Operacionais Padronizados aplicados aos Estabelecimentos Produtores/Industrializadores de Alimentos e a Lista de Verificação das Boas Práticas de Fabricação em Estabelecimentos Produtores/Industrializadores de Alimentos. Diário Oficial da União 2002; 21 out., and the checklist of the study entitled “quality management in the preparation of enteral diets and infant formulas in the lactary of a university hospital”¹⁶16 Siqueira DR. Gestão da qualidade no preparo de dietas enterais e fórmulas infantis no lactário de um hospital universitário [dissertação]. Rio de Janeiro: Universidade Federal do Rio de Janeiro; 2016..

Validation of the instrument’s content by specialists

This stage used the Delphi technique and consisted of the following stages: Selection of specialists; Validation of the content; Analysis of the reproducibility and reliability of the specialists’ answers²¹21 Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000; 32(4):1008-1015.^,2222 Silva RF, Tanaka OY. Técnica Delphi: identificar a competência genérica exigida para médicos e enfermeiros que atuam em uma atenção primária à saúde. Rev Esc Enferm USP 1999; 33(3):207-216..

Selection of the specialists

The following criteria were adopted to create the panel of specialists: (1) Have at least three years of experience in lactaries; (2) Have a lato sensu and/or stricto sensu specialization and/or a Master’s in Business Administration (MBA); (3) Have experience in drafting instruments for lactaries, having experience or not in the validation of instruments for hospital UANs. To achieve this, the lattes curriculum vitae of the specialists was analyzed, and contact was made with the coordination department of the hospital lactaries to gain access to the e-mail and/or telephone number of the nutritionists who worked in this sector.

Validation of content

The specialists who agreed to participate on the panel were advised to sign the free and informed consent form. An e-mail was sent with the instructions to validate the content and checklist of production processes infant formula, pediatric enteral diets, and expressed human milk. In addition, a Google Drive file was made available with free access to the bibliography used in the write-up of the instrument. The specialists were advised to evaluate the content of the instrument considering the following aspects: (1) “If the item was expressed in a logical and intuitive manner”; (2) “If the item was described with the correct semantics (grammar and vocabulary)”; (3) “If the item contains current nomenclatures”; (4) “If the item was described in a clear and intelligible manner”; (5) “If the item contained relevant information that aided in the application of the instrument”; and (6) “If the item measured what was supposed to be evaluated - the hygiene sanitary conditions in the lactary”. To achieve this, the 5-point Likert scale was used: (1) Strongly disagree; (2) Disagree; (3) Neutral; (4) Agree; (5) Strongly agree²³23 R Likert. A technique for the measurement of attitudes. Arch Psychol 1932; 140(22):5-55.. The item classified by the specialists as “strongly disagree”, “disagree”, or “neutral” were changes, that is, for each subsequent round there was a verification of the results of the scale, and these items were re-written or excluded. The instrument was later re-sent with new structures and content for re-evaluation, according to that set forth in the Delphi technique. A space was made available in which the specialists could criticize and make suggestions, when necessary. However, it was requested that, together with the comments/suggestions, the specialists also informed the bibliography (when possible) on which the inclusion or exclusion was based.

The answers were tabulated in the Software Microsoft Excel® during three rounds of this study. In the final round, the specialists were informed by e-mail that the content of the instrument had been validated. The agreement among the specialists was measured by the Content Validity Index (CVI), considering the item validated when it presented an agreement of above 0.80 or 80% among the specialists²⁴24 Wynd CA, Schaefer MA. The osteoporosis risk assessment tool: Establishing content validity through a panel of experts. Appl Nurs Res 2002; 15(3):184-188., according to the following equation²⁵25 Tilden VP, Nelson CA, May BA. Use of qualitative methods to enhance content validity. Nurs Res 1990; 39(3):172-175.^,2626 Guillemin F. Cross-cultural adaptation and validation of heatth status measures. Scand J Rheumatol 1995; 24(2):61-63..

Equation 1:

Analysis of the reproducibility and reliability of the specialists’ answers

The reliability (internal consistency) of the instrument was measured using the statistical tests: Intraclass Correlation Coefficient (ICC) and Cronbach’s Alpha Coefficient, which evaluated the homogeneity of the obtained answers²⁷27 Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966; 19(1):3-11.. The following classification was adopted: ICC greater than or equal to 0.75 indicated an excellent reliability; between 0.4 and 0.74 indicated a satisfactory reliability; and less than 0.4 indicated a weak reliability²⁷27 Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966; 19(1):3-11.. For the Cronbach’s Alpha Coefficient, it was suggested that α should be greater than or equal to 0.7²⁸28 Bland JM, Altman DG. Statistics notes: Cronbach's alpha. Br Med J 1977; 314(7080):572.. For the data analysis, the Statistical Package for the Social Sciences, version 25 (IBM SPSS), was used.

Validation of the appearance of the instrument by the target public

The selection of the target public for whom this instrument was created, in this case, the nutritionists who work in the lactary, followed by the application of the instrument with the validated content, validated the instrument’s appearance and analyzed the reproducibility and reliability of the nutritionists’ answers.

Selection of the target public

The following selection criteria were used: (1) nutritionist with latu sensu, stricto sensu specialization or MBA; (2) practical experience in the management of hospital lactaries; (3) experience in the validation of hygiene sanitary control instruments in the hospital UAN; (4) experience in drafting quality control instruments in hospital lactaries.

Application of the instrument with the content validated by specialists

This application was conducted in the reference lactary of the city of Rio de Janeiro from September to October 2020. This institution was chosen, as it contemplated all of the production processes included in the instrument’s content, which is a crucial factor when evaluating reproducibility and reliability. The nutritionists that agreed to participate in this phase of the study signed the free and informed consent form. The instrument was applied by three nutritionists in the same lactary and under identical conditions of day and time. It was requested that the nutritionists avoid contact with one another in order to prevent any influence upon the answers.

Each item for the evaluation of the lactary could be classified as: “Conforms” (C) - when the item presented adequate hygiene sanitary conditions; “Does not conform” (NC) - when the item did not present adequate hygiene sanitary conditions; “Not Applicable” (NA); “Not Observed” (NO); along with a space for “Observations” (OBS), when necessary²⁹29 Mello AG, Carmo C, Leite S, Miguel M, Colares L. Elaboração, validação de conteúdo e da confiabilidade do instrumento para avaliação higiênico-sanitária de serviços de alimentação. Vig Sanit Debate 2014; 2(3):86-93.. The percentage of adequacy of the hygiene sanitary conditions (PACHS, in Portuguese) was calculated using the following formulas²⁹29 Mello AG, Carmo C, Leite S, Miguel M, Colares L. Elaboração, validação de conteúdo e da confiabilidade do instrumento para avaliação higiênico-sanitária de serviços de alimentação. Vig Sanit Debate 2014; 2(3):86-93.:

Equation 2:

Equation 3:

The lactary was considered to be of adequate hygiene sanitary conditions when the PACHS was >76%; partially adequate, when the PACHS was between 51% and 75%; and inadequate, when the PACHS<50%²⁷27 Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966; 19(1):3-11.^,2929 Mello AG, Carmo C, Leite S, Miguel M, Colares L. Elaboração, validação de conteúdo e da confiabilidade do instrumento para avaliação higiênico-sanitária de serviços de alimentação. Vig Sanit Debate 2014; 2(3):86-93..

From the results obtained by the PACHS calculations, the Kolmogorov-Smirnov statistical test was performed. It was observed that the data did not present a normal distribution. For this reason, the Kruskal-Wallis parametric statistics test was performed³⁰30 Mckight PE, Najab J. Kruskal-Wallis test. Nova Jersey: Corsini Encyclopedia of Psychology; 2010. to verify if there was a statistically significant difference among the analyses carried out by the specialists in the lactary. For all tests, the significance level was set at 5%.

Validation of the appearance of the instrument

The following aspects were analyzed: (1) “The header information is organized in a logical and intuitive manner”; (2) “The instructions to guide the application of the instrument are organized in a logical and intuitive manner”; (3) “The options to check the answers of the instrument are clear, objective, and easy to mark”; (4) “The items of the blocks drafted in the instrument are semantically correct (grammar and vocabulary)”; (5) “The graphic layout is presented in an organized, simple, and objective manner”; (6) “The size of the instrument is adequate and relevant enough to evaluate the hygiene sanitary conditions of the hospital lactaries”; (7) “The method to evaluate the calculation of the adequacy of the hygiene sanitary conditions of the hospital lactaries, based on the adaptation of Decree RDC nº 275/2002²⁰20 Brasil. Resolução RDC nº 275, de 21 de outubro de 2002. Dispõe sobre o Regulamento Técnico de Procedimentos Operacionais Padronizados aplicados aos Estabelecimentos Produtores/Industrializadores de Alimentos e a Lista de Verificação das Boas Práticas de Fabricação em Estabelecimentos Produtores/Industrializadores de Alimentos. Diário Oficial da União 2002; 21 out. and the scientific literature²⁹29 Mello AG, Carmo C, Leite S, Miguel M, Colares L. Elaboração, validação de conteúdo e da confiabilidade do instrumento para avaliação higiênico-sanitária de serviços de alimentação. Vig Sanit Debate 2014; 2(3):86-93. is conclusive and capable of guiding the professional in the proper correction measures when non-conformities are indicated”.

Each aspect analyzed in this study was classified by the nutritionists, using the 5-point Likert scale²³23 R Likert. A technique for the measurement of attitudes. Arch Psychol 1932; 140(22):5-55., and the items that were classified as “strongly disagree”, “disagree”, or “do not agree” were changed according to the contribution of the nutritionists. To evaluate the agreement among the nutritionists, the Appearance Validity Index (AVI) was used, with the formula adapted from the CVI²⁵25 Tilden VP, Nelson CA, May BA. Use of qualitative methods to enhance content validity. Nurs Res 1990; 39(3):172-175.^,2626 Guillemin F. Cross-cultural adaptation and validation of heatth status measures. Scand J Rheumatol 1995; 24(2):61-63. to perform the calculation, and was considered to be validated when an agreement of above 0.70 or 70% was reached.

Analysis of the reproducibility and reliability of the nutritionists’ answers

To verify the reliability (internal consistency) of the evaluations of the lactaries, the ICC²⁷27 Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966; 19(1):3-11. and the Cronbach’s Alpha²⁸28 Bland JM, Altman DG. Statistics notes: Cronbach's alpha. Br Med J 1977; 314(7080):572. Coefficient statistics tests were applied in order to evaluate the homogeneity of the answers, according to that described above. To analyze the data, the IBM SPSS Statistics software, version 25, was used.

Results and discussion

Drafting of the instrument

Of the 10 studies found, 7 were not eligible and only 3 were eligible; however, only one study from the search was used, as it proved to run in line with the proposal of the present study. Moreover, during the content validation process, one contribution study was sent to the specialists to reference the instrument.

Validation of the instrument’s content by specialists

Selection of specialists

The panel for the instrument’s content validation consisted of five specialists, of whom 60% (3) had a Master´s degree, 20% (1) a specialization, and 20% (1) an MBA. Regarding the work of the specialists, 40% (2) worked in public hospitals, 40% (2) in private hospitals, and 20% (1) conducted scientific and hospital consulting. All specialists had experience in the validation of hygiene sanitary control instruments in a hospital UAN and in the drafting of quality control instruments in hospital lactaries. The average time of experience in their areas of work was of 11 years, with a standard deviation (SD) of +6.96.

Content validation

The checklist of production processes infant formula, pediatric enteral diets, and expressed human milk presented the following structure: (1) Header - specifications of the lactary, (2) Instruction guide to completion instructions, and (3) 16 blocks with 225 evaluation items.

For the content validation of the header, two rounds of the Delphi technique were necessary, with the following adjustments: insertion of the volume of productions per day of enteral diets and expressed human milk, together with the list of equipment commonly used in the best practices of the production processes in hospital lactaries, according to that requested by the specialists. To guide completion instructions, two rounds of the Delphi technique were also required so that the content could be validated and the following information be included - the term “expressed human milk” included both pasteurized expressed human milk (LHOP, in Portuguese) and raw expressed human milk (LHOC, in Portuguese), which is exclusively from the mother to the child, according to that requested by the specialists.

The CVI measurements of the aspects that were evaluated by the specialists for the validation of the content of the header and the guide to completion instructions were of 0.88 (minimum 0.88 and maximum of 0.92), respectively, were: “Organized in a logical and intuitive manner”; “Described in a clear and intelligible manner”; “Aids in the application of the instrument”; “Drafted semantically correct (grammar and vocabulary)”.

For the validation of the content of the instrument’s items, three rounds of the Delphi technique were necessary. The first version of the instrument consisted of 236 evaluation items. After the content validation by the specialists, the instrument then contained 227 items.

In the first round, 37% (84) of the items had their content validated, 21 items were excluded, and 131 items did not reach an agreement and were changed in relation to the writing to be re-evaluation. In addition, according to the contributions of the specialists, 13 items were added and evaluated in the second round, according to that set forth by the Delphi technique.

In the second round of validation of the checklist of production processes infant formula, pediatric enteral diets, and expression human milk, 54% (123) of the items had their content validated; only 21 items did not reach an agreement among the specialists, with one item being excluded and the others modified. In the third round, 9% (20) of the items were re-evaluated and validated in their content.

The CVI measurement obtained by each block that made up the evaluated instrument can be observed in Table 1. The CVI values obtained in the present study ran in line with that proposed by scientific literature³¹31 Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health 1997; 20(3):269-274., which considered the instrument with the validated content when the agreement among the specialists was above 80% (0.80). In the validation studies, the hygiene sanitary control instruments in UAN³²32 Wendisch C. Avaliação da Qualidade de Unidades de Alimentação e Nutrição (UAN) Hospitalares: construção de um instrumento [dissertação]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca; 2010.

33 Ceniccola GD. Validação de conteúdo de instrumento para avaliar os procedimentos da nutrição enteral em ambiente hospitalar [dissertação]. Brasília: Faculdade de Ciências da Saúde; 2013.

34 Camargo RGM, Caivano S, Bandoni DH, Domene SMA. Alimentação saudável no ambiente escolar: consenso entre especialistas. Rev Nutr 2016; 29(6):809-819.

35 Viterbo LMF. Desenvolvimento de instrumento quantitativo para inspeção sanitária em serviços de alimentação [dissertação]. Salvador: Universidade Católica do Salvador; 2017.

36 Colares LGT, Figueiredo VDO, Ferreira AA, Oliveira AGM. Good environmental practices check list for food services: elaboration, content validation and inter-rater reliability. Braz J Food Technol 2018; 21:e2017066.^-3737 Ruas LP. Construção e validação de instrumento de avaliação do gerenciamento da qualidade da água em estabelecimentos hospitalares [dissertação]. Diamantina: Universidade Federal dos Vales do Jequitinhonha e Mucuri; 2019. and the collective food management³⁸38 Freitas JF. Evidências de validade em um instrumento de avaliação de competências profissionais na gestão prática do nutricionista na gestão da alimentação coletiva [tese]. Natal: Universidade Federal do Rio Grande do Norte; 2020. were validated with a CVI of greater than or equal to 0.80 for the analyzed aspects, and corroborate with the results from the present study.

Analysis of the reproducibility and reliability of the answers given by the specialists

The answers from the specialists showed homogeneity, since the items that made up the instrument contained an ICC>0.80, and 90% (203) of the items showed a Cronbach’s Alpha >0.80. These results demonstrate an excellent correlation and reliability that was nearly perfect, according to the classifications described by scientific literature²⁷27 Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966; 19(1):3-11.^,2828 Bland JM, Altman DG. Statistics notes: Cronbach's alpha. Br Med J 1977; 314(7080):572.^,3939 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33(1):159-174.. Regarding the 10% (22) of the items, which showed a Cronbach’s Alpha value of below 0.70, these values did not affect the reliability of the scale. The present study opted not to exclude these items as they were important in the evaluation of the hygiene sanitary conditions in hospital lactaries. The decision not to exclude the instrument items with Cronbach’s Alpha values below 0.70, without interference in the internal consistency, was also observed in another study⁴⁰40 Venegas ME, Alvarado OS, Elizondo NR, Carrillo KS. Validação do construto e da confiabilidade de uma escala de inteligência emocional aplicada a estudantes de enfermagem. Rev Latinoam Enferm 2015; 23(1):139-147., in which the reliability of the items from the instrument entitled “scale of emotional intelligence applied to nursing students” was evaluated in the aspects of “perception”, “understanding”, and “regulation”. The results that were obtained in the present study were similar to another study⁴¹41 Santos TEA. Construção e validação de instrumento para avaliação da utilidade da "Autoavaliação para Melhoria do Acesso e da Qualidade de Atenção Básica - AMAQ" [dissertação]. Rio Grande do Norte: Universidade Federal do Rio Grande do Norte; 2019., which evaluated and validated the content of an instrument entitled the “evaluation of the use of self-assessment to improve access to and quality of primary health care” (AMAQ) and achieved an internal consistency of the Cronbach’s Alpha value of greater than 0.80 among the 7 specialists for the evaluation of the aspects of “relevance”, “representativity”, and “clarity”.

Validation of the appearance of the instrument for the target public

Selection of the target public

The checklist of production processes infant formula, pediatric enteral diets, and expressed human milk with its validated content was submitted to the process of the validation of the instrument’s appearance for the target public. This stage counted on the participation of three nutritionists who have practical experience in lactary management, of whom, 80% (2) had a Master´s degree and 20% (1) has a latu sensu specialization; 33.33% (3) were employed in the lactary in which the instrument was applied, and were, respectively, government employees and an outsourced employee, while 33.33% (1) was classified as a scientific and hospital consultant. Moreover, all of the nutritionists had experience in the validation of instruments, and 80% (2) had experience in drafting quality control instruments for hospital lactaries. The average time of experience in the area of work was of 5 years (SD+3.77).

Validation of appearance

The results of the validation of appearance, which were performed by three nutritionists during the first application of the instrument with validated content (full version), were collected in the reference lactary of Rio de Janeiro and presented an agreement of above 0.70 for the AVI of the five aspects, which were, respectively, “The information of the header was organized in a logical and intuitive manner” (0.87); “the instruction written for the application of the instrument were organized in a logical and intuitive manner” (0.73); “the options to check the answers of the instrument were presented in a manner that was clear, objective, and easy to mark” (0.80); “the items written in the instrument were semantically correct (grammar and vocabulary)” (0.80); “the graphic layout(components of the appearance of the instrument: visual, page formatting, margins, provision of the items) was provided in an organized, simple, and objective manner” (0.87).

However, the aspects of “Instrument size (number of items)” and “Method of evaluation of the percentage of adequacy of the hygiene sanitary conditions” showed an agreement of below 0.70. The nutritionists found the instrument to be extensive and suggested that the instrument be divided into two versions:

1. Routine version - for daily supervision, which consisted of 10 items distributed in 9 blocks, as follows: block (1) “Hygiene health and conduct of handlers”; block (2) “Control of the preparation process for the hygienization of utensils and equipment”; block (3) “Hygienization of baby bottles, nipples, and accessories”; block (4) “Sterilization of baby bottles, nipples, and accessories”; block (5) “Handling of infant formulas, enteral diets, and expressed human milk”; block (6) “Procedures for final heating of infant formulas”; block (7) “Disposal of infant formulas, enteral diets, and expressed human milk”; block (8) “Procedures for the final heating of the infant formulas, enteral diets, and expressed human milk”; block (9) “Distribution of infant formulas, enteral diets, and expressed human milk”.

2. Management version - to be used in the supervision done every 15 days or monthly, which consisted of 107 items distributed in 7 blocks, as follows: block (1) “Receiving of infant formulas, enteral diets, complementary inputs, and LHO”; block (2) “Storage of infant formulas, enteral diets, complementary inputs, and LHO”; block (3) “Dressing/changing room”; block (4) “Room for the preparation of the infant formula, enteral diet, and LHO”; block (5) “Climatization”; block (6) “Water supply”; block (7) “Control of vectors and plagues”.

These adjustments represented an important contribution in this stage of the study, as they minimized the number of items that were redundant and facilitated the application of these in the lactary’s daily routine.

During the calculation of the PACHS, it was observed that the appearance of the formula proposed by Decree RDC nº 275/2002²⁰20 Brasil. Resolução RDC nº 275, de 21 de outubro de 2002. Dispõe sobre o Regulamento Técnico de Procedimentos Operacionais Padronizados aplicados aos Estabelecimentos Produtores/Industrializadores de Alimentos e a Lista de Verificação das Boas Práticas de Fabricação em Estabelecimentos Produtores/Industrializadores de Alimentos. Diário Oficial da União 2002; 21 out. as well as by the scientific literature²⁹29 Mello AG, Carmo C, Leite S, Miguel M, Colares L. Elaboração, validação de conteúdo e da confiabilidade do instrumento para avaliação higiênico-sanitária de serviços de alimentação. Vig Sanit Debate 2014; 2(3):86-93., generated overestimated and underestimated results. Thus, the nutritionists suggested the adaptation of this. In this sense, the formulas described in equations 4 and 5 were adopted, in which the items classified as “not applicable” and the items classified as “not observed” were subtracted from the total number of items.

Equation 4:

Equation 5:

Where: NA = not applicable; NO = not observed.

After the adjustments, the instruments were submitted to a new stage for the validation of the appearance of the instrument by the target public, showing an AVI=0.93 both for the size of the instrument and for the method of PACHS evaluation; therefore, the instrument was considered to have a validated appearance.

The instrument with the content and appearance validated was once again applied in the reference lactary. Tables 2 and 3 show, respectively, the results of the lactary evaluations in which the routine version was used, as well as the management version of the checklist of production processes infant formula, pediatric enteral diets, and expressed human milk. It is important to highlight that in the instrument’s routine version, the block 6: “procedures of final heating of the infant formulas | autoclave process” was not evaluated, since the reference lactary did not use this procedure to heat infant formulas, but rather used a water bath at 65ºC for 15 minutes, which would affirm the conformity in the evaluation of the best practices of the handling process.

When applying the management version of production processes infant formula, pediatric enteral diets, and expressed human milk, it was observed that one item presented non-conformity - the lack of an exclusive dressing room for the lactary employees -, which did not meet the legal standards²⁰20 Brasil. Resolução RDC nº 275, de 21 de outubro de 2002. Dispõe sobre o Regulamento Técnico de Procedimentos Operacionais Padronizados aplicados aos Estabelecimentos Produtores/Industrializadores de Alimentos e a Lista de Verificação das Boas Práticas de Fabricação em Estabelecimentos Produtores/Industrializadores de Alimentos. Diário Oficial da União 2002; 21 out., which corroborates with another similar study⁴²42 Reis HF, Flavio EF, Guimarães RSP. Avaliação das condições higienicossanitárias de uma unidade de alimentação e nutrição hospitalar de Montes Claros, MG. RCU 2015; 17(2):68-81. that evaluated the hygiene sanitary conditions of a hospital UAN, where the lack of an exclusive dressing room for the food handlers was identified.

In block (4): “Room to prepare the infant formula, enteral diet, and LHO”, the subitem “Environment and safety” were also evaluated as non-conforming, in the item related to the signaling of the emergency exit in the lactary, that is, the location had no signaling and thus did not meet the legal standards²⁰20 Brasil. Resolução RDC nº 275, de 21 de outubro de 2002. Dispõe sobre o Regulamento Técnico de Procedimentos Operacionais Padronizados aplicados aos Estabelecimentos Produtores/Industrializadores de Alimentos e a Lista de Verificação das Boas Práticas de Fabricação em Estabelecimentos Produtores/Industrializadores de Alimentos. Diário Oficial da União 2002; 21 out.. This result was similar to a study⁴³43 Borges, CMD, Moura C, Oliveira FM, Costa, GM, Veloso JBR, Faria LF, Oliveira MER. Fluxograma de operacionalização na produção de fórmulas infantis e seus pontos críticos de controle em lactário no município de Uberlândia (MG). E-RAC 2018; 8(1):1-24. which analyzed the operational production process of infant formulas and their critical points of control, and highlighted the lack of the signaling of the emergency exit in the lactary of a public hospital in Brazil.

Analysis of the reproducibility and reliability of the answers given by the nutritionists

The results presented in Tables 2 and 3 show that the instrument presented reproducibility, that is, it proposed what it wanted to measure in conditions that were identical to the analysis, agreement, and homogeneity originating from the same distribution among the nutritionists during the evaluation of the hygiene sanitary conditions in the reference lactary. No statistically significant difference (p-value>0.05) was found among the answers given by the nutritionists regarding the lactary, the PACHS calculation of the blocks, and the lactary as a whole.

Moreover, the results from Tables 4 and 5 reinforce that the checklist of production processes infant formula, pediatric enteral diets, and expressed human milk showed reproducibility, reliability, and internal consistency, and that 81% (13) of the blocks of this instrument presented Cronbach’s Alpha and ICC values of greater than 0.80. According to scientific literature⁴⁴44 Streiner D, Norman G. Health measurement scales. Oxford: Oxford University Press; 2015., for the instrument to present reliability, it was suggested that the ICC be greater or equal to 0.75.

Some blocks of the checklist of production processes infant formula, pediatric enteral diets, and expressed human milk were not evaluated as regards the internal consistency of the instrument. These include block (4) “Sterilization of the baby bottles, nipples, and accessories” in the routine version, and block (3) “Dressing/changing room” and block (7) “Control of vectors and plagues” in the management version, since variance was not observed during the nutritionists’ evaluation, that is, a similar evaluation was given for the cited blocks and presented no statistically significant difference.

Conclusion

The checklist of production processes infant formula, pediatric enteral diets, and expressed human milk had the content and appearance validated; provided reproducibility, reliability, and internal consistency; and showed that it attends to the goal for which it was drafted, that is, to evaluate the hygiene sanitary conditions of the hospital lactaries.

The panel of specialists who worked on the validation of the content played an important role in defining the content of the instrument, in such a way that they measured exactly what they proposed to measure.

The nutritionists who worked on the validation of the appearance helped make the instrument more functional in order to evaluate specific situations during the supervision of the production process of the pediatric diets in a lactary. At the end of the process, two versions of the checklist of production processes infant formula, pediatric enteral diets, and expressed human milk were formulated: the routine version, which contemplated the daily supervision blocks, and the management version, which contemplated the blocks of supervision that occurred every 15 days or monthly.

Therefore, the present study was pioneer in the process of drafting and validating a checklist of production processes the infant formula, pediatric enteral diets, and expressed human milk, which can optimize the service of the professional of the area and provide guidance so that future innovations can be drafted, standardized, and consolidated in the realm of hospital lactaries.

References

Publication Dates

History