ABSTRACT

OBJECTIVE

To validate an instrument designed to assess health promotion in the school environment.

METHODS

A questionnaire, based on guidelines from the World Health Organization and in line with the Brazilian school health context, was developed to validate the research instrument. There were 60 items in the instrument that included 40 questions for the school manager and 20 items with direct observations made by the interviewer. The items’ content validation was performed using the Delphi technique, with the instrument being applied in 53 schools from two medium-sized cities in the South region of Brazil. Reliability (Cronbach’s alpha and split-half) and validity (principal component analysis) analyses were performed.

RESULTS

The final instrument remained composed of 28 items, distributed into three dimensions: pedagogical, structural and relational. The resulting components showed good factorial loads (> 0.4) and acceptable reliability (> 0.6) for most items. The pedagogical dimension identifies educational activities regarding drugs and sexuality, violence and prejudice, auto care and peace and quality of life. The structural dimension is comprised of access, sanitary structure, and conservation and equipment. The relational dimension includes relationships within the school and with the community.

CONCLUSIONS

The proposed instrument presents satisfactory validity and reliability values, which include aspects relevant to promote health in schools. Its use allows the description of the health promotion conditions to which students from each educational institution are exposed. Because this instrument includes items directly observed by the investigator, it should only be used during periods when there are full and regular activities at the school in question.

School Health; Health Promotion; Program Evaluation; Evaluation of Research Programs and Tools; Reproducibility of Results; Validation Studies

INTRODUCTION

Since the 1980s, which saw the signing of the Ottawa Charter for health promotion, schools have been seen as suitable environments for promoting health. This theme has been further expanded on several fronts, such as on the Jakarta Declaration (1997), in the creation of the European Network of Health Promoting Schools²⁹29. Yoshimura N, Jimba M, Poudel KC, Chanthavisouk C, Iwamoto A, Phommasack B et al. Health promoting schools in urban, semi-urban and rural Lao PDR. Health Promot Int. 2009;24(2):166-76. DOI:10.1093/heapro/dap004 and on initiatives involving all continents²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,1717. Lee A, St Leger L, Cheng FFK. The status of health-promoting schools in Hong Kong and implications for further development. Health Promot Int. 2007;22(4):316-26. DOI:10.1093/heapro/dam029^,2626. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755. To strengthen disease prevention and health promotion in Brazilian schools, the Brazilian Government instituted the Programa Saúde na Escola (PSE – School Health Program)¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005, along with actions to promote healthy eating, a culture of peace, human rights, reduced consumption of alcohol and other drugs and sexual and reproductive health, as well as anthropometric, oral and visual evaluations and periodical vaccinations.

These initiatives have been recognized as beneficial, but the instruments to assess them have been debated¹⁸18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2828. Warwick I, Aggleton P, Chase E, Schagen S, Blenkinsop S, Schagen I et al. Evaluating healthy schools: perceptions of impact among school-based respondents. Health Educ Res. 2005;20(6):697-708. DOI:10.1093/her/cyh024. The available evaluations are specific, for nutritional health⁵5. Coleman KJ, Shordon M, Caparosa SL, Pomichowski ME, Dzewaltowski DA. The healthy options for nutrition environments in schools (Healthy ONES) group randomized trial: using implementation models to change nutrition policy and environments in low income schools. Int J Behav Nutr Phys Act. 2012;9:80. DOI:10.1186/1479-5868-9-80^,99. Graham D, Appleton S, Rush E, McLennan S, Reed P, Simmons D. Increasing activity and improving nutrition through a schools-based programme: Project Energize. 1. Design, programme, randomisation and evaluation methodology. Public Health Nutr. 2008;11(10):1076-84. DOI:10.1017/S136898000700153X, mental health²⁷27. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, substance use²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,2222. Page RM, Danielson M. Multi-country, cross-national comparison of youth tobacco use: findings from global school-based health surveys. Addict Behav. 2011;36(5):470-8. DOI:10.1016/j.addbeh.2011.01.008 or physical activity¹⁴14. Leatherdale S, Manske S, Faulkner G, Arbour K, Bredin C. A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act. 2010;7:6. DOI:10.1186/1479-5868-7-6. Only a small number of studies measure environmental conditions⁴4. Brooks FM, Magnusson J, Spencer N, Morgan A. Adolescent multiple risk behaviour: an asset approach to the role of family, school and community. J Public Health (Bangkok). 2012;34(suppl 1):i48-56. DOI:10.1093/pubmed/fds001. Thus, more comprehensive options to evaluate health-promoting practices in school have emerged, which include using the schools’ environments and their conditions as an element for analysis, while using various forms of data collection and adapting the experiences of others to the needs and characteristics of their locations or regions¹⁵15. Lee A, Cheng FFK, St Leger L. Evaluating health-promoting schools in Hong Kong: development of a framework. Health Promot Int. 2005;20(2):177-86. DOI:10.1093/heapro/dah607^,1818. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2626. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755. The validation of these instruments is still incipient and limited¹⁸18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2828. Warwick I, Aggleton P, Chase E, Schagen S, Blenkinsop S, Schagen I et al. Evaluating healthy schools: perceptions of impact among school-based respondents. Health Educ Res. 2005;20(6):697-708. DOI:10.1093/her/cyh024.

The objective of this article was to validate the instrument whose purpose is to evaluate health promotion in the school environment.

METHODS

To select the items that would be included in the questionnaire, we reviewed the literature in the following databases: Web of Science, PubMed, SciELO, EBSCO Information Services, Psycoinfo, Educational Resources Information Center (Eric), Eric Proquest, Science Direct and the Virtual Health Library (VHL), using evaluation AND health promotion AND school environment as descriptors. We selected empirical articles and literature reviews, published between April 2004 and April 2013, which considered the school environment as a study variable, as well as documents from the public domain such as public policy protocols and international guidelines. The following were excluded: clinical trials and observational studies whose study populations were schools but did not evaluate the school context; articles written in languages other than English, Spanish, Portuguese or French; and publications that did not address the topic of interest. The databases were consulted in 2013, and we read the abstracts to determine whether they would be read in full and whether they would be included in the study or not. All the selected articles were read in two steps: recognition of the design and confirmation of inclusion; and selection of the items regarding health promotion in school to be included in the instrument. We found 436 empirical articles or reviews, of which 353 were excluded (243 for not addressing the topic in question, 81 for being publications that only used schoolchildren as the study population, four for not being available in the aforementioned languages and 25 for being duplicates). From the total selected, after reviewing and reading all their abstracts, 83 articles were read in full. In addition, 14 documents from the public domain were used (ordinances and resolutions from International Bodies, Education or Health Ministries, the Public Ministry and State Health or Education Secretariats).

For the purposes of analyzing the selected articles and other documents, 40 questioning items directed towards the school’s manager (questioned) and 20 items obtained through direct observation (observed) were created. The 60 items were grouped into three dimensions, according to their main characteristic: pedagogical, structural and relational (Tables 1 to 5). The pedagogical dimension was made up of 14 items and looked at themes and activities related to the learning process, from a perspective of attaining healthy environments. Items considered relevant and feasible to be analyzed individually or cross-sectionally were: healthy eating and physical activity¹⁴14. Leatherdale S, Manske S, Faulkner G, Arbour K, Bredin C. A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act. 2010;7:6. DOI:10.1186/1479-5868-7-6, personal hygiene care²⁶26. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, sexual and reproductive health⁸8. Gerouki M. Sexuality and relationships education in the Greek primary schools: see no evil, hear no evil, speak no evil. Sex Education. 2007;7(1):81-100. DOI:10.1080/14681810601134710; prevention of licit and illicit drug use²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,1919. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2222. Page RM, Danielson M. Multi-country, cross-national comparison of youth tobacco use: findings from global school-based health surveys. Addict Behav. 2011;36(5):470-8. DOI:10.1016/j.addbeh.2011.01.008; culture of peace and human rights and personal interaction skills in terms of inclusion, respect, initiative and tolerance¹⁶16. Lee A, Cheng FFK, Yuen H, Ho M, Lo A, Fung Y et al. Achieving good standards in health promoting schools: preliminary analysis one year after the implementation of the Hong Kong Healthy Schools Award scheme. Public Health. 2007;121(10):752-60. DOI:10.1016/j.puhe.2007.01.014^,1919. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2525. Ruotti C. Violência em meio escolar: fatos e representações na produção da realidade. Educ Pesq. 2010;36(1):339-55. DOI:10.1590/S1517-97022010000100010.

The structural dimension included 33 items. We considered resources in terms of physical and installed capacity, suitability of space for educational activities¹¹11. Jurg ME, Kremers SPJ, Candel MJJM, Van der Wal MF, Meij JSBD. A controlled trial of a school-based environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promot Int. 2006;21(4):320-30. DOI:10.1093/heapro/dal032^,1414. Leatherdale S, Manske S, Faulkner G, Arbour K, Bredin C. A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act. 2010;7:6. DOI:10.1186/1479-5868-7-6, and appropriateness of personal safety and sanitary conditions¹⁷17. Lee A, St Leger L, Cheng FFK. The status of health-promoting schools in Hong Kong and implications for further development. Health Promot Int. 2007;22(4):316-26. DOI:10.1093/heapro/dam029. We also included items regarding the relationship with the surrounding community and school partnerships that make acquiring resources for health promotion and disease prevention possible¹³13. Kirby DB, Baumler E, Coyle KK, Basen-Engquist K, Parcel GS, Harrist R et al. The “Safer Choices” intervention: its impact on the sexual behaviors of different subgroups of high school students. J Adolesc Health. 2004;35(6):442-52. DOI:10.1016/S1054-139X(04)00071-0^,1414. Leatherdale S, Manske S, Faulkner G, Arbour K, Bredin C. A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act. 2010;7:6. DOI:10.1186/1479-5868-7-6.

The relational dimension, with 13 items, linked the conditions that were considered necessary to construct an ethos to promote a pleasant environment from a social point of view, which focused on the relationships and the conditions that had been established within the school community. Aspects regarding the relationship between students, teachers and the community, the presence of violence, actions to promote protagonism and respect for rules of coexistence were also considered³3. Bond L, Patton G, Glover S, Carlin JB, Butler H, Thomas L et al. The Gatehouse Project: can a multilevel school intervention affect emotional wellbeing and health risk behaviours? J Epidemiol Community Health. 2004;58(12):997-1003. DOI:10.1136/jech.2003.009449^,1919. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755^,2929. Yoshimura N, Jimba M, Poudel KC, Chanthavisouk C, Iwamoto A, Phommasack B et al. Health promoting schools in urban, semi-urban and rural Lao PDR. Health Promot Int. 2009;24(2):166-76. DOI:10.1093/heapro/dap004.

Each of the 60 items was designed in the form of a question, which corresponded to dichotomous categorical variables (yes; no). As some items involve direct observation, the use of the instrument requires that the person applying it visit the school and verify whether or not the proposed condition exists.

The first proposed version was submitted to content validation using the Delphi method¹⁰10. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32:1008-15.^,1212. Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205-12. DOI:10.1111/j.1365-2648.2006.03716.x^,2424. Piola SF, Vianna SM, Vivas-Consuelo D. Estudo Delphi: atores sociais e tendências do sistema de saúde brasileiro. Cad Saude Publica. 2002;18 supl:S181-90. DOI:10.1590/S0102-311X2002000700018. We invited five specialists in the areas of education and health, contacted by electronic means or in person. All were requested to sign a form of free and informed consent. All received the instrument so they could give their opinions as to whether each question and guidance item was properly designed (yes; no). The aim of this procedure was to survey the understandability of the questionnaire, so that the respondents could support their answers through justifications and suggestions to improve the text. After the first evaluation by means of the Delphi method, among the 61 sent questions (60 items from the instrument and one instruction), the agreement among the evaluators was greater than 70.0% for all the items. None of the questions were dismissed and a second evaluation was performed with only seven questions receiving suggestions or criticism. A 100% agreement, was, then, achieved, which meant that a third evaluation was not required.

The redesigned instrument was presented to 55 school managers from two medium-sized cities (of around 70,000 inhabitants) in the South region of Brazil. It was requested that the schools’ director or another member of the management team participate by completing the questionnaire. Each institution defined who would be its responding representative. Two private schools from the same city refused to participate in the research. We visited 31 schools in the first city and 22 in the second, totaling 53 interviews. The questionnaires were applied by three researchers following their training and after they had applied a pilot study in four schools from a different city with similar characteristics.

The data were inputted into SPSS 20.0 statistical software and submitted to descriptive analysis. Some items of the questionnaire were excluded during the successive analysis stages according to the following criteria: questions with answers from, at most, two schools (among the 53 visited) in any of its categories, because the low frequency hindered its belonging to any construct; items that, with their exclusion, increased the internal consistency of the scale and items with weak loads (< 0.3) on more than one factor (not belonging to any factor).

Each dimension proposed in the literature was subjected to factorial analysis using principal component analysis. The adequacy of the data from the sample was evaluated using the Kaiser-Meyer-Olkin (KMO) test, whose recommended value must exceed 0.6, and the Bartlett’s sphericity test in which a specific statistical significance should be reached (p < 0.05). The number of factors to be extracted from each dimension was defined using open criteria, i.e., only those with eigenvalues above 1, done to identify the factors that contribute to the variance in the original variables. To minimize the number of variables with high loads in each factor, we used the Varimax rotation which obtained the representatives components from the instrument’s underlying dimensions. Values greater than or equal to 0.3 for the values of the commonalities were considered acceptable, which are understood as the proportions of the variances for each variable included in the analysis and are explained by the extracted components.

To ensure that each item corresponded to the construct’s underlying structure, a factorial load criteria equal to or greater than 0.4 was considered so that the item belonged to the construct. Despite the indication that items with a factorial load higher than one factor were excluded⁷7. Figueiredo Filho DB, Silva Júnior JA. Visão além do alcance: uma introdução à análise fatorial. Opin Publica. 2010;16(1):160-85. DOI:10.1590/S0104-62762010000100007, we allocated it where the item would contribute more theoretically and significantly to the final cluster of a factor.

We used Cronbach’s alpha to evaluate internal consistency, which evaluates how well a cluster of items unidimensionally measures the latent construct proposed by the scale. Values equal to or greater than 0.6 were considered acceptable. The split-half reliability was also tested, which is a measure of consistency in which the sample is divided into two and the test scores for each half are compared with each other. If the results are similar, it is believed that the same construct is being measured in the two halves. Minimum values of 0.6 are acceptable for each dimension¹1. Almeida LS, Freire T. Metodologia da investigação em psicologia e educação. 4 ed. Braga: Psiquilibrios; 2007.^.

This study was approved by the Research Ethics Committee at the Universidade do Vale do Rio dos Sinos (Process 025/2013). Only after making contact with the municipal and state bodies responsible for the schools were their respective managers invited to participate. The interviews were only performed after the free and informed consent forms were read and signed. The schools’ identifications were recorded in the instrument, with the confidentiality of data being maintained.

RESULTS

Twenty-eight questions remained on the instrument after the construct’s validation analysis. Tables 1, 2 and 3 show the results of the principal components analysis and the internal consistency conducted for each of the dimensions proposed by the literature.

The requirements for factor analysis were achieved with a KMO above 0.6 in the three dimensions and a statistically significant Bartlett’s sphericity test (p < 0.001). The commonalities for most of the items also reached desirable values close to or greater than 0.4. During the analysis of internal consistency for dimensions and factors, the majority of the analyses showed satisfactory results for a homogeneous construct (above 0.6).

Table 1 presents the pedagogical dimension, which maintained a total of 13 items. The KMO was equal to 0.705 and Bartlett’s sphericity reached statistical significance (p < 0.001). The principal component analysis showed four components: drugs and sexuality (explaining 37.2% of the variance); violence and prejudice (16.9% of the variance); self-care (10.5% of the variance); peace and quality of life (9.4% of the variance), all with eigenvalues greater than 1. The split-half test value was 0.856 for the dimension. Cronbach’s Alpha was above 0.7 for the extracted factors, except for the component named peace and quality of life, which had a value of 0.61. The commonality (h²) only produced values above the recommended minimum of 0.3.

Regarding the structural dimension (Table 2), the KMO was equal to 0.639 and the Bartlett’s test of sphericity reached statistical significance (p < 0.001). From the principal component analysis, there were three constant components: access, conservation, and equipment and sanitary structure, with eigenvalues above 1, which explains the 33.1%, 16.1% and 14.4% of the variance, respectively. The value in the split-half test for this dimension was 0.805. Cronbach’s alpha was above 0.7 in the component named access, with values close to 0.6 in the other two components (0.63 and 0.58). The commonality (h²) only produced values above the recommended minimum of 0.3.

In the relational dimension (Table 3), the KMO was equal to 0.681 and the Bartlett’s test of sphericity reached statistical significance (p < 0.001). During the principal component analysis, the items were grouped into: community relations (explaining 38.1% of variance) and relationships at school (explaining 17.1% of the variance), all with eigenvalues greater than 1. The value in the split-half test was 0.646 in this dimension. Cronbach’s alpha was below 0.7 in both factors. The commonality (h²) only produced values above the recommended minimum of 0.3.

We excluded 12 items in the frequency analysis of the responses (Table 4) and 20 in the analysis of principal components and reliability, according to the exclusion criteria employed (Table 5).

DISCUSSION

The evaluated instrument includes improvements compared to existing ones: it has a reduced dimension, with easy applicability; the items are all presented in the form of questions with “yes or no” answers, which avoids ambiguities regarding whether or not that feature or process exists; it is the first that includes items that have been directly observed by whomever collected the data, which reduces the bias from managers who have certain ideals about their own school; it was designed to be applied only once in each school, without being proposed to different members of the school community, which avoids the information collected reflecting the ideological perspectives or oscillating between dissatisfaction and satisfaction with the school, which can vary widely in the same community; and work issues such as sexuality and violence, which are part of daily life for communities in developing countries, but are not included or are not featured in other instruments.

The instrument’s final composition can be considered appropriate since, in addition to the satisfactory values from the overall reliability and validity analyses, the dimensions are composed of items that coalesce in components that are compatible with themes indicated as priorities for health promotion in the school environment. These are: drugs²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,33. Bond L, Patton G, Glover S, Carlin JB, Butler H, Thomas L et al. The Gatehouse Project: can a multilevel school intervention affect emotional wellbeing and health risk behaviours? J Epidemiol Community Health. 2004;58(12):997-1003. DOI:10.1136/jech.2003.009449^,2020. Martino-McAllister J, Wessel MT. An evaluation of a social norms marketing project for tobacco prevention with middle, high, and college students; use of funds from the Tobacco Master Settlement (Virginia). J Drug Educ. 2005;35(3):185-200. DOI:10.2190/2KJH-558A-VXK1-E8QD^,2222. Page RM, Danielson M. Multi-country, cross-national comparison of youth tobacco use: findings from global school-based health surveys. Addict Behav. 2011;36(5):470-8. DOI:10.1016/j.addbeh.2011.01.008, sexuality⁴4. Brooks FM, Magnusson J, Spencer N, Morgan A. Adolescent multiple risk behaviour: an asset approach to the role of family, school and community. J Public Health (Bangkok). 2012;34(suppl 1):i48-56. DOI:10.1093/pubmed/fds001^,88. Gerouki M. Sexuality and relationships education in the Greek primary schools: see no evil, hear no evil, speak no evil. Sex Education. 2007;7(1):81-100. DOI:10.1080/14681810601134710^,1313. Kirby DB, Baumler E, Coyle KK, Basen-Engquist K, Parcel GS, Harrist R et al. The “Safer Choices” intervention: its impact on the sexual behaviors of different subgroups of high school students. J Adolesc Health. 2004;35(6):442-52. DOI:10.1016/S1054-139X(04)00071-0^,2121. Moizés JS, Bueno SMV. Compreensão sobre sexualidade e sexo nas escolas segundo professores do ensino fundamental. Rev Esc Enferm USP. 2010;44(1):205-12. DOI:10.1590/S0080-62342010000100029, violence¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2525. Ruotti C. Violência em meio escolar: fatos e representações na produção da realidade. Educ Pesq. 2010;36(1):339-55. DOI:10.1590/S1517-97022010000100010, prejudice¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2525. Ruotti C. Violência em meio escolar: fatos e representações na produção da realidade. Educ Pesq. 2010;36(1):339-55. DOI:10.1590/S1517-97022010000100010, self-care²⁶26. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755^,2929. Yoshimura N, Jimba M, Poudel KC, Chanthavisouk C, Iwamoto A, Phommasack B et al. Health promoting schools in urban, semi-urban and rural Lao PDR. Health Promot Int. 2009;24(2):166-76. DOI:10.1093/heapro/dap004, culture of peace¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2525. Ruotti C. Violência em meio escolar: fatos e representações na produção da realidade. Educ Pesq. 2010;36(1):339-55. DOI:10.1590/S1517-97022010000100010^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, quality of life¹⁵15. Lee A, Cheng FFK, St Leger L. Evaluating health-promoting schools in Hong Kong: development of a framework. Health Promot Int. 2005;20(2):177-86. DOI:10.1093/heapro/dah607^,1818. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2626. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, accessibility and security¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2525. Ruotti C. Violência em meio escolar: fatos e representações na produção da realidade. Educ Pesq. 2010;36(1):339-55. DOI:10.1590/S1517-97022010000100010^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, sanitary conditions¹⁵15. Lee A, Cheng FFK, St Leger L. Evaluating health-promoting schools in Hong Kong: development of a framework. Health Promot Int. 2005;20(2):177-86. DOI:10.1093/heapro/dah607

16. Lee A, Cheng FFK, Yuen H, Ho M, Lo A, Fung Y et al. Achieving good standards in health promoting schools: preliminary analysis one year after the implementation of the Hong Kong Healthy Schools Award scheme. Public Health. 2007;121(10):752-60. DOI:10.1016/j.puhe.2007.01.014

17. Lee A, St Leger L, Cheng FFK. The status of health-promoting schools in Hong Kong and implications for further development. Health Promot Int. 2007;22(4):316-26. DOI:10.1093/heapro/dam029^-1818. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021, conservation of the structure and equipment available to the school community¹¹11. Jurg ME, Kremers SPJ, Candel MJJM, Van der Wal MF, Meij JSBD. A controlled trial of a school-based environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promot Int. 2006;21(4):320-30. DOI:10.1093/heapro/dal032^,1414. Leatherdale S, Manske S, Faulkner G, Arbour K, Bredin C. A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act. 2010;7:6. DOI:10.1186/1479-5868-7-6^,1818. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, in addition to relationships within the school and with the community³3. Bond L, Patton G, Glover S, Carlin JB, Butler H, Thomas L et al. The Gatehouse Project: can a multilevel school intervention affect emotional wellbeing and health risk behaviours? J Epidemiol Community Health. 2004;58(12):997-1003. DOI:10.1136/jech.2003.009449^,44. Brooks FM, Magnusson J, Spencer N, Morgan A. Adolescent multiple risk behaviour: an asset approach to the role of family, school and community. J Public Health (Bangkok). 2012;34(suppl 1):i48-56. DOI:10.1093/pubmed/fds001^,1818. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2424. Piola SF, Vianna SM, Vivas-Consuelo D. Estudo Delphi: atores sociais e tendências do sistema de saúde brasileiro. Cad Saude Publica. 2002;18 supl:S181-90. DOI:10.1590/S0102-311X2002000700018.

In addition to the efforts made by the World Health Organization, who proposed a broad instrument to investigate health in schools (the Global School-based Student Health Survey [GSHS])²²22. Page RM, Danielson M. Multi-country, cross-national comparison of youth tobacco use: findings from global school-based health surveys. Addict Behav. 2011;36(5):470-8. DOI:10.1016/j.addbeh.2011.01.008, the Center for Disease Control in the United States has also developed their own instrument²⁶26. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x. Research initiatives in other countries have also led to the creation of instruments with this purpose²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,1616. Lee A, Cheng FFK, Yuen H, Ho M, Lo A, Fung Y et al. Achieving good standards in health promoting schools: preliminary analysis one year after the implementation of the Hong Kong Healthy Schools Award scheme. Public Health. 2007;121(10):752-60. DOI:10.1016/j.puhe.2007.01.014^,1818. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021. In Brazil, in 2012, the Pesquisa Nacional de Saúde do Escolar (PeNSE – National Survey of School Health) ^aaMinistério do Planejamento, Orçamento e Gestão (BR); Instituto Brasileiro de Geografia e Estatística. Pesquisa nacional de saúde do escolar 2012. Rio de Janeiro (RJ): IBGE; 2013. included items regarding the school environment in its questionnaire, to be proposed to the managers of the visited institutions.

Like the instrument featured in this article, the set of items in PeNSE did not employ the Likert scale and the two models are not similar concerning the formulation of various items. No direct observation items were included, which is different from what was employed in some studies on health at school, mainly for observing structural resources, equipment and physical area on the school premises¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2929. Yoshimura N, Jimba M, Poudel KC, Chanthavisouk C, Iwamoto A, Phommasack B et al. Health promoting schools in urban, semi-urban and rural Lao PDR. Health Promot Int. 2009;24(2):166-76. DOI:10.1093/heapro/dap004.

All the instruments developed to date¹1. Almeida LS, Freire T. Metodologia da investigação em psicologia e educação. 4 ed. Braga: Psiquilibrios; 2007. ⁷7. Figueiredo Filho DB, Silva Júnior JA. Visão além do alcance: uma introdução à análise fatorial. Opin Publica. 2010;16(1):160-85. DOI:10.1590/S0104-62762010000100007^,1818. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2222. Page RM, Danielson M. Multi-country, cross-national comparison of youth tobacco use: findings from global school-based health surveys. Addict Behav. 2011;36(5):470-8. DOI:10.1016/j.addbeh.2011.01.008^,2626. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755 are central to the principles of the Ottawa Charter for health promotion: health policies at school and personal skills in health, which resemble the pedagogical dimension of the instrument from this study; physical environment and provision of health services, which resemble the structural dimension; social environment; and school and community relations, which resemble the relational dimension. The origin and insertion of the researchers in different national contexts lead instruments to express specific characteristics, which are reflected as differences in the cluster of its items. For example, sexual health is present in this and in an Australian study²⁷27. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, while the Korean instrument¹⁸18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021 analyzes protective measures against disasters and extreme weather situations, which are of little use in some contexts.

The GSHS questionnaire has a very extensive number of items regarding agreed rules in the school environment and investigates these items requesting that its terms are explicit in written form, which may not be customary in some cultures. For example, details of internal regulations included within public documents are not common in Brazilian schools.

Another difference might also be perceived in the security item, since the transit of vehicles within the school environment was a concern in countries with lower rates of urban violence¹⁸18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^,2626. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, while for the Brazilian reality, access to school and the school environment seemed more relevant¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005. In Brazil, violence represents a social problem and the school is seen as a suitable setting to help deal with this situation, which requires both educational and public security interventions¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2525. Ruotti C. Violência em meio escolar: fatos e representações na produção da realidade. Educ Pesq. 2010;36(1):339-55. DOI:10.1590/S1517-97022010000100010.

The overall analysis of the pedagogical dimension suggested a good level of acceptable validity and reliability, with the highest factorial load being in the drugs and sexuality component. The explicit dimension was satisfactorily in line with health promotion in school, prioritizing actions that give schools the opportunity to make healthy choices and encourage self-care.

The sexual health theme presented a good factorial load, but was excluded due to its representation in the structural dimension, where it appeared as a direct observation of the existence of informational materials on gender and sexuality in the common areas of the school. The theme had already been excluded in frequency analysis, as it was absent in 52 of the 53 schools visited. This, which is apparently contradictory to the load of the theme in the pedagogical dimension, is suggestive of the difficulty, high even today, of information related to sexuality being accessible in the school context–with the subject still a matter that is avoided. Without a more explicit proposal of information, other than as a classroom activity (thus, of a eventual and collective nature), access to information and some kind of individual support, while preserving the intimacy of those looking for it, appears to be of limited scope⁸8. Gerouki M. Sexuality and relationships education in the Greek primary schools: see no evil, hear no evil, speak no evil. Sex Education. 2007;7(1):81-100. DOI:10.1080/14681810601134710^,1313. Kirby DB, Baumler E, Coyle KK, Basen-Engquist K, Parcel GS, Harrist R et al. The “Safer Choices” intervention: its impact on the sexual behaviors of different subgroups of high school students. J Adolesc Health. 2004;35(6):442-52. DOI:10.1016/S1054-139X(04)00071-0^,2121. Moizés JS, Bueno SMV. Compreensão sobre sexualidade e sexo nas escolas segundo professores do ensino fundamental. Rev Esc Enferm USP. 2010;44(1):205-12. DOI:10.1590/S0080-62342010000100029.

During the overall analysis of the structural dimension, a good level of validity and reliability was considered acceptable. The component regarding the sanitary structure presented a Cronbach’s alpha slightly below what is deemed acceptable, however, it was made up of essential items²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,1111. Jurg ME, Kremers SPJ, Candel MJJM, Van der Wal MF, Meij JSBD. A controlled trial of a school-based environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promot Int. 2006;21(4):320-30. DOI:10.1093/heapro/dal032^,1414. Leatherdale S, Manske S, Faulkner G, Arbour K, Bredin C. A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act. 2010;7:6. DOI:10.1186/1479-5868-7-6^,2626. Sherwood-Puzzello CM, Miller M, Lohrmann D, Gregory P. Implementation of CDC’s School Health Index in 3 Midwest middle schools: motivation for change. J Sch Health. 2007;77(6):285-93. DOI:10.1111/j.1746-1561.2007.00209.x and its preservation reinforced the overall evaluation of the dimension. This dimension’s items had good factorials loads, with better results in the access component, which encompassed relevant aspects in terms of structural features of accessibility and safety in the school environment, which is also in agreement with the literature²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,1616. Lee A, Cheng FFK, Yuen H, Ho M, Lo A, Fung Y et al. Achieving good standards in health promoting schools: preliminary analysis one year after the implementation of the Hong Kong Healthy Schools Award scheme. Public Health. 2007;121(10):752-60. DOI:10.1016/j.puhe.2007.01.014

17. Lee A, St Leger L, Cheng FFK. The status of health-promoting schools in Hong Kong and implications for further development. Health Promot Int. 2007;22(4):316-26. DOI:10.1093/heapro/dam029

18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021^-1919. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755.

During the general analysis of the relational dimension, we observed good validity and acceptable reliability. We found adequate factorial loads in the constituent items of each component, showing the school’s social climate¹⁹19. Liberal EF, Aires RT, Aires MT, Osório ACA. Escola segura. J Pediatr (Rio J). 2005;81:s155-s163. DOI:10.1590/S0021-75572005000700005^,2424. Piola SF, Vianna SM, Vivas-Consuelo D. Estudo Delphi: atores sociais e tendências do sistema de saúde brasileiro. Cad Saude Publica. 2002;18 supl:S181-90. DOI:10.1590/S0102-311X2002000700018. The students’ participation item in the processes of picking the school was excluded from this study’s validity and reliability analysis and represented a considerable and important loss of information¹⁵15. Lee A, Cheng FFK, St Leger L. Evaluating health-promoting schools in Hong Kong: development of a framework. Health Promot Int. 2005;20(2):177-86. DOI:10.1093/heapro/dah607. This fact may have been due to the sample size used, which was smaller than what is recommended for validation analysis. The number of five subjects per item has been suggested²³23. Pasquali L. Testes referentes a construto: teoria e modelo de construção. In: Pasquali L, organizador. Instrumentos psicológicos: manual prático de elaboração. Brasília (DF): Laboratório de Pesquisa em Avaliação e Medida - LabPAM; 1999. p.37-41.. An instrument of 60 items should be tested in approximately 300 schools. It is possible that increasing the number of schools, relevant items that did not remain in this case, would be included, or even a greater number of factors could be found. This also has an impact on the reliability analyses, since scales with few items tend to provide lower Cronbach’s alpha results⁶6. Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78(1):98-104. DOI:10.1037/0021-9010.78.1.98.

The limitation represented by the low number of schools where the instrument was applied might also explain, at least in part, why the instrument did not have an excellent performance, but rather only satisfactory with good factorials loads (> 0.4) and acceptable reliability (> 0.6). The Korean instrument¹⁸18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021 was designed to be answered by members of the school community, unlike the one from this study, which has a singular application. When only a single data collection exist per school by a properly trained investigator, with items from direct observation, more objective data tends to be generated. The seven factors of that instrument had Cronbach’s alphas ranging from 0.86 and 0.97 for the instrument’s cluster. The Korean instrument was the first to be proposed and subjected to reliability and validity analyses, which includes a confirmatory factor analysis. The other instruments are not able to provide sufficient assurance for validity and reliability¹⁸18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021. The Australian instrument²2. Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health. 2004;74(1):23-9. DOI:10.1111/j.1746-1561.2004.tb06597.x^,2727. Stewart D. Implementing mental health promotion in schools: a process evaluation. Int J Ment Health Promot. 2008;10(1):32-41. DOI:10.1080/14623730.2008.9721755, for example, is referred to as having had good performance in the overall reliability analysis, with a Cronbach’s alpha result of 0.88, but without reaching consistent factorial structures so as to employ factor analysis and validate the construct¹⁸18. Lee EY, Shin YJ, Choi BY, Cho HSM. Reliability and validity of a scale for health-promoting schools. Health Promot Int. 2014;29(4):759-67. DOI:10.1093/heapro/dat021.

Another limiting factor of this study was that it only evaluated schools in cities located in the Southern region of Brazil, which were similar in terms of population size. Studies with other populations and with greater heterogeneity of school environments would enable the evaluation of the consistency of the results and obtain a better evaluation of the instrument.

The data suggest that the proposed structure is consistent, with a sufficient number of dimensions and factors. This instrument can be applied as it currently exists, with 28 items, but it is recommended that further studies are undertaken, even with 60 original items, so that the local realities of the different regions of Brazil are included, with their distinct social and demographic characteristics. In addition, other psychometric properties such as reproducibility and criteria-related validity still need to be studied in such a way as to complement and further qualify the validation process of the instrument.

REFERENCES

Publication Dates

History