Abstract

The Drug Resistance Educational Program (PROERD) is Brazil’s most widespread school-based prevention program; its current curriculum is based on the North American Keepin’ it REAL (kiR) program. There is no evidence of the effectiveness of PROERD in preventing drug use, pointing to the need for further studies to understand these findings. The aim of the study was to synthesis the evidence of the effect of the kiR curriculum (PROERD) through a systematic review. We found 17 studies that reported the effects of different versions of kiR on drug use and/or violence. Except for the Brazilian study, no studies were found that assessed the effect on drug use of the version applied by police officers (DARE-kiR), the same one implemented by PROERD. Favorable evidence of kiR in drug use prevention was found for the 7th-grade curriculum, which contradicts the PROERD’s null-effect results. No international evidence of the effect of kiR was found in the 5th-grade curriculum, in the same line as the PROERD’s study. It is suggested that PROERD’s 7th-grade curriculum should be revised to reflect international results and that the 5th-grade curriculum should be reconsidered in light of the negative international evidence.

Key words:
Primary prevention; Substance use; Systematic review

Introduction

In order to reduce drug use among adolescents worldwide, evidence-based prevention programs that promote the development of life skills have been implemented in schools¹1 United Nations Office on Drugs and Crime (UNODC). International Standards on Drug Use Prevention - Second updated edition [Internet]. Viena: United Nations Office on Drugs and Crime, WHO; 2018 [cited 2021 mar 13]. Available from: http://www.unodc.org/documents/prevention/standards_180412.pdf.
http://www.unodc.org/documents/preventio... . Keepin’ it REAL (kiR) is an example of a school-based prevention program founded on scientific evidence, sensitive to the cultural narrative of the environment, and grounded, among others social skills, in the promotion of anti-drug rules, teaching resistance skills (refuse, explain, avoid, and leave), and decision-making²2 Gosin M, Marsiglia FF, Hecht ML. keepin' it R.E.A.L.: A Drug Resistance Curriculum Tailored to the Strengths and Needs of Pre-Adolescents of the Southwest. J Drug Educ 2003; 33(2):119-142.. kiR was developed by a team of researchers from the “REAL Prevention” group at the University of Pennsylvania, in the United States³3 Marsiglia FF, Hecht ML. Keepin'it REAL: An evidence-based program. SantaCruz: ETR Associates; 2005.. The program theoretical basis integrates a series of solid psychosocial constructs, such as: 1 - narrative engagement theory⁴4 Miller-Day M, Hecht ML. Narrative Means to Preventative Ends: A Narrative Engagement Framework for Designing Prevention Interventions. Health Commun 2013; 28(7):657-670., which highlights the importance of communicating personal narratives in drug prevention messages; 2 - cultural foundation theory, which argues that preventive messages should be transmitted by the target population’s culture⁵5 Hecht ML, Krieger JLR. The Principle of Cultural Grounding in School-Based Substance Abuse Prevention. J Lang Soc Psychol 2006; 25(3):301-319.; 3 - social and emotional learning theory, which helps children better understand their feelings and show empathy for others⁶6 Durlak JA, Domitrovich CE, Weissberg RP, Gullotta TP. Handbook of Social and Emotional Learning: Research and Practice. New York: Guilford Press; 2015.; and 4 - theory of normative beliefs for non-use of drugs, which suggests the importance of involving adolescents to think critically about their perceptions of drug use approval⁷7 Cialdini RB, Reno RR, Kallgren CA. A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. J Pers Soc Psychol 1990; 58(6):1015-1026..

Over the years, kiR developers have adapted and produced different program versions, according to the culture, race, country, or area where it was replicated (e.g., in a mostly Latino population, in a rural area, or for Mexican students). All kiR versions maintained the central and theoretical program elements, which were culturally adapted for the populations they aimed at. Several studies have already been conducted to assess the effectiveness of these various program versions in different population samples, with apparently contradictory results⁸8 Elek E, Wagstaff DA, Hecht ML. Effects of the 5th and 7th Grade Enhanced Versions of the Keepin' it Real Substance Use Prevention Curriculum. J Drug Educ 2010; 40(1):61-79.^,99 Hecht ML, Marsiglia FF, Elek E, Wagstaff DA, Kulis S, Dustman P, Miller-Day M. Culturally Grounded Substance Use Prevention: An Evaluation of the keepin'' it R.E.A.L. Curriculum. Prev Sci 2003; 4(4):233-248.. Although kiR is considered an evidence-based program, the lack of a synthesis of the results of these different studies (which evaluated the different program versions) makes it difficult to understand its real effectiveness.

The Drug Abuse Resistance Education Keepin’ it REAL (D.A.R.E.-kiR) prevention program is one of the kiR versions, adapted by the Los Angeles Police Department (United States): Drug Abuse Resistance Education (D.A.R.E.). Since 2009, police officers have been implementing the D.A.R.E.-kiR program in American schools through two curricula: elementary school (for 5^th-grade students), and middle school (for 7^th-grade students). The Brazilian Military Police, through the Drug Resistance Educational Program (PROERD), in partnership with the American D.A.R.E., translated the D.A.R.E.-kiR program, and since 2014 has been implementing this curriculum in Brazilian schools. The current PROERD curriculum (which is therefore the D.A.R.E.-kiR curriculum translated into the Portuguese Language) was entitled “Caindo na Real,” and has been applied by trained police officers in public and private schools in the format of 10-week lessons. The program has activities guided by the student and instructor manual and presents two curricula: one for 5th- and other for 7^th-grade students of elementary school. In 2019, the first randomized controlled trial (RCT) was conducted to assess the preventive PROERD effectiveness regarding its current “Caindo na Real” version, in both curricula (5^th and 7^th grades) in the city of São Paulo. The RCT did not find evidence that PROERD is being effective in reducing drug consumption among students in both age groups, suggesting a review of the curriculum elements that have been implemented with a view to culturally adapting the program to the Brazilian reality¹⁰10 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413..

PROERD is currently the most prevalent drug and violence prevention program in Brazilian schools¹¹11 Pereira APD, Sanchez ZM. Characteristics of school-based drug prevention programs in Brazil. Cien Saude Colet 2020; 25(8):3131-3142., and has been applied in those places for almost three decades. Initially, PROERD had another curriculum, also developed in partnership with D.A.R.E.¹²12 Clayton RR, Cattarello AM, Johnstone BM. The Effectiveness of Drug Abuse Resistance Education (Project DARE): 5-Year Follow-Up Results. Prev Med (Baltim) 1996; 25(3):307-318.; however, after the publication of studies showing unfavorable results of the program among American students, both the American and the Brazilian Police decided to replace it with the current D.A.R.E.-kiR curriculum¹³13 Lynam DR, Milich R, Zimmerman R, Novak SP, Logan TK, Martin C, Leukefeld C, Clayton R. Project DARE: No effects at 10-year follow-up. J Consult Clin Psychol 1999; 67(4):590-593.. Since its origin, PROERD has been worked with almost ten million children and adolescents in the state of São Paulo alone, and in 2018 the program reached 646,457 students and involved 716 police officers’ work¹⁴14 Programa Educacional de Resistência às Drogas e à violência (PROERD). Produtividade PROERD [Internet]. 2018 [acessado 2022 jun 22]. Disponível em: http://www4.policiamilitar.sp.gov.br/unidades/dpcdh/index.php/dados-proerd/.
http://www4.policiamilitar.sp.gov.br/uni... . These numbers tend to be even higher, since the program has been approved as law to become public policy in several Brazilian cities and states, and it is the target of legislative and executive articulations¹⁵15 São Paulo. Assembleia Legislativa do Estado de São Paulo. Lei no 17.171, de 11 de outubro de 2019. Determina que todas as escolas públicas do ensino fundamental e médio do Estado apresentem aos seus alunos, ao menos uma vez no ano letivo, o Programa Educacional de Resistência às Drogas e à Violência - Proerd, e fixa outras providências. Assessoria Técnica da Casa Civil; 2019..

The recent neutral preventive results of the evaluation of “PROERD/Caindo na Real”¹⁰10 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413., as well as the lack of synthesis regarding kiR effectiveness, raise many questions about the evidence that supports the implementation of the current PROERD curriculum as a public policy in Brazilian schools. Thus, this article aims to investigate and synthesize the evidence available in the literature on kiR efficacy/effectiveness in preventing drug use and violence in order to contribute to understanding the program non-effectiveness findings, as well as helping in the direction of future recommendations on the continuity of this program version as a public policy.

Methodology

A systematic literature review was performed based on the recommendation of the Cochrane Collaboration Statement and PRISMA protocol¹⁶16 Moher D. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med 2009; 151(4):264.. The research question used to guide this review was: “Is the kiR prevention program effective in reducing drug use and violence among adolescents?” The acronym PICO (Population, Intervention, Comparison, Outcome) was used to guide this review: the adolescents were the population in question; kiR was the intervention studied; the comparison was between adolescents who did not have contact with the program and those who had contact with other program versions that were not culturally adapted, and drug use and/or violence was the outcome analyzed.

Search on Databases

The systematic review was based on four databases (PubMed, ERIC, SciELO and Cochrane) using the search term “Keepin’ it REAL”, since all publications evaluating the program usually present the program name in the title and/or or abstract. Thus, a single term search was chosen. There was no limitation of study year of publication or language. The search resulted in 105 articles (52 on PubMed, 24 on ERIC, three on SciELO, and 26 on Cochrane) that were reviewed according to the criteria described below. As a way to ensure search effectiveness, the references of the selected articles were also reviewed; however, no new articles were found that were in accordance with the scope of this review.

Inclusion and exclusion criteria

The inclusion criteria were: 1 - articles evaluating the kiR effects by means of RCT, considering that this is the gold standard for evaluating the intervention effects, and 2 - articles reporting the program effect on drug use and/or violence, since these are the expected PROERD outcomes. The exclusion criteria were: 1 - articles evaluating the kiR effect combined with another intervention (e.g., kiR + intervention for family), and 2 - articles evaluating the effect of only one component of the intervention and not the entire intervention (e.g., videos only).

Article extraction based on selection criteria

In the first phase, titles and abstracts were checked out for the inclusion and exclusion criteria described above. In the second phase, the articles selected were analyzed in full to verify if they would really be included in the review analysis. The first author read all the titles and abstracts and selected the articles that were later read in full by himself and by a second author independently. Thus, the two authors reviewed the full texts of the articles, verifying if they really met the inclusion criteria, and also data extraction and quality evaluation. In case of disagreement between authors, a third senior author was consulted. All articles that did not meet the inclusion criteria after full-text review were excluded from the review. Data from the articles were extracted considering the following items: study country and city, study year, sample profile, school grade in which the program was implemented (e.g., 5^th or 7^th grade), follow-up time (in months), description of the intervention, evaluated outcomes, and main results.

Quality of articles

The quality of articles with regard to the risk of bias was evaluated based on the Cochrane scale for evaluating the quality of systematic reviews¹⁷17 Higgins JPT, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA; Cochrane Bias Methods Group; Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011; 343:d5928., commonly used for evaluation of school prevention programs¹⁸18 Foxcroft DR, Tsertsvadze A. Universal family-based prevention programs for alcohol misuse in young people. Cochrane Database Syst Rev 2011; 9(9):CD009308.. This scale evaluates studies considering the following criteria: generation of an adequate allocation sequence; allocation concealment; blinding of investigated, investigators, and outcome evaluators, and data on incomplete results (description of losses and exclusions). Since the implementation of a double-blind approach in the evaluation of behavioral interventions is not feasible, the Cochrane scale was adapted so as not to consider this criterion when evaluating the quality of articles. Thus, trials were evaluated in five domains and scored from 1-3 in each domain (1=high risk of bias, 2=uncertain risk, 3=low risk)¹⁹19 Newton NC, Champion KE, Slade T, Chapman C, Stapinski L, Koning I, Tonks Z, Teesson M. A systematic review of combined student- and parent-based programs to prevent alcohol and other drug use among adolescents. Drug Alcohol Rev 2017; 36(3):337-351.. Scores were summed across the five domains to generate a total risk-of-bias score for each study, with a possible 5-15 range. Higher scores indicate higher quality and lower risk of bias. The level of risk of bias for each study had no influence on being included in the systematic review.

Results

In the first phase, by reviewing titles and abstracts, 37 articles were excluded for being duplicates and 47 articles were excluded for other reasons (40 for not being RCTs, three for evaluating only part of the intervention components, two for evaluating the effect of other interventions combined with kiR, and three for not evaluating “drug use” and/or “violence” as an outcome). In the second phase, 20 articles were analyzed in full, and three articles were excluded for not being RCTs (Figure 1). Finally, 17 studies were included in the review, and the following variables were extracted from each article: country, city, year in which the study was conducted, number of schools and students analyzed, follow-up time in months, description(s) of the intervention(s), evaluated outcomes, and main results of the studies (Chart 1).

Study characteristics

Among the 17 articles included in this review (Chart 1), 10 were conducted in the USA and seven in other countries (n=1 in Guatemala, n=4 in Mexico, n=1 in Spain, and n=1 in Brazil). The sample size ranged from a small study with 107 students from three schools to a large study with 6,035 students from 35 schools.

Most articles (n=14) reported data from the evaluation of the program adapted from the version developed for 7^th-grade students, and only four articles reported results of the kiR curriculum effect among 5^th-grade students of American schools. Both grades provided for in these curricula have the age equivalent in Brazil with the same classification: 5^th and 7^th grades. All articles report findings on the program being implemented by teachers, with the exception of the article conducted in Brazil (where the program is implemented by police officers). The vast majority of articles (n=13) reported having evaluated the intervention in public schools, and the other articles did not provide this information.

All the reviewed articles have one of the program developers among the authors (with the exception of the Brazilian study), that is, all RCTs conducted to evaluate the program effectiveness were conducted by the same researchers who developed and adapted keepin’ it REAL.

Intervention description

The reviewed articles report results of different kiR versions (14 different versions were identified: 10 versions with 7^th- and 8^th-grade students as target population; four versions with 5^th-grade students as target population, and one version with 6^th-grade students as target population), each of them adapted according to the culture, race, country, or area where the program was implemented. All versions were implemented by teachers trained in the classroom, except the Brazilian one (implemented by police officers). Firstly, the program was designed with 7^th-grade students in mind and had three different versions developed based on the students’ ethnicity/race (considered the original program versions): Mexican/Mexican American, KiR-Black/White, KiR-multicultural⁹9 Hecht ML, Marsiglia FF, Elek E, Wagstaff DA, Kulis S, Dustman P, Miller-Day M. Culturally Grounded Substance Use Prevention: An Evaluation of the keepin'' it R.E.A.L. Curriculum. Prev Sci 2003; 4(4):233-248.. Subsequently, adaptations were made to the curriculum to reach younger students, resulting in the KiR-multicultural for 5^th-grade students²⁰20 Hecht ML, Elek E, Wagstaff DA, Kam JA, Marsiglia F, Dustman P, Reeves L, Harthun M. Immediate and Short-Term Effects of the 5th Grade Version of the Keepin' it Real Substance Use Prevention Intervention. J Drug Educ 2008; 38(3):225-251.. kiR-Plus (developed to deal with the transition period that adolescents go through) and kiR Acculturation Enhanced -kiR-AE (developed especially for Hispanic students in the process of acculturation in the USA) were also developed for both 5^th- and 7^th-grade students⁸8 Elek E, Wagstaff DA, Hecht ML. Effects of the 5th and 7th Grade Enhanced Versions of the Keepin' it Real Substance Use Prevention Curriculum. J Drug Educ 2010; 40(1):61-79.^,2121 Marsiglia FF, Kulis S, Yabiku ST, Nieri TA, Coleman E. When to Intervene: Elementary School, Middle School or Both? Effects of keepin' It REAL on Substance Use Trajectories of Mexican Heritage Youth. Prev Sci 2011; 12(1):48-62.. In addition, four more versions were created for 7^th-grade students: a version to be applied in rural areas²²22 Hecht ML, Shin Y, Pettigrew J, Miller-Day M, Krieger JL. Designed Cultural Adaptation and Delivery Quality in Rural Substance Use Prevention: an Effectiveness Trial for the Keepin' it REAL Curriculum. Prev Sci 2018; 19(8):1008-1018.; a version for American indigenous peoples (Living in 2 Worlds)²³23 Kulis SS, Ayers SL, Harthun ML. Substance Use Prevention for Urban American Indian Youth: A Efficacy Trial of the Culturally Adapted Living in 2 Worlds Program. J Prim Prev 2017; 38(1-2):137-158.; a version culturally adapted for Mexicans (Mantente Real Mexico)²⁴24 Marsiglia FF, Booth JM, Ayers SL, Nuño-Gutierrez BL, Kulis S, Hoffman S. Short-Term Effects on Substance Use of the Keepin' It REAL Pilot Prevention Program: Linguistically Adapted for Youth in Jalisco, Mexico. Prev Sci 2014; 15(5):694-704.

25 Marsiglia FF, Kulis SS, Booth JM, Nuño-Gutierrez BL, Robbins DE. Long-Term Effects of the keepin' it REAL Model Program in Mexico: Substance Use Trajectories of Guadalajara Middle School Students. J Prim Prev 2015; 36(2):93-104.

26 Kulis SS, Garcia-Perez H, Marsiglia FF, Ayers SL. Testing a Culturally Adapted Youth Substance Use Prevention Program in a Mexican Border City: Mantente REAL. Subst Use Misuse 2021; 56(2):245-257.^-2727 Kulis SS, Marsiglia FF, Medina-Mora ME, Nuño-Gutiérrez BL, Corona MD, Ayers SL. Keepin' It REAL-Mantente REAL in Mexico: a Cluster Randomized Controlled Trial of a Culturally Adapted Substance Use Prevention Curriculum for Early Adolescents. Prev Sci 2021; 22(5):645-657., and another version culturally adapted for the Spanish people (Mantente Real España). There is also Mantente Real version that was only translated into the Spanish language and not adapted, which was applied to 6^th-grade students in Guatemala²⁸28 Kulis SS, Marsiglia FF, Porta M, Arévalo Avalos MR, Ayers SL. Testing the keepin' it REAL Substance Use Prevention Curriculum Among Early Adolescents in Guatemala City. Prev Sci 2019; 20(4):532-543..

It is important to clarify that the authors produced several articles from the same data collection, that is, from the same RCT with the same student sample. Of the 17 articles included in this review, five report data from the same 7th-grade student sample in the city of Phoenix (Arizona-USA): the first trial, carried out in 1998, evaluated the kiR effect among 7^th-grade students⁹9 Hecht ML, Marsiglia FF, Elek E, Wagstaff DA, Kulis S, Dustman P, Miller-Day M. Culturally Grounded Substance Use Prevention: An Evaluation of the keepin'' it R.E.A.L. Curriculum. Prev Sci 2003; 4(4):233-248., and three program versions (Mexican/Mexican American; Black/White; multicultural) were compared with the control group. The first article reported the primary results⁹9 Hecht ML, Marsiglia FF, Elek E, Wagstaff DA, Kulis S, Dustman P, Miller-Day M. Culturally Grounded Substance Use Prevention: An Evaluation of the keepin'' it R.E.A.L. Curriculum. Prev Sci 2003; 4(4):233-248., and the other articles presented secondary and subgroup analyses²⁹29 Kulis S, Marsiglia FF, Elek E, Dustman P, Wagstaff DA, Hecht ML. Mexican/Mexican American Adolescents and keepin' it REAL: An Evidence-Based Substance Use Prevention Program. Child Sch 2005; 27(3):133-145.

30 Yabiku S, Kulis S, Marsiglia FF, Lewin B, Nieri T, Hussaini S. Neighborhood Effects on the Efficacy of a Program to Prevent Youth Alcohol Use. Subst Use Misuse 2007; 42(1):65-87.

31 Kulis S, Yabiku ST, Marsiglia FF, Nieri T, Crossman A. Differences by Gender, Ethnicity, and Acculturation in the Efficacy of the keepin' it Real Model Prevention Program. J Drug Educ 2007; 37(2):123-144.^-3232 Kulis S, Nieri T, Yabiku S, Stromwall LK, Marsiglia FF. Promoting Reduced and Discontinued Substance Use among Adolescent Substance Users: Effectiveness of a Universal Prevention Program. Prev Sci 2007; 8(1):35-49..

Another three articles report findings from a second RCT conducted in 2004, with a different student sample but also in the city of Phoenix (Arizona, USA), to assess the kiR-Plus and kiR-AE curricula effect among 5^th- and 7^th-grade students⁸8 Elek E, Wagstaff DA, Hecht ML. Effects of the 5th and 7th Grade Enhanced Versions of the Keepin' it Real Substance Use Prevention Curriculum. J Drug Educ 2010; 40(1):61-79.^,2020 Hecht ML, Elek E, Wagstaff DA, Kam JA, Marsiglia F, Dustman P, Reeves L, Harthun M. Immediate and Short-Term Effects of the 5th Grade Version of the Keepin' it Real Substance Use Prevention Intervention. J Drug Educ 2008; 38(3):225-251.^,2121 Marsiglia FF, Kulis S, Yabiku ST, Nieri TA, Coleman E. When to Intervene: Elementary School, Middle School or Both? Effects of keepin' It REAL on Substance Use Trajectories of Mexican Heritage Youth. Prev Sci 2011; 12(1):48-62.. This was the only RCT found that evaluated the curriculum aimed at 5^th-grade students.

Next, other original RCTs with different samples were conducted to evaluate other program versions. A study carried out in 2009 in Phoenix (Arizona-USA) compared a new version of the program adapted for American indigenous people (Living in 2 Worlds) with the original version of kiR, among 7^th- and 8^th-grade students²³23 Kulis SS, Ayers SL, Harthun ML. Substance Use Prevention for Urban American Indian Youth: A Efficacy Trial of the Culturally Adapted Living in 2 Worlds Program. J Prim Prev 2017; 38(1-2):137-158.. Another study, carried out in 2009 in Pennsylvania and Ohio (USA), evaluated the effect of the program version adapted for rural areas among 7^th-grade students²²22 Hecht ML, Shin Y, Pettigrew J, Miller-Day M, Krieger JL. Designed Cultural Adaptation and Delivery Quality in Rural Substance Use Prevention: an Effectiveness Trial for the Keepin' it REAL Curriculum. Prev Sci 2018; 19(8):1008-1018..

Five different RCTs were conducted to evaluate the kiR 7^th-grade version adapted for the Spanish language, the Mantente Real (MR) program. The first study carried out in Guatemala in 2013 evaluated the version which was only translated into the Spanish language and not adapted, and the sample was composed of 6^th-grade students²⁸28 Kulis SS, Marsiglia FF, Porta M, Arévalo Avalos MR, Ayers SL. Testing the keepin' it REAL Substance Use Prevention Curriculum Among Early Adolescents in Guatemala City. Prev Sci 2019; 20(4):532-543.. Two articles reported short and long-term results of another RCT carried out in the city of Guadalajara, Mexico, to evaluate the MR version already culturally adapted for 7^th- and 8^th-grade Mexican students²⁴24 Marsiglia FF, Booth JM, Ayers SL, Nuño-Gutierrez BL, Kulis S, Hoffman S. Short-Term Effects on Substance Use of the Keepin' It REAL Pilot Prevention Program: Linguistically Adapted for Youth in Jalisco, Mexico. Prev Sci 2014; 15(5):694-704.^,2525 Marsiglia FF, Kulis SS, Booth JM, Nuño-Gutierrez BL, Robbins DE. Long-Term Effects of the keepin' it REAL Model Program in Mexico: Substance Use Trajectories of Guadalajara Middle School Students. J Prim Prev 2015; 36(2):93-104.. The Mexican MR also had two other RCTs conducted more recently (2017-2018) with 7^th-grade students, with two different samples²⁶26 Kulis SS, Garcia-Perez H, Marsiglia FF, Ayers SL. Testing a Culturally Adapted Youth Substance Use Prevention Program in a Mexican Border City: Mantente REAL. Subst Use Misuse 2021; 56(2):245-257.^,2727 Kulis SS, Marsiglia FF, Medina-Mora ME, Nuño-Gutiérrez BL, Corona MD, Ayers SL. Keepin' It REAL-Mantente REAL in Mexico: a Cluster Randomized Controlled Trial of a Culturally Adapted Substance Use Prevention Curriculum for Early Adolescents. Prev Sci 2021; 22(5):645-657.. Finally, a recent pilot study (2021), conducted in two cities in Spain, evaluated the effect of the MR version adapted for the Spanish students³³33 Cutrín O, Kulis S, Maneiro L, MacFadden I, Navas MP, Alarcón D, Gómez-Fraguela JA, Villalba C, Marsiglia FF. Effectiveness of the Mantente REAL Program for Preventing Alcohol Use in Spanish Adolescents. Psychosoc Interv 2021; 30(3):113-122..

As previously reported in the introduction, in 2019 two RCTs evaluated the version translated into the Portuguese language of the D.A.R.E.-kiR curriculum, called “PROERD/Caindo na Real” (applied by police officers): an RCT evaluated the curriculum directed at 5^th-grade students and another RCT evaluated the curriculum for 7^th-grade students, both in schools in the city of São Paulo¹⁰10 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413..

In total, only two RCTs (one of them with three published articles) evaluated the kiR effect among 5^th-grade students; nine RCTs (15 published articles) evaluated the effect of different program versions aimed at 7^th-8^th-grade students, and one RCT (one published article) evaluated the Mantente Real version directed at 6th-grade Spanish students. It is noteworthy that there is an overlap of articles, because three articles reported the results of the program for 5^th- and 7^th-grade students⁸8 Elek E, Wagstaff DA, Hecht ML. Effects of the 5th and 7th Grade Enhanced Versions of the Keepin' it Real Substance Use Prevention Curriculum. J Drug Educ 2010; 40(1):61-79.^,1010 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413.^,2121 Marsiglia FF, Kulis S, Yabiku ST, Nieri TA, Coleman E. When to Intervene: Elementary School, Middle School or Both? Effects of keepin' It REAL on Substance Use Trajectories of Mexican Heritage Youth. Prev Sci 2011; 12(1):48-62..

Except for the Brazilian study, no other study evaluated the D.A.R.E.-kiR version, that is, all findings refer program versions different from the one adapted and disseminated by US police officers (version translated and implemented in Brazil by PROERD). Therefore, there is evidence that either the D.A.R.E.-kiR version was not evaluated in the United States (where it has been highly disseminated in schools) by RCTs for drug use and violence outcomes, or its evaluation was not published in journals indexed on the databases included in this study.

Evaluated outcomes

The results of the two RCTs that evaluated the kiR version directed at 5th-grade students showed that the program does not seem to be effective in reducing drug use by students⁸8 Elek E, Wagstaff DA, Hecht ML. Effects of the 5th and 7th Grade Enhanced Versions of the Keepin' it Real Substance Use Prevention Curriculum. J Drug Educ 2010; 40(1):61-79.^,1010 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413.^,2020 Hecht ML, Elek E, Wagstaff DA, Kam JA, Marsiglia F, Dustman P, Reeves L, Harthun M. Immediate and Short-Term Effects of the 5th Grade Version of the Keepin' it Real Substance Use Prevention Intervention. J Drug Educ 2008; 38(3):225-251.. On the other hand, except for the Brazilian study (PROERD/Caindo na Real), the results of other RCTs (n=8) that evaluated the curriculum for 7^th-8^th-grade students, in the different versions (Mexican/Mexican American, Black/White, multicultural, kiR-Plus, kiR-AE, Mantente Real, Living in 2 Worlds, and kiR for rural areas) showed a positive effect on reducing episodes of alcohol intoxication³³33 Cutrín O, Kulis S, Maneiro L, MacFadden I, Navas MP, Alarcón D, Gómez-Fraguela JA, Villalba C, Marsiglia FF. Effectiveness of the Mantente REAL Program for Preventing Alcohol Use in Spanish Adolescents. Psychosoc Interv 2021; 30(3):113-122., use of alcohol²⁵25 Marsiglia FF, Kulis SS, Booth JM, Nuño-Gutierrez BL, Robbins DE. Long-Term Effects of the keepin' it REAL Model Program in Mexico: Substance Use Trajectories of Guadalajara Middle School Students. J Prim Prev 2015; 36(2):93-104.^,2626 Kulis SS, Garcia-Perez H, Marsiglia FF, Ayers SL. Testing a Culturally Adapted Youth Substance Use Prevention Program in a Mexican Border City: Mantente REAL. Subst Use Misuse 2021; 56(2):245-257.^,2828 Kulis SS, Marsiglia FF, Porta M, Arévalo Avalos MR, Ayers SL. Testing the keepin' it REAL Substance Use Prevention Curriculum Among Early Adolescents in Guatemala City. Prev Sci 2019; 20(4):532-543.^,2929 Kulis S, Marsiglia FF, Elek E, Dustman P, Wagstaff DA, Hecht ML. Mexican/Mexican American Adolescents and keepin' it REAL: An Evidence-Based Substance Use Prevention Program. Child Sch 2005; 27(3):133-145.^,3131 Kulis S, Yabiku ST, Marsiglia FF, Nieri T, Crossman A. Differences by Gender, Ethnicity, and Acculturation in the Efficacy of the keepin' it Real Model Prevention Program. J Drug Educ 2007; 37(2):123-144.^,3333 Cutrín O, Kulis S, Maneiro L, MacFadden I, Navas MP, Alarcón D, Gómez-Fraguela JA, Villalba C, Marsiglia FF. Effectiveness of the Mantente REAL Program for Preventing Alcohol Use in Spanish Adolescents. Psychosoc Interv 2021; 30(3):113-122., cigarettes²²22 Hecht ML, Shin Y, Pettigrew J, Miller-Day M, Krieger JL. Designed Cultural Adaptation and Delivery Quality in Rural Substance Use Prevention: an Effectiveness Trial for the Keepin' it REAL Curriculum. Prev Sci 2018; 19(8):1008-1018.

23 Kulis SS, Ayers SL, Harthun ML. Substance Use Prevention for Urban American Indian Youth: A Efficacy Trial of the Culturally Adapted Living in 2 Worlds Program. J Prim Prev 2017; 38(1-2):137-158.^-2424 Marsiglia FF, Booth JM, Ayers SL, Nuño-Gutierrez BL, Kulis S, Hoffman S. Short-Term Effects on Substance Use of the Keepin' It REAL Pilot Prevention Program: Linguistically Adapted for Youth in Jalisco, Mexico. Prev Sci 2014; 15(5):694-704.^,2828 Kulis SS, Marsiglia FF, Porta M, Arévalo Avalos MR, Ayers SL. Testing the keepin' it REAL Substance Use Prevention Curriculum Among Early Adolescents in Guatemala City. Prev Sci 2019; 20(4):532-543.^,3131 Kulis S, Yabiku ST, Marsiglia FF, Nieri T, Crossman A. Differences by Gender, Ethnicity, and Acculturation in the Efficacy of the keepin' it Real Model Prevention Program. J Drug Educ 2007; 37(2):123-144., marijuana²²22 Hecht ML, Shin Y, Pettigrew J, Miller-Day M, Krieger JL. Designed Cultural Adaptation and Delivery Quality in Rural Substance Use Prevention: an Effectiveness Trial for the Keepin' it REAL Curriculum. Prev Sci 2018; 19(8):1008-1018.^,2525 Marsiglia FF, Kulis SS, Booth JM, Nuño-Gutierrez BL, Robbins DE. Long-Term Effects of the keepin' it REAL Model Program in Mexico: Substance Use Trajectories of Guadalajara Middle School Students. J Prim Prev 2015; 36(2):93-104.^,2626 Kulis SS, Garcia-Perez H, Marsiglia FF, Ayers SL. Testing a Culturally Adapted Youth Substance Use Prevention Program in a Mexican Border City: Mantente REAL. Subst Use Misuse 2021; 56(2):245-257.^,2828 Kulis SS, Marsiglia FF, Porta M, Arévalo Avalos MR, Ayers SL. Testing the keepin' it REAL Substance Use Prevention Curriculum Among Early Adolescents in Guatemala City. Prev Sci 2019; 20(4):532-543.^,2929 Kulis S, Marsiglia FF, Elek E, Dustman P, Wagstaff DA, Hecht ML. Mexican/Mexican American Adolescents and keepin' it REAL: An Evidence-Based Substance Use Prevention Program. Child Sch 2005; 27(3):133-145., and other illicit drugs in the last 30 days²⁶26 Kulis SS, Garcia-Perez H, Marsiglia FF, Ayers SL. Testing a Culturally Adapted Youth Substance Use Prevention Program in a Mexican Border City: Mantente REAL. Subst Use Misuse 2021; 56(2):245-257.^,2727 Kulis SS, Marsiglia FF, Medina-Mora ME, Nuño-Gutiérrez BL, Corona MD, Ayers SL. Keepin' It REAL-Mantente REAL in Mexico: a Cluster Randomized Controlled Trial of a Culturally Adapted Substance Use Prevention Curriculum for Early Adolescents. Prev Sci 2021; 22(5):645-657.. Positive results of the 7th-grade curriculum were also evidenced among those young people who, before starting the intervention, already reported using alcohol and/or other drugs²⁷27 Kulis SS, Marsiglia FF, Medina-Mora ME, Nuño-Gutiérrez BL, Corona MD, Ayers SL. Keepin' It REAL-Mantente REAL in Mexico: a Cluster Randomized Controlled Trial of a Culturally Adapted Substance Use Prevention Curriculum for Early Adolescents. Prev Sci 2021; 22(5):645-657.^,3232 Kulis S, Nieri T, Yabiku S, Stromwall LK, Marsiglia FF. Promoting Reduced and Discontinued Substance Use among Adolescent Substance Users: Effectiveness of a Universal Prevention Program. Prev Sci 2007; 8(1):35-49.^,3333 Cutrín O, Kulis S, Maneiro L, MacFadden I, Navas MP, Alarcón D, Gómez-Fraguela JA, Villalba C, Marsiglia FF. Effectiveness of the Mantente REAL Program for Preventing Alcohol Use in Spanish Adolescents. Psychosoc Interv 2021; 30(3):113-122., as well as among those who presented risky behaviors²⁶26 Kulis SS, Garcia-Perez H, Marsiglia FF, Ayers SL. Testing a Culturally Adapted Youth Substance Use Prevention Program in a Mexican Border City: Mantente REAL. Subst Use Misuse 2021; 56(2):245-257..

Of the three studies that evaluated the kiR effect on violence outcomes²³23 Kulis SS, Ayers SL, Harthun ML. Substance Use Prevention for Urban American Indian Youth: A Efficacy Trial of the Culturally Adapted Living in 2 Worlds Program. J Prim Prev 2017; 38(1-2):137-158.^,2828 Kulis SS, Marsiglia FF, Porta M, Arévalo Avalos MR, Ayers SL. Testing the keepin' it REAL Substance Use Prevention Curriculum Among Early Adolescents in Guatemala City. Prev Sci 2019; 20(4):532-543., the only version that showed positive results was MR, which proved to be effective in reducing bullying episodes²⁷27 Kulis SS, Marsiglia FF, Medina-Mora ME, Nuño-Gutiérrez BL, Corona MD, Ayers SL. Keepin' It REAL-Mantente REAL in Mexico: a Cluster Randomized Controlled Trial of a Culturally Adapted Substance Use Prevention Curriculum for Early Adolescents. Prev Sci 2021; 22(5):645-657..

The studies that carried out comparisons between the adapted versions and the originals showed that using the adapted version is always more effective²²22 Hecht ML, Shin Y, Pettigrew J, Miller-Day M, Krieger JL. Designed Cultural Adaptation and Delivery Quality in Rural Substance Use Prevention: an Effectiveness Trial for the Keepin' it REAL Curriculum. Prev Sci 2018; 19(8):1008-1018.^,2323 Kulis SS, Ayers SL, Harthun ML. Substance Use Prevention for Urban American Indian Youth: A Efficacy Trial of the Culturally Adapted Living in 2 Worlds Program. J Prim Prev 2017; 38(1-2):137-158.^,2727 Kulis SS, Marsiglia FF, Medina-Mora ME, Nuño-Gutiérrez BL, Corona MD, Ayers SL. Keepin' It REAL-Mantente REAL in Mexico: a Cluster Randomized Controlled Trial of a Culturally Adapted Substance Use Prevention Curriculum for Early Adolescents. Prev Sci 2021; 22(5):645-657..

Quality evaluation

Information on the quality of the studies is specified in Chart 2. Although no study was classified as low risk in all five domains, only seven out of the 17 identified articles (42%) had none or only one of the five domains at low risk of bias. The risk of reporting selective results was not clear for all articles since most studies do not report having previously published the protocol. All articles presented most items as high risk or uncertain risk of bias, that is, four or more items evaluated were classified as high and/or uncertain risk of bias. Only three articles showed a low risk of bias regarding the generation of an adequate allocation sequence.

Discussion

PROERD is the most widespread drug and violence prevention program in Brazilian schools, mobilizing police officers from all states and occupying a prime space in the school curriculum. The neutral results of the current curriculum in preventing drug use¹⁰10 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413., as well as the difficulty of synthesis in relation to the various evaluation studies published on kiR, the curriculum that gave rise to PROERD, motivated this systematic review of the literature on the kiR effectiveness in preventing drug use and violence. The systematic review of the 17 articles that evaluated the kiR effect on preventing drug use and/or violence, by means of RCTs, showed that: 1 - Almost all of the identified articles evaluated the effect of kiR applied by teachers, but not the D.A.R.E.-kiR version applied by police officers; 2 - Most RCTs evaluate different kiR versions, and only the Mexican Mantente Real has more than one effectiveness evaluation, with different samples; 3 - Adapted kiR versions are more effective when compared with the originals; 4 - Consistent favorable effects were found only in the curriculum aimed at 7^th-grade students and only for drug use; 5 - No study showed success of the curriculum aimed at 5^th-grade students, and only one RCT was conducted in addition to the Brazilian one to assess the effectiveness of this curriculum; 6 - Most articles report findings from studies carried out in the US; and 7 - All identified articles have one of the program developers among the authors.

The first finding to be highlighted is the absence of studies that demonstrate the effectiveness of the version corresponding to the curriculum implemented by PROERD (D.A.R.E.-kiR). This finding is in line with a previously published review that draws attention to the high dissemination of the D.A.R.E.-kiR curriculum in American schools with no evidence of effectiveness³⁴34 Caputi TL, Thomas McLellan A. Truth and D.A.R.E.: Is D.A.R.E.'s new Keepin' it REAL curriculum suitable for American nationwide implementation? Drugs Educ Prev Policy 2017; 24(1):49-57.. kiR has at least 14 different curriculum versions, and most of them do not have more than one evaluative study carried out with different samples, which makes it difficult to generalize the results to different samples. Thus, it seems inappropriate the dissemination of PROERD using as evidence of success the kiR curricula different from the same version that has been used in Brazil: the one adapted from the American D.A.R.E. (D.A.R.E.-kiR). D.A.R.E.-kiR is implemented by uniformed police officers, while all other evaluated versions (reported in this review) are implemented by teachers and other school professionals; the influence of police officers in drug use prevention is still not entirely clear, and further studies in this area are required³⁵35 El-Khatib Z, Herrera C, Campello G, Mattfeld E, Maalouf W. The Role of Law Enforcement Officers/Police in Drug Prevention within Educational Settings - Study Protocol for the Development of a Guiding Document Based on Experts' Opinions. Int J Environ Res Public Health 2021; 18(5):2613.. It is important to mention that there is a published study reporting data on the evaluation of D.A.R.E.-kiR effectiveness in American schools; however, it did not meet the criteria to be analyzed in this review for being a quasi-experimental study, which evaluated the program effect on secondary outcomes (not drug use and/or violence). Results showed promising D.A.R.E.-kir effects on resistance to peer pressure, confidence in explaining cigarette refusal, and knowledge and responsible decision-making skills in elementary school students³⁶36 Day LE, Miller-Day M, Hecht ML, Fehmie D. Coming to the new D.A.R.E.: A preliminary test of the officer-taught elementary keepin' it REAL curriculum. Addict Behav 2017; 74(May):67-73..

The fact that kiR has many versions shows the relevance of adapting the program to the reality where it is implemented. Furthermore, studies that compared program versions not culturally adapted with adapted versions clearly showed the importance of cultural adaptation for the program to achieve the expected effects²²22 Hecht ML, Shin Y, Pettigrew J, Miller-Day M, Krieger JL. Designed Cultural Adaptation and Delivery Quality in Rural Substance Use Prevention: an Effectiveness Trial for the Keepin' it REAL Curriculum. Prev Sci 2018; 19(8):1008-1018.^,2323 Kulis SS, Ayers SL, Harthun ML. Substance Use Prevention for Urban American Indian Youth: A Efficacy Trial of the Culturally Adapted Living in 2 Worlds Program. J Prim Prev 2017; 38(1-2):137-158.^,2727 Kulis SS, Marsiglia FF, Medina-Mora ME, Nuño-Gutiérrez BL, Corona MD, Ayers SL. Keepin' It REAL-Mantente REAL in Mexico: a Cluster Randomized Controlled Trial of a Culturally Adapted Substance Use Prevention Curriculum for Early Adolescents. Prev Sci 2021; 22(5):645-657.. One of the central kiR pillars is the cultural narrative²2 Gosin M, Marsiglia FF, Hecht ML. keepin' it R.E.A.L.: A Drug Resistance Curriculum Tailored to the Strengths and Needs of Pre-Adolescents of the Southwest. J Drug Educ 2003; 33(2):119-142.^,55 Hecht ML, Krieger JLR. The Principle of Cultural Grounding in School-Based Substance Abuse Prevention. J Lang Soc Psychol 2006; 25(3):301-319., and the activities use examples from the young people’s daily lives to teach drug resistance techniques, which corroborates the need for cultural adaptation in each population where the program is replicated. The science of prevention supports these recommendations, emphasizing that all programs that will be implemented in contexts other than those in which they were originally developed must undergo a cultural assessment process³⁷37 Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, Hartman M, Fernandez ME, Mullen PD. A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci 2018; 13(1):125..

In addition to the existing cultural differences between behaviors related to drug use and the strategies used to resist peer pressure⁵5 Hecht ML, Krieger JLR. The Principle of Cultural Grounding in School-Based Substance Abuse Prevention. J Lang Soc Psychol 2006; 25(3):301-319., it is important to mention the educational differences between the US, where the original curriculum was created, and Brazil. Many activities in the curriculum for 5^th-grade students involve reading, text interpretation and writing skills that are not compatible with the knowledge of Brazilian 5^th-grade students. In the Program for International Student Assessment (PISA), reading performance (which measures the ability to understand, use, and reflect on written texts) shows that Brazil occupies the 59^th position in relation to the USA, which is in the 24^th position³⁸38 Programa Internacional de Avaliação de Estudantes (PISA). Brasil no PISA 2015: análises e reflexões sobre o desempenho dos estudantes brasileiros. São Paulo: Fundação Santillana; 2016.. In the case of PROERD, it seems that there was no cultural adaptation, but only translation into the Brazilian Portuguese language. The lack of cultural adaptation of PROERD to the Brazilian reality was considered the main hypothesis to explain the negative program effects on preventing drug use¹⁰10 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413.^,3939 Valente JY, Sanchez ZM. Short-Term Secondary Effects of a School-Based Drug Prevention Program: Cluster-Randomized Controlled Trial of the Brazilian Version of DARE's Keepin' it REAL. Prev Sci 2022; 23(1):10-23.. Similarly to PROERD, other programs are also only translated into a new language or have superficial adaptations, such as changes in colors and images⁴⁰40 Castro FG, Barrera M, Martinez CR. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci 2004; 5(1):41-45.. These data reflect one of the current challenges in the prevention field: ensure the effectiveness of prevention curricula when implemented in different contexts and produce more rigorous cultural adaptation guidelines and practices³⁷37 Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, Hartman M, Fernandez ME, Mullen PD. A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci 2018; 13(1):125.^,4141 Baumann AA, Powell BJ, Kohl PL, States MU, Penalba V. Cultural Adaptation and Implementation of Evidence-Based Parent-Training: A Systematic Review and Critique of Guiding Evidence. Child Youth Serv Rev 2016; 1(53):113-120..

kiR seems to have consistent favorable effects when applied to 7^th-grade students for the outcomes of use of alcohol, cigarette and marijuana, both for students who did not use drugs and for those who already used drugs or had other risk behaviors (before starting the program). Despite the relevant program results among 7^th-grade students, most studies were carried out in the USA, a fact that also makes it difficult to extrapolate and compare the results for the Brazilian population. Furthermore, as discussed in the previous paragraph, these favorable results are from the program versions adapted to the implementation site reality. This corroborates the need for cultural adaptation of the Brazilian program version. On the other hand, like the Brazilian study, no study showed success in the curriculum aimed at 5^th-grade students. This result raises concern, since the largest population coverage of PROERD is among 5th-grade students, and this is the least evaluated curriculum (only one RCT, in addition to the one conducted in Brazil), and with unfavorable preventive evidence abroad⁸8 Elek E, Wagstaff DA, Hecht ML. Effects of the 5th and 7th Grade Enhanced Versions of the Keepin' it Real Substance Use Prevention Curriculum. J Drug Educ 2010; 40(1):61-79.^,2020 Hecht ML, Elek E, Wagstaff DA, Kam JA, Marsiglia F, Dustman P, Reeves L, Harthun M. Immediate and Short-Term Effects of the 5th Grade Version of the Keepin' it Real Substance Use Prevention Intervention. J Drug Educ 2008; 38(3):225-251., and neutral evidence in Brazil¹⁰10 Sanchez ZM, Valente JY, Gusmões JDP, Ferreira-Junior V, Caetano SC, Cogo-Moreira H, Andreoni S. Effectiveness of a school-based substance use prevention program taught by police officers in Brazil: Two cluster randomized controlled trials of the PROERD. Int J Drug Policy 2021; 98:103413..

With respect to the findings related to the classification of article bias, in general, the articles were classified as having uncertain bias or high risk of bias in most domains, especially in relation to reporting adequate allocation generation, allocation concealment and risk of reporting selective results. Fourteen of the 17 studies did not perform or did not report how the generation of adequate random sequence was performed, which is an essential step in an RCT to ensure comparability between groups and reduce selection bias⁴²42 Wholey J, Hatry H, Newcomer K. Handbook of Practical Program Evaluation. 3a ed. Wholey JS, Hatry HP, Newcomer K, editors. San Francisco: Jossey-Bass Ed; 2010.. Similar results were found in other systematic reviews of school prevention programs¹⁹19 Newton NC, Champion KE, Slade T, Chapman C, Stapinski L, Koning I, Tonks Z, Teesson M. A systematic review of combined student- and parent-based programs to prevent alcohol and other drug use among adolescents. Drug Alcohol Rev 2017; 36(3):337-351., which calls attention to the need for a greater rigor in the implementation of RCTs for evaluation of prevention programs to minimize the risks of bias and increase the possibility of internal validity of the results⁴³43 Solomon P, Cavanaugh MM, Draine J. Randomized controlled trials : design and implementation for community-based psychosocial interventions. New York: Oxford University Press; 2009..

This article has some limitations, mainly with regard to the difficulty of synthesizing the findings of the studies according to the different program versions. Despite the studies reporting the effect of kiR versions culturally adapted for different populations, they all have the same key elements and theoretical framework. In addition, although there were many publications on the program effects, it was found that few studies were conducted to evaluate the same program version, and the vast majority of the program versions were evaluated only once. What happens is that some articles report results of different analyzes from the same population sample. Another important limitation to be discussed refers to the fact that all published evaluations of kiR (except the Brazilian one) have at least one of the program developers among the authors. This fact constitutes a conflict of interest and may imply bias in the results⁴⁴44 Gorman DM. Can we trust positive findings of intervention research? The role of conflict of interest. Prev Sci 2018; 19(3):295-305.. However, this appears to be a common limitation in the drug use prevention program field, as there are relatively few published reviews that do not involve the program developer, and there are few cases where there is complete separation between the program developer, program implementer, and program evaluator⁴⁵45 Gorman DM, Conde E. Conflict of interest in the evaluation and dissemination of "model" school-based drug and violence prevention programs. Eval Program Plann 2007; 30(4):422-429..

Considering that PROERD is today the school prevention program with the greatest capillarity in Brazilian schools and that it will have considerable expansion in the coming years, it is important to invest not only in effectiveness assessments, but also in studies that fill the gaps in knowledge of the Keepin’ it REAL (kiR) effectiveness, the American program that gave rise to the current PROERD curriculum.

The results of this review show that kiR presents evidence of consistent effectiveness in reducing drug use when considering the curriculum implemented in the 7^th grade. However, no evidence was found that kiR works for 5^th-grade students. These findings corroborate the results obtained by the evaluation of PROERD effectiveness regarding the lack of effectiveness for the 5th-grade curriculum, but contradict the findings of this review regarding the null findings of the PROERD curriculum for the 7^th grade. The international evidence regarding the effect of kiR on other populations and the theoretical model of the program highlight the greater program effectiveness when subjected to cultural adaptation, which contrasts with the absence of cultural adaptation of PROERD for the Brazilian population, evidenced by the recent evaluation of the program effectiveness.

References

Publication Dates

History