REVIEW REVISÃO
Evaluation of STD/AIDS prevention programs: a review of approaches and methodologies
Avaliação de programas de prevenção de DST/AIDS: revendo abordagens e metodologias
Marly Marques da CruzI; Elizabeth Moreira dos SantosI; Simone MonteiroII
IEscola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz, Rio de Janeiro, Brasil
IIInstituto Oswaldo Cruz, Fundação Oswaldo Cruz, Rio de Janeiro, Brasil
ABSTRACT
The article presents a review of approaches and methodologies in the evaluation of STD/AIDS prevention programs, searching for theoretical and methodological support for the institutionalization of evaluation and decision-making. The review included the MEDLINE, SciELO, and ISI Web of Science databases and other sources like textbooks and congress abstracts from 1990 to 2005, with the key words: "evaluation", "programs", "prevention", "STD/AIDS", and similar terms. The papers showed a predominance of quantitative outcome or impact evaluative studies with an experimental or quasi-experimental design. The main use of evaluation is accountability, although knowledge output and program improvement were also identified in the studies. Only a few evaluative studies contemplate process evaluation and its relationship to the contexts. The review aimed to contribute to the debate on STD/AIDS, which requires more effective, consistent, and sustainable decisions in the field of prevention.
Sexually Transmitted Diseases; Acquired Immunodeficiency Syndrome; Program Evaluation; Disease Prevention
RESUMO
O artigo apresenta uma revisão de abordagens e metodologias sobre avaliação de programas de prevenção de DST/AIDS, buscando um aporte teórico-metodológico que subsidie a institucionalização da avaliação e a tomada de decisão. A revisão foi realizada nas bases de dados do MEDLINE, SciELO, ISI Web of Science e outras fontes, tais como livros, textos e resumos apresentados em congressos, no período de 1990 a 2005, das palavras-chave: "avaliação", "programas", "prevenção", "DST/AIDS" e termos similares. Nos artigos levantados há uma predominância de estudos avaliativos quantitativos de resultado ou de impacto, do tipo experimental ou quase-experimental. O principal uso da avaliação é o da prestação de contas, embora a produção do conhecimento e a melhoria do programa possam ser identificados nos estudos examinados. Poucos são os estudos avaliativos que contemplam a avaliação de processo e discutem a sua relação com os contextos. Pretendeu-se, com essa revisão, contribuir na tematização e aprofundamento de um debate relevante no cenário das políticas de enfrentamento das DST/AIDS, que requer respostas mais efetivas, consistentes e sustentáveis no campo da prevenção.
Doenças Sexualmente Transmissíveis; Síndrome de Imunodeficiência Adquirida; Avaliação de Programas; Prevenção de Doenças
Introduction
The Brazilian and international context for dealing with the HIV/AIDS epidemic highlights how crucial it is to expand research to back the organization of systems for monitoring and evaluating prevention and control programs and policies. The expansion and diversification of actions to prevent HIV infection and AIDS, both by government and civil society organizations, expresses the need for consistent information on the effectiveness of such measures in order to back decision-making and improve the quality of programs 1,2.
The current challenges for STD/AIDS prevention programs are to prevent new infections, improve the quality of prevention, and care for individuals that are already infected and those presenting the clinical syndrome. Even considering the successful experiences in the area 3,4,5, the programs are centered on behavior change, individual responsibility, and distribution of prevention inputs and are failing to reach the more vulnerable groups that most need them 6.
In general, little progress has been made in the discussion on the theoretical and methodological foundations to orient prevention work, training of program evaluators, and the development of appropriate tools for evaluating STD/AIDS prevention programs. In Brazil, some efforts can be identified in training evaluators, like the recent creation of specialization and in-service Master's courses for evaluation of control programs targeting endemic diseases, with an emphasis on STD/AIDS 7, the creation of centers specializing in evaluation, and nationwide workshops on monitoring and evaluation with a focus on program improvement and aimed at negotiating a common language for evaluation.
The search for more appropriate methodologies for the evaluation of prevention programs focused on the attempt to identify certain approaches and theoretical and methodological models in the respective literature. One such approach was logframe models 8,9 that provide a better description, comparison, and value assessment of the effects, fundamental requirements for any evaluative process. Another key approach was utilization focused evaluation, proposed by Patton 10, characterized by an emphasis on evaluations of programs based on their usefulness, given the focus on program improvement with participation by potential users throughout the entire evaluative process.
Among the existing evaluation approaches, we selected those that examine program implementation analysis, which prioritizes program improvement without overlooking the role of context. Context is thus approached as one of the explanatory dimensions for how the program's inputs, processes, and effects are influenced (and in turn influence) the programs' organizational context. According to Hartz 8, evaluation should not only explain to what extent an outcome results from an intervention and how this outcome can be explained as a function of the level of the intervention's implementation (in light of the quality standards to be achieved), but should also distinguish between the program's effects and those that the evaluation itself can have on the organization.
HIV/AIDS program evaluations, particularly those involving the prevention component, should be centered on risk control and vulnerability factors. They thus evaluate complex, often juxtaposed interventions that involve different levels and strategies. Such evaluations should correspond to models capable of monitoring and explaining changes related to the intervention, incorporating assessment and recommendations.
The literature on STD/AIDS prevention evaluation shows the primacy of the logic of inputs consumption for measuring the prevention's success, translating the effectiveness of actions through a strictly quantitative approach 11. The effects of behavior changes, habits, beliefs, and safe sex practices have received little attention 12. In other words, these studies focus on the measurement of given effects and their relationship to the respective interventions, without due attention to the processes leading to these effects, such as contextualization of the circumstances in which the phenomenon occurs, involving cumulative and slow cultural changes in behaviors and attitudes.
The current study thus discusses common STD/AIDS prevention program evaluation approaches and methodologies in academic output, in order to seek a theoretical and methodological basis for backing the definition of fundamental quality indicators, criteria, and standards to improve preventive practices. The idea is to contribute to the debate on policies for dealing with the HIV/AIDS epidemic, aimed at developing more consistent and sustainable responses and institutionalization of evaluation processes.
Methods
The literature review consulted the MEDLINE, SciELO, ISI Web of Science, and other available databases and bibliographic sources, such as textbooks and congress abstracts from 1990 to 2005. The search logically combined the following key words: "evaluation", "programs", "prevention", "STD/AIDS", and similar terms.
The initial literature search produced 94 references, of which 72 studies were selected. The main challenge for examining the references was to extract the most relevant articles according to the nature and quality of the content under the proposed review.
We thus adopted the following selection criteria: (a) preferentially literature review material on program evaluation; (b) specific discussion on STD/AIDS prevention program evaluation; (c) methodological contribution to the field of STD/AIDS prevention program evaluation. The pertinent articles were summarized following this comparative content analysis.
Conceptual aspects of program evaluation
Recent years have witnessed a growing interest in evaluation of public health policies in the Brazilian Unified Health System (SUS). There have certainly been strides in the field, but there is still widespread controversy and concern resulting from the range and variety of concepts on evaluation, leading to a conceptual and methodological polysemy, making it "mandatory" for managers or researchers to explain their options.
Despite the literature's lack of consensus on the concept of evaluation, authors from various schools share the idea that evaluation always judges the value or merit of the given object 7,8,9,13. Even so, as noted by Nemes 13, many evaluative studies propose to evaluate without the necessary comparative parameters for such a judgment, thus producing "quasi-evaluations". The term refers to descriptive analyses or studies that include the idea of valorization, but fail to issue a judgment.
In the health field, the literature indicates that Donabedian has made one of the main contributions to evaluation, proposing health services evaluation through a model capable of systematizing quality attributes like efficacy, efficiency, effectiveness, optimization, acceptability, and legitimacy 14. The evaluation matrix built by the author is based on a systemic concept and epidemiological and operational indicators of structure, process, and outcome, viewed as orienting the evaluation of the service, system, or program.
Although Donabedian emphasizes that the relationship between structure, process, and outcome is dynamic and functional, the model is rigid and fails to incorporate contexts (institutional, political, or external) that influence the interrelationship between the intervention and its effects. The author's perspective, anchored in health services evaluation, encounters problems when transposed to preventive, education-based interventions.
Patton 10 and those drawing on the theoretical model as the reference for program evaluation 8,9 propose to go beyond the Donabedian approach to the extent that they do not limit themselves to the structure-process-outcome trilogy. Their perspective seeks to give a broader meaning to the judgment, including contextual elements and their role in the program's internal components.
A key reference for evaluation in the field of health is Contrandriopolus et al. 15, contemplating the relationship between intervention and effects and taking the influence of contexts into account. These authors contend that evaluation consists basically of making a value judgment concerning an intervention or service or any component thereof, with the objective of aiding decision-making. The intervention is thus treated as a set of means (physical, human, financial, and symbolic) organized in a specific context, at a given moment, to produce goods and services that will modify a problematic situation 15.
Although agreeing with the idea defended by Contrandriopolus and other authors in relation to the presence of a value judgment and decision-making in the evaluation process, Vieira-da-Silva 16 proposes a review of the concept of intervention suggested by them. In her view, the intervention should be replaced by social practices, with health practice as a particular case. In addition, she does a more in-depth analysis of the meaning ascribed to the notion of judgment, varying from the formulation of a dichotomous (quantitative and qualitative) value judgment to an analysis involving the phenomenon's meaning.
The above author's remarks and suggestions are pertinent, since they involve a reflection on a program's dynamic, interactive, and multifaceted nature, as in the case of STD/AIDS prevention programs, as well as the exercise of ascribing values. After all, such practices relate to different vectors displaying correspondences between congruencies and contingencies in the relationship between intervention and effects.
From a very similar perspective, Patton takes a step forward with utilization-focused evaluation as his theoretical reference 10. The point of departure is that evaluations should be judged according to usefulness and use in what is needed to correct detected distortions. According to the author, evaluation is a participatory process involving the principal actors in all stages of the evaluation, including decision-making concerning the evaluative process itself. An important dimension in this approach is the systematic investigation of how the output of the evaluation is used and which strategies strengthen this utilization.
What this approach presents as a possibility for the evaluation of STD/AIDS prevention programs is the use of evaluation concepts and techniques aimed at the empowerment of those whose programs are being evaluated. In reality, what the author conceives are possibilities for expanding the uses of evaluation by different sectors of society, prioritizing stakeholder participation and empowerment (i.e., potential and priority users in the evaluative process) 10.
Evaluation approaches and models in the evaluation of STD/AIDS prevention programs
Establishing a typology or even identifying evaluation approaches is not any easy task. These approaches are usually identified and classified according to non-systematic criteria, i.e., the classifications combine methodological aspects with the uses, targets, objects, and stakeholders to define and name them.
For the purposes of the current study, the theoretical model of evaluation is seen as an abstract construct that involves: (a) the approach, uses, and focus of the evaluation; (b) the evaluation design; and (c) the modalities used in disseminating the findings. Approaches are defined as the strategies related to the level and modalities of involvement of potential users in the evaluative process. Uses entail the possible utilization of the findings, or accountability, improvement in the intervention, and knowledge output. Focus of the evaluation is the structural component of the program to be examined, i.e., process, outcome, and impact. The predicted combination of use and focus determines the evaluative question and the design possibilities to be considered. The design includes the sampling plan when relevant and the data collection and analysis techniques. The theoretical model includes the ways that the evaluation findings are disseminated.
For example, the utilization-focused approach provides for user participation during all moments of the evaluative process 10. This approach can be applied to evaluations with different types of uses, but it is especially recommended in evaluations focused on program improvement and institutional development. Consistently, this approach is most appropriate for process studies for which case studies would be the prime design, although other designs can be used for the same approach.
The last decade has witnessed the consolidation of two major trends in the field of program evaluation that intervene in the production of studies in the field of STD/AIDS prevention. The first involves the incorporation of methodological procedures from scientific research based primarily on quasi-experimental and observational study designs 17. The focus of these studies is to establish a causal relationship between intervention and effect. The jargon itself changes progressively, and the expression "evaluation plan" is replaced by "evaluation theoretical model" in the designs. The latter expression differs from the former vis-à-vis the replacement of normative evaluations, viewed in this context as "scientific", with complex (outcome and impact) evaluations of the evaluative research type, seeking to establish the causal nexuses between intervention and effect.
The second trend includes a set of models prioritizing the evaluation of implementation and examining the intermediate stages in these relations. An important aspect relates to the replacement of studies on production and productivity (coverage and yield) with those approaching implementation with a focus on program improvement through parameters that expand the concept of implementation beyond just services supply, i.e., including access, quality, and comprehensiveness indicators in the analyses 18.
A review of intervention research and evaluation of HIV/AIDS prevention programs by Peersman & Rugg 19 analyzed 142 studies, mostly using experimental or quasi-experimental designs to evaluate results. One of the main characteristics of these HIV prevention interventions was the predominance of the individual behavior component, while few focused on social, political, and structural components.
The studies analyzed by the above authors mostly used pre- and post-intervention group designs and control or comparison groups; multiple-group designs with randomized and non-randomized allocation to the groups. The results included measurements of behavior and results of diseases related to HIV prevention interventions.
These different approaches definitely appear as theoretical and methodological options for the evaluation of various practices, as in the case of STD/AIDS prevention programs, and can meet the interests of financers, managers, and potential users of the programs. However, we chose to concentrate the discussion on implementation analysis with a focus on the improvement of prevention programs, given the gaps and challenges in this field.
Implementation analysis with a focus on the improvement of STD/AIDS prevention programs
Given investments in prevention policies in Brazil, it is urgent to reflect on how the respective programs are implemented. Public policies for STD/AIDS prevention have focused predominantly on transmission of information, individual responsibility, and a risk-based epidemiological approach 11,20. Thus, the basis for STD/AIDS prevention work is still the biomedical intervention based on a reductionist and fragmented rationality, emphasizing the role of inputs (condoms and educational materials) and technistic responses.
From this perspective, it is important to invest in the evaluation of STD/AIDS prevention programs with a view towards their improvement. But why opt for implementation analysis? What is the potential contribution of such evaluations?
In the evaluation of STD/AIDS prevention programs and other areas there is a greater focus on outcome or impact evaluation. In the review by Peersman & Rugg 19, most of the evaluations focused on results, with few including process indicators. Those that did include process indicators were related to the quality of the intervention material, the implemented intervention, and user satisfaction with the service.
In the review by the above authors, although the target population for the majority of the interventions consisted of more vulnerable groups (sex workers, men who have sex with men, IV drug users, etc.), a large share of the analyses were carried out in formal spaces (health services, schools) by professionals in charge of the interventions 19. Considering that, in general, individuals at high risk of HIV infection experience difficulties in accessing these services, there appears to be a gap in access to prevention and care that can increase the vulnerability of these groups.
An evaluation of the impact of an AIDS education program for young adults in six training centers in a city in the United Kingdom focused on the program's effects related to KABP changes (knowledge, attitudes, beliefs, practices) using a pre- and post-test approach with the intervention and control groups 21. The young people in the intervention group improved their level of knowledge and changed their beliefs and attitudes toward condom use. However, they acknowledged that knowledge is necessary but not sufficient for behavior change, especially in community-based interventions.
Both the above-mentioned study and Main et al. 22, focusing on the evaluation of a school-based education program, prove the effectiveness of HIV prevention interventions related to the observed effects in KABP change. However, these studies suggest that such programs should be reformulated in order to incorporate life experiences, a participatory approach, and training in the use of peer methodology. This would involve a change in theoretical premises and the orientation of such actions and their methodologies, pointing to implementation analysis.
Implementation analysis proposes to verify whether the planned intervention was actually executed as scheduled, based on the established or negotiated quality standard. However, in the case of complex interventions consisting of sequential elements with which the context can interact in different ways (for example, STD/AIDS prevention), the relationship between what is planned and what is implemented is neither direct nor linear.
Brazil still has limited academic output involving the evaluation of STD/AIDS prevention programs. As for normative evaluations, according to Camargo Jr. 11, evaluative processes for STD/AIDS prevention activities in Brazil are characterized primarily by counting, i.e., the number of condoms distributed, number of educational materials produced, and/or number of workshops held. Such indicators are important for planning and evaluating the interventions, but their exclusive use does not allow evaluating changes in the sphere of the program's implementation.
Taking another line of argument, Aggleton 23 analyzed the issue of evaluating HIV/AIDS education and health promotion, emphasizing how the logic of success in this area is oriented towards financing. The author highlights that in some cases evaluation is linked to future funding, to subsequent support being conditioned on the success of current activities, while in other cases it is linked to a concern with the results, or achieving the objectives. In this sense, he takes a critical view of the economic implications embedded in the evaluation process, especially in relation to the party demanding the evaluation.
As presented by the above-mentioned author, some evaluative studies of STD/AIDS prevention programs deal primarily with evaluation modalities based on economic rationality 23. Above all, the emphasis on the economic component is anchored in the cost-benefit relationship with no direct relation to effectiveness, which should not only contemplate maximizing quality from the perspective of the program's performance, but also encompass relations between systems, services, and users. Aggleton makes an important contribution by identifying strong points and critical points for the evaluation of HIV/AIDS education and health promotion, but does not provide models or evaluation approaches for this area, which would be highly relevant given the paucity of investment in this direction.
According to the literature, implementation analysis seeks to identify the processes involved in the production of effects by an intervention, relating the program's internal rationality to its context 24,25, without overlooking that it should be sensitive to the political, organizational, and external contexts and should not overlook the analysis of what was planned as compared to what was implemented (the planned versus unintentional effects as well as those actually implemented).
According to Hartz & Vieira-da-Silva 18, implementation or implantation analysis focuses precisely on the relationship between the intervention (policies, programs, services, and actions) and the context in the production of effects, which is particularly important when the intervention is complex (with multiple components) and contingency-based. According to the authors, various technical, ethical, and political factors are related to their organizational success or resistance and are indispensable for measuring and analyzing the effectiveness of such actions.
Some authors also refer to implementation analysis as formative evaluation, since it can be conducted in the initial stages of the program's implementation, or because it is the type of investigation that provides rapid feedback to program managers and financers 17,18. This type of evaluation is generally used to understand the intervention needs and make decisions on how to implement or improve it in order to maximize its success.
Implementation analysis is distinguished from the evaluation of an intervention's outcome or impact based on the "black box experiment" strategy, where the intervention is treated as a dichotomous variable (absence versus presence of the intervention), aimed at summative objectives. The prime issue in this type of evaluation lies in the absence of theoretical orientation concerning the intervention under evaluation, as well as the absence of explanatory factors leading to such effects.
In this case, implementation analysis allows one to "open the black box", to the extent that it helps clarify whether the observed effects have actually resulted from program itself. "Opening the black box" explains the set of factors influencing the effects obtained from an intervention and allows formulating recommendations to improve it 18.
Implementation analysis thus relates directly to the capacity to utilize the results of evaluative research to make decisions on how to improve the program and enhance the possibility of generalizing an intervention to other contexts, because depending on how it is processed, one is able to retrieve the wealth of relations between the intervention as conceived theoretically and its transposition to the real world.
An evaluative study by Antunes et al. 26 on the impact of a program designed to encourage young people to think about their sexuality and AIDS prevention focused on decision-making and sexual prevention linked to reproductive choices and the gender context. However, the research instrument failed in its attempt to evaluate this link. The study reaches pertinent and consistent conclusions as to the effects obtained, but it does not identify the evaluation question or standards for comparison and fails to make a judgment on the program's value or merit.
The study by Paiva 27 is a process evaluation whose object of interest is an AIDS prevention educational project in public schools in São Paulo. One of the results relates to the need for condoms to be more accessible to young people, with better planning of distribution in appropriate places to ensure a direct and continuous supply. The author also highlights other organizational, subjective, and inter-subjective aspects for ensuring access to quality actions in STD/AIDS prevention.
The results suggest that although young people know how to prevent STD/AIDS, there is a gap between knowledge and practice, given different variables that appear as determinants of the choice to adopt safer sex practices. According to the study's findings 26,27, condoms are used regularly in sporadic encounters or at the beginning of a relationship, but over time a feeling of immunity is expressed as a demonstration of trust in the other partner.
Another Brazilian experience in this field refers to the evaluation of STD/AIDS prevention activities and drug abuse in primary and secondary schools in 14 Brazilian State capitals 28. The study was based on a qualitative methodology in order to conduct a broad evaluation (of the process, outcome, and impact). The study suffers from methodological shortcomings, since it fails to specify the evaluation plan with a description of the intervention, evaluative questions, standards, and expected uses of the evaluation. Although the purpose is specified, namely to conduct a process, outcome, and impact evaluation, the effort focused more on the diagnosis than on how the evaluation was conducted, so it was thus closer to what we referred to previously as a quasi-evaluation (which, by the way, is typical of most of the evaluation studies we analyzed).
Gomes et al. 29, evaluating AIDS-related information and values among schoolchildren in three Brazilian cities in 2000 and 2001, demonstrated that the Cuidar ("Caring") Program, the target intervention, had no effect on the level of AIDS-related information, while identifying positive aspects related to the capacity for reflection and argumentation by these young people on the ways HIV is transmitted. The attempt was to conduct an evaluation of the implementation process and results of a program using a quasi-experimental design, without defining an evaluation plan. The data show that the actions may not directly reach the level of information (which was already high at baseline) but can affect young people's attitudes and behaviors towards the problem. According to the authors, improving the quality of prevention activities in the group requires incorporating them into the promotion of self-esteem and self-care as a way of obtaining better results.
Such studies generally show that the main challenge for implementation analysis lies in the role of internal evaluators and their capacity to define evaluation parameters and incorporate factors from the socio-historical institutional context in which the actions take place. Inclusion of the context marks the cleavage between evaluations inserted in the social world with a commitment to change in social situations, and those that are involved merely in technical and operational changes.
In this sense, one of the main contributions by Mann et al. 30 was to list some basic requirements for success in STD/AIDS prevention activities: information and education; social and health services; and an adequate social environment. By systematizing a program's capacity to reduce the vulnerability to HIV/AIDS both at the functional and structural levels, the authors defined parameters that can guide the formulation of the indices to be analyzed, corresponding to: (a) the commitment agenda; (b) implementation of the commitment agenda, planning, negotiation, management, response to the needs for prevention and treatment, fundraising, sustained effort, and progress and impact evaluation; and (c) transformation of the activity into quality action.
Along this line of analysis, Castiel 31, based on research focusing on health education experiences in AIDS prevention, concluded that information and education can only be effective if coupled with the other two requirements mentioned by Mann et al. 30, given that the perspectives are hardly promising in contexts with precarious social and health services and an inadequate social environment.
The above perspective calls attention to the influence of organizational and external contexts on the intervention and consequently on the effects produced by a prevention program. The greater the social, economic, and institutional support, the better the odds of adopting safer sex practices and reducing vulnerability to HIV/AIDS 30. STD/AIDS prevention activities must thus entail an effort at incorporating these different contexts into the analysis.
This challenge thus involves including evaluation models and approaches related to the strategic dimensions of transmissible disease control, i.e., interruption of transmission, case management, and promotion of quality of life. Incorporation of quality of life necessarily links the evaluation to positive health concepts, identifying this positive side with its own conceptual identity, beyond mere opposition to disease or risks. This approach allows considering not only biomedical effects, but also those related to the fight against discrimination and for human rights and diversity and the reduction of inequalities.
Final remarks
This article aimed to raise food for critical thought on the advances and challenges in health program evaluation approaches and models, particularly for STD/HIV/AIDS prevention programs. It focused on the relevance of process monitoring and evaluation for improving the quality of activities in this area, with health promotion as the prime object of interventions for promoting healthier sexual and reproductive practices.
Based on the literature review, the effort to institutionalize evaluation in this area requires the incorporation of new meanings and practices in management tools and the introduction of qualitative frameworks capable of overcoming the primacy of the quantitative (essentially epidemiological) approach to evaluation, while dealing with the dynamics and complex processes involved in analyzing issues like quality of life, access to quality prevention actions, and responsiveness.
All of the above requires a review of concepts and practices in services, programs, and projects towards a kind of rationality compatible with the purposes of monitoring and evaluation, based on scientific and empirical evidence emerging from the context of implementing STD/AIDS prevention activities. This possibility requires both a reconfiguration of the value ascribed to the quality of information in the available records and the use of information generated by evaluative studies with a focus on utilization and program improvement.
The pertinence of this orientation is expressed in evaluation's potential as an important management tool, capable of producing information to guide health actions, indicating where problems have occurred and how the process can be altered to seek the most appropriate effects. After all, to evaluate ultimately presupposes to change, to improve in keeping with the negotiated quality standards.
Given the above, we suggest that implementation analysis of STD/AIDS prevention programs allows more adequate orientation of supply and demand, in keeping with the target population's needs. We thus contend that proposals in this direction can contribute to a systematic approach, capable of guiding decision-making and improving such interventions, based on health practice with greater justice, equity, and quality.
Contributors
M. M. Cruz collaborated in the survey, literature review, and structuring and drafting of the paper. E. M. Santos and S. Monteiro collaborated in the draft and final review of the paper.
References
1. Rugg D, Caräel M, Baerma JT, Novak J. Global advances in monitoring and evaluation of HIV/AIDS: from AIDS case reporting to program improvement. In: Rugg D, Peersman G, Caräel M, editors. Global advances in HIV/AIDS monitoring and evaluation: new directions for evaluation, no. 103. San Francisco: Jossey-Bass; 2004. p. 33-48.
2. Greet P, Rugg D. Intervention research and program evaluation: the need to move beyond monitoring. In: Rugg D, Peersman G, Caräel M, editors. Global advances in HIV/AIDS monitoring and evaluation: new directions for evaluation, no. 103. San Francisco: Jossey-Bass; 2004. p. 141-58.
3. Kanchanachitra C. Best practice in reduction of vulnerability of girls to HIV/AIDS. Warasan Prachakon Lae Sangkhom 1998; 7:53-83.
4. Peersman GV, Levy AJ. Focus and effectiveness of HIV-prevention efforts for young people. AIDS 1998; 12 Suppl A:S191-6.
5. Moraes DR. Caracterização da proposta pedagógica de prevenção em saúde sexual e reprodutiva de educadores de uma organização não-governamental do Município do Rio de Janeiro Brasil [Dissertação de Mestrado]. Rio de Janeiro: Escola Nacional de Saúde Pública, Fundação Oswaldo Cruz; 2004.
6. Parker R, Camargo Jr. KR. Pobreza e HIV/AIDS: aspectos antropológicos e sociológicos. Cad Saúde Pública 2000; 16 Suppl 1:89-102.
7. Santos EM, Natal S, organizadoras. Especialização em avaliação de programas de controle de processos endêmicos, com ênfase em DST/HIV/AIDS. Rio de Janeiro: Editora Fiocruz; 2006.
8. Hartz ZMA, organizadora. Avaliação de saúde: dos modelos conceituais à prática na análise de implantação de programas. Rio de Janeiro: Editora Fiocruz; 1997.
9. Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR Recomm Rep 1999; 48 (RR-11):1-40.
10. Patton MQ. Utilization focused evaluation: the news century text. 3rd Ed. Thousand Oaks: Sage Publications; 1997.
11. Camargo Jr. KR. Políticas públicas e prevenção em HIV/AIDS. In: Parker R, Galvão J, Bessa MS, organizadores. Saúde, desenvolvimento e política: respostas frente à AIDS no Brasil. Rio de Janeiro: Associação Brasileira Interdisciplinar de AIDS/São Paulo: Editora 34; 1999. p. 227-62.
12. Mertens T, Caräel M, Sato P, Cleland J, Ward H, Smith GD. Prevention indicators for evaluating the progress of national AIDS programmes. AIDS 1994; 8:1359-69.
13. Nemes MIB. Avaliação em saúde: questões para os programas de DST/AIDS no Brasil. Rio de Janeiro: Associação Brasileira Interdisciplinar de AIDS; 2001.
14. Donabedian A. An introduction to quality assurance in health care. New York: Oxford University Press; 2003.
15. Contandriopoulos AP, Champagne F, Denis JL, Pineault R. A avaliação na área da saúde: conceitos e métodos. In: Hartz ZMA, organizadora. Avaliação de saúde: dos modelos conceituais à prática na análise de implantação de programas. Rio de Janeiro: Editora Fiocruz; 1997. p. 29-47.
16. Vieira-da-Silva LM. Conceitos, abordagens e estratégias para avaliação. In: Hartz ZMA, Vieira-da-Silva LM, organizadores. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Salvador: EDUFBA/Rio de Janeiro: Editora Fiocruz; 2005. p. 15-39.
17. Coyle SL, Boruch RF, Turner CF, editors. Evaluating AIDS prevention programs expanded edition. Washington DC: National Academies Press; 1991.
18. Hartz Z, Vieira-da-Silva LM, organizadoras. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Salvador: EDUFBA/Rio de Janeiro: Editora Fiocruz; 2005.
19. Peersman GV, Rugg D. Intervention research and program evaluation: the need to move beyond monitoring. In: Rugg D, Peersman G, Caräel M, editors. Global advances in HIV/AIDS monitoring and evaluation: new directions for evaluation, no. 103. San Francisco: Jossey-Bass; 2004. p. 141-58.
20. Monteiro S. Qual prevenção? AIDS, sexualidade e gênero em uma favela carioca. Rio de Janeiro: Editora Fiocruz; 2002.
21. Bellingham K, Gillies P. Evaluation of an education programme for young adults. J Epidemiol Community Health 1993; 47:134-8.
22. Main DS, Iverson DC, McGloin J, Banspach SW, Collins JL, Rugg DL, et al. Preventing HIV infection among adolescents: evaluation of a school-based education program. Prev Med 1994; 23:409-17.
23. Aggleton P. Monitoramento e avaliação de educação em saúde e promoção da saúde voltada para o HIV/AIDS. In: Czeresnia D, Santos EM, Barbosa RHS, Monteiro S, organizadoras. AIDS: pesquisa social e educação. São Paulo: Editora Hucitec; 1995. p. 193-206.
24. Centers for Disease Control and Prevention. Monitoring & evaluation capacity building for program improvement field guide, version 1. Atlanta: Centers for Disease Control and Prevention; 2003.
25. Denis JL, Champagne F. Análise da implantação. In: Hartz ZMA, organizadora. Avaliação de saúde: dos modelos conceituais à prática na análise de implantação de programas. Rio de Janeiro: Editora Fiocruz; 1997. p. 49-88.
26. Antunes MC, Stall RD, Paiva V, Peres CA, Paul J, Hudes M, et al. Evaluating an AIDS sexual risk reduction program for young adults in public night schools in Sao Paulo, Brazil. AIDS 1997; 11 Suppl 1:S121-7.
27. Paiva V. Fazendo arte com camisinha: sexualidades jovens em tempos de AIDS. São Paulo: Summus; 2000.
28. Abramovay M, Rua MG. Avaliação das ações de prevenção de DST/AIDS e uso indevido de drogas nas escolas de ensino fundamental e ensino médio em capitais brasileiras. Brasília: Organização das Nações Unidas para a Educação, a Ciência e a Cultura/Ministério da Saúde/Joint United Nations Programme on HIV/AIDS/United Nations Office on Drugs and Crime; 2001.
29. Gomes R, Assis SG, Souza ER, Deslandes SF, Njaine K, Malaquias JF. Informações e valores de jovens sobre a AIDS: avaliação de escolares de três cidades brasileiras. Ciênc Saúde Coletiva 2005; 10:381-8.
30. Mann J, Tarantola D, Netter TW. Como avaliar a vulnerabilidade à infecção pelo HIV e AIDS. In: Mann J, Tarantola D, Netter TW, organizadores. A AIDS no mundo. Rio de Janeiro: Relume-Dumará/Associação Brasileira Interdisciplinar de AIDS/Universidade do Estado do Rio de Janeiro; 1993.
31. Castiel LD. Força e vontade: aspectos teórico-metodológicos do risco em epidemiologia e prevenção do HIV/AIDS. Rev Saúde Pública 1996; 30:91-100.
Correspondence:
M. M. Cruz
Departamento de Endemias Samuel Pessoa, Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz.
Rua Jorge Rudge 14, apto. 802, Rio de Janeiro, RJ
20550-220, Brasil.
marly@ensp.fiocruz.br
Submitted on 30/Jun/2005
Final version resubmitted on 17/Jul/2006
Approved on 05/Dec/2006