Abstract

The provision of ARVs is central to HIV/AIDS programs, because of its impact on the course of the disease and on quality of life. Although first-line treatments costs have declined, treatment-associated expenses are steeper each year. Sustainability is therefore an important variable for the success of treatment programs. A conceptual framework on sustainability of ARV provision was developed, followed by data collection instruments. The pilot study was undertaken in Brazil. Bolivia, Peru and Mozambique, were visited. Key informants were identified and interviewed. Investigation of sustainability related to ARV provision involved implementation and routinization events of provision schemes. Evidence of greater sustainability potential was observed in Peru, where provision is implemented and routinized by the National HIV/AIDS program and expenditures met by the government. In Mozambique, provision is dependent on donations and external aid, but the country displays a great effort to incorporate ARV provision and care in routine healthcare activities. Bolivia, in addition to external dependence on financing and management of drug supply, presents problems regarding implementation and routinization. The conceptual framework was useful in recognizing events that influence sustainable ARV provision in these countries.

Program sustainability; HAART; HIV; Aids

Resumo

A provisão de medicamentos ARV é central para programas de HIV/Aids, devido a seu impacto no curso da doença e na qualidade de vida. Embora os custos de tratamentos de primeira linha tenham diminuído, os gastos dos programas com os tratamentos tem aumentado a cada ano. A sustentabilidade torna-se fator fundamental para o sucesso dos programas. Um modelo conceitual para avaliação da sustentabilidade da provisão de ARV e instrumentos de coleta de dados foram desenvolvidos. Um estudo piloto foi realizado no Brasil e a pesquisa de campo cobriu Bolívia, Moçambique e Peru. Informantes-chaves foram identificados e entrevistados. Eventos críticos de implementação e rotinização foram investigados na história dos programas. Foi observado maior potencial para sustentabilidade no Peru, onde a provisão está implementada e rotinizada e os gastos são cobertos pelo governo nacional. Em Moçambique, o financiamento da provisão é quase totalmente dependente de ajuda internacional, mas há grandes esforços voltados à expansão da cobertura e rotinização do cuidado a PVH. Na Bolívia, além da dependência externa para o financiamento há problemas de implementação e gerenciamento da provisão. O modelo avaliativo mostrou-se útil na identificação de fatores que influenciam a capacidade para sustentabilidade dos programas nesses países.

Sustentabilidade de programas; HAART; HIV; Aids

Introduction

The fight against AIDS was set out as a global commitment by the United Nations (UN) Millennium Declaration and in the landmark UN Declaration of Commitment on HIV/AIDS (2001) which recognizes “that access to medication in the context of pandemics such as HIV/AIDS is one of the fundamental elements to achieve…” the realization of the right to health¹1. United Nations (UN). United Nations Millennium Declaration, A/RES/55/2. New York: UN; 2000.,²2. UNAIDS. Declaration of Commitment on HIV/AIDS. Geneva: UNAIDS; 2001.. In 2006, UN Members move towards Universal Access to HIV prevention, treatment and care by 2010. By the end of 2011, around eight million (54%) eligible people living with HIV (PLHIV) were receiving antiretroviral therapy (ART) in low and middle-income countries (LMIC). As a consequence, AIDS-related illnesses and mortality dropped by more than 50%. The 2011 UN High Level Meeting reaffirms previous commitments and establishes, inter alia, the target of treating 15 million PLHIV by 2015³3. Joint United Nations Programme on HIV/AIDS (UNAIDS). World AIDS Day Report 2012 - Results. Geneva: UNAIDS; 2012..

Despite major advances and successes achieved in the last ten years, problems related with access to antiretroviral medicines (ARV) still persist and tend to increase in many developing countries: the maintenance and expansion of long-term financing; reducing early mortality by promptly starting ART and other medications against co-morbidities; implementation of novel 2010 WHO guidelines, which are more comprehensive and include costlier medicines in early treatment stages; improving the management of long-term treatment, which includes access to viral load tests needed for better clinical assessment and progression to second and third-line regimens⁴4. Eholié S-P, Aoussi FE, Ouattara IS, Bissagnéné E, Anglaret X. HIV treatment and care in resource-constrained environments: challenges for the next decade. J Int AIDS Soc 2012; 15:17334..

Although ARV prices have been decreasing over time⁵5. Médecins Sans Frontières (MSF). Untangling the web of antiretroviral price reductions. Geneve: MSF; 2011.

6. Meiners C, Sagaon-Teyssier L, Hasenclever L, Moatti J-P. Modeling HIV/AIDS Drug Price Determinants in Brazil: Is Generic Competition a Myth? PLoS ONE 6(8):e23478.-⁷7. Médecins Sans Frontières (MSF). Untangling the web of antiretroviral price reductions. Geneve: MSF; 2013., population treatment-associated expenses are steeper each year, especially in face of new and costly medicines incorporated in treatment guidelines, most of which present very limited competition or none at all⁸8. Hoen ET, Berger J, Calmy A, Moon S. Driving a decade of change: HIV/AIDS, patents and access to medicines for all. J Int AIDS Soc 2011; 14:15.. According to Hoen et al.⁸8. Hoen ET, Berger J, Calmy A, Moon S. Driving a decade of change: HIV/AIDS, patents and access to medicines for all. J Int AIDS Soc 2011; 14:15. the “policy space to produce or import generic versions (…) is shrinking” in developing countries, due to patent regimen pressures.

Provision of antiretroviral medicines includes procurement, supply chains, and operational and information systems, in order to ensure high quality services. Upscaling AIDS care and prevention is a complex and dynamic process that depends upon countries’ capacity to adopt new interventions and assess interactions among the interventions’ key components as well as between programs and health systems⁹9. Atun R, Bataringaya J. Building a Durable Response to HIV/AIDS: Implications for Health Systems. J Acquir Immune Defic Syndr 2011; 57(Supl. 2):S91-S95..

This transition from emergency AIDS response to long-term response of large treatment cohorts requires health systems strengthening⁹9. Atun R, Bataringaya J. Building a Durable Response to HIV/AIDS: Implications for Health Systems. J Acquir Immune Defic Syndr 2011; 57(Supl. 2):S91-S95. and regular access to ART⁵5. Médecins Sans Frontières (MSF). Untangling the web of antiretroviral price reductions. Geneve: MSF; 2011.. Sustainability of ART provision has been pointed out as a major concern⁹9. Atun R, Bataringaya J. Building a Durable Response to HIV/AIDS: Implications for Health Systems. J Acquir Immune Defic Syndr 2011; 57(Supl. 2):S91-S95.

10. Sivaram S, Celentano DD. Training outreach workers for AIDS prevention in rural India: is it sustainable? Health Policy and Planning 2003; 18(4):411-420.

11. Rowe L, Rechel B. Fighting tuberculosis and HIV/AIDS in Northeast Europe: sustainable collaboration or political rhetoric? Eur J Public Health 2006; 16(6):609-614.-¹²12. Greco DB, Simão M. Brazilian policy of universal access to AIDS treatment: sustainability challenges and perspectives. AIDS 2007; 21(Supl. 4):S37-S45..

Schell et al.¹³13. Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, Bunger AC. Public health program capacity for sustainability: a new framework. Implementation Science 2013; 8(15):1-9. point out that a minority of sustainability studies makes attempts at conceptualizing sustainability and at developing assessment tools accordingly. Stirman et al.¹⁴14. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implementation Science 2012; 7(17):1-19. argue that the absence of a working definition of sustainability and of guidance by an explicit theoretical model is an important limitation to the body of research. Scheirer¹⁵15. Scheirer MA. Linking Sustainability Research to Intervention Types. Am J Public Health 2013; 103(4):e73-80. suggests that the use of conceptual frameworks¹⁶16. Chen HT. Theory-driven evaluations. Newbury: Park Sage Publications; 1990., as well as in-depth assessments on program implementation, could help overcome limitations in sustainability studies.

The objective of this study was to challenge a conceptual framework applied to assess ARV provision sustainability in Peru, Bolivia and Mozambique.

Methodology

Sustainability and related concepts

Sustainability may be interpreted in various ways, but is closely related to financial and organizational aspects, both of which depend on the commitment or political will of governments¹⁷17. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education Research 1998; 13(1):87-108.. They involve the ability of a system to acquire the financial resources needed to fund programs, provide products, organize services and manage all non-financial resources¹⁸18. Scheirer MA. Is Sustainability Possible? A Review and Commentary on Empirical Studies of Program Sustainability. Am J Eval 2005; 26(3):320-347.,¹⁹19. Bossert TJ. Can they get along without us? Sustainability of donor-supported health projects in Central America and Africa. Soc Sci Med 1990; 30(9):1015-1023..

From an organizational point of view, Pluye et al.²⁰20. Pluye P, Potvin L, Denis J-L. Making public health programs last: conceptualizing sustainability. Evaluation and Program Planning 2004; 27(2):121-133. propose three aspects of sustainability that integrate the program with the organizational context and the political environment. The first aspect is implementation. According to Mazmanian and Sabatier²¹21. Mazmanian D, Sabatier PA. A Framework for implementation analysis. In: Mazmanian D, Sabatier PA, editors. Implementation and public policy. Boston, London: University Press of America; 1989. p. 18-48. implementation is the carrying out of a basic policy decision, usually incorporated in a statute but which can also take the form of important executive orders or court decisions. Denis and Champagne²²22. Denis J-L, Champagne F. Análise da implantação. In: Hartz ZMA, editor. Avalliação em Saúde: dos modelos conceituais à prática na análise de implantação de programas. Rio de Janeiro: Fiocruz; 1997. p. 49-88. define implementation as the extension of the operational capacity of an intervention or the transfer of an intervention at an operational level; while for Love²³23. Love A. Implementation Evaluation. In: Wholey JS, Hatry HP, Newcomer KE, editors. Handbook of practical program evaluation. 2nd ed. San Francisco: Jossey-Bass; 2004. p. 63-97. implementation is defined as activities focused on the carrying out of programs.

The second is routinization, which refers to the process that leads to the establishment of routines within the activities required by the program to achieve its objectives. Routines are operational procedures integrated into the organization, which reflect shared know-how or memory, exhibit adaptation to the context, convey values and other cultural artifacts, and adhere to rules such as plans and procedural manuals²⁰20. Pluye P, Potvin L, Denis J-L. Making public health programs last: conceptualizing sustainability. Evaluation and Program Planning 2004; 27(2):121-133..

The third aspect is standardization, which describes the existence of formal, legal, normative or institutional standards that guide the intervention. Standardization of established routines helps to simplify and disseminate them while at the same time introducing institutional standards such as guidelines, legislation and policies. Standardization strengthens institutional identity and anchors program sustainability²⁰20. Pluye P, Potvin L, Denis J-L. Making public health programs last: conceptualizing sustainability. Evaluation and Program Planning 2004; 27(2):121-133..

For the purposes of this study, sustainability was defined as an attribute of an intervention, program or policy that emerges from the implementation process by means of the routinization and standardization of a set of durable activities and resources aimed at program-related objectives.

In this perspective, sustainability can be assessed through the identification of critical events, which have built the program history, i.e. elements in temporal sequences constituting particular patterns, which can be explored to analyze underlying processes or mechanisms²⁴24. Pluye P, Potvin L, Denis J-L, Pelletier J, Mannoni C. Program sustainability begins with the first events. Evaluation and Program Planning 2005; 28(2):123-37.,²⁵25. Steel D. Social Mechanisms and Causal Inference. Philosophy of the Social Sciences 2004; 34(1):55-78.. Conceptual events have been described as being associated to greater sustainability, some of which are thought to pertain specifically to: implementation (investment of adequate resources; compatibility of the activities with those of the organization); routinization (resource stabilization; risk-taking); or, both implementation and routinization (incentives; adaptation of activities; objectives fit; transparent communication; sharing cultural artifacts)²⁴24. Pluye P, Potvin L, Denis J-L, Pelletier J, Mannoni C. Program sustainability begins with the first events. Evaluation and Program Planning 2005; 28(2):123-37..

Framework development

The first step for developing the framework was the definition of the following assumptions:

• Regularity of treatment provision is an important factor in the success of treatment programs²⁶26. Hacker MA, Kaida A, Hogg RS, Bastos FI. The first ten years: achievements and challenges of the Brazilian program of universal access to HIV/AIDS comprehensive management and care,1996-2006. Cad Saude Publica 2007; 23(Supl. 3):S345-S59..

• Central coordination is essential to the functioning of a sustainable AIDS program²⁷27. Joint United Nations Programme on HIV/AIDS (UNAIDS). “Three Ones” key principles - “Coordination of National Responses to HIV/AIDS” Guiding principles for national authorities and their partners. Conference Paper 1; Washington Consultation 25.04.042004..

• Issues related to sustainability should be considered during both planning and implementation processes, which are concomitant and continuous¹⁷17. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education Research 1998; 13(1):87-108.,²⁰20. Pluye P, Potvin L, Denis J-L. Making public health programs last: conceptualizing sustainability. Evaluation and Program Planning 2004; 27(2):121-133..

• Program implementation and routinization may be assessed by means of identification and analysis of events throughout the program history²⁴24. Pluye P, Potvin L, Denis J-L, Pelletier J, Mannoni C. Program sustainability begins with the first events. Evaluation and Program Planning 2005; 28(2):123-37..

The building of the framework started with a simple list of research issues to be addressed in investigating achievement of ARV provision programs objectives and of program sustainability in order to determine feasibility in the investigation of sustainability.

In order to integrate theoretical categories (conceptual events) and operational elements, these issues were organized in four separate dimensions: Program Resources; Activities; Reinforcement Strategies; and, Context and Organizational Culture. The resulting framework helped structure data collection and analysis.

Field study and analysis

A pilot study, conducted in Brazil, helped to secure concept formulation (sustainability of ARV provision) within data collection instruments (content validity), and to identify small inconsistencies in the forms as well as to organize the data collection process. The framework was submitted to challenge through semi-structured interviews with program coordinators and other key informants in three countries: Mozambique, Peru and Bolivia. Key informants were selected according to their knowledge of program history, estimated by the level and length of their involvement in the program over time.

All interviewees were contacted personally through E-mail or telephone before country visits. A brief explanation of the nature and purposes of the study was given. If the intended interviewee manifested interest, an executive summary and project information were sent. After preliminary consent, further contact was made to set up appointments. These were followed up once the research team reached the country. Interviews began with a thorough reading of the written consent form in order to eliminate doubts or misunderstandings in regard to data collection, analysis and use of information. Specific authorization was asked for recordings.

Interviews guided identification of publicly available documents such as country bills and legislation on HIV/AIDS; program standards and norms; treatment protocols; planning and evaluation reports and newspaper clippings were collected. These materials were provided by interviewees themselves.

Recordings were transcribed and objective data were entered into a worksheet. Two researchers reviewed each interview and corresponding documentation, and wrote a detailed case description for each country, according to the dimensions: Program Resources; Activities; Reinforcement Strategies; and, Context and Organizational Culture. The entire group revised case descriptions; documentation and transcriptions were double-checked every time any ambiguity or inconsistency was detected, making the corresponding modifications. Then authors examined the case descriptions and sorted out the information (critical events in program history) according to the theoretical categories (conceptual events). Savaya et al.²⁸28. Savaya R, Spiro S, Elran-Barak R. Sustainability of Social Programs: A Comparative Case Study Analysis. Am J Eval 2008; 29(4):478-493. organized their comparative case study analysis on the sustainability of social programs in a similar fashion.

Ethics Statement

Our ethical procedures were carried out after consultation with our IRB and adhered to Ethics in Research Legislation in effect at the time of the study²⁹29. Brasil. Ministério da Saúde (MS). Conselho Nacional de Saúde. Resolução nº 196 de 10 de outubro de 1996. Diretrizes e Normas Regulamentadoras de Pesquisas Envolvendo Seres Humanos. Diário Oficial da União 1996; 16 out.. Key informant interviews referred only to the public history of programs. No personal data was collected. Participation was strictly voluntary and all interviewees gave written informed consent. Furthermore, there was no identification of respondents’ position or name in data collection forms. Confidentiality and anonymity was assured.

Results

Framework

Chart 1 presents the framework as organized as Program Resources, Activities, Reinforcement Strategies and Cultural and Organizational Issues. In each of these dimensions, conceptual events that correspond to program operational elements are listed. These, in turn, are detailed in the topics that were investigated.

Two main events are listed as Program ‘Resources’: adequate investment and stabilization of resources. First, it is necessary to acknowledge the existence of resources, either human, financial or material in nature. Stabilization deals with a constant and uninterrupted ‘flow’ that guarantees the occurrence of trained personnel, funds and structure of provision. In this sense, framework guides data collection towards volume and source of financial resources over time, budget planning, program coverage and human resources profile.

‘Activities’ describe the procedures of ARV provision within the organizational context. Two different events are presented: Compatibility of program activities with those of the organization and adaptation of activities. Compatibility suggests that program activities can be absorbed by the parent organization without any disruption of the routines. If the organization does not carry out activities as required by the intervention, adjustment or adaptation of activities is needed, in order to minimize or eliminate competing routines that may lead to activity failure. Examining adaptation of activities will show if this adjustment is happening easily and with flexibility. Interview issues included a description of the host organizations’ activities profile and of those required for performing ARV provision, and the stakeholders’ assessment of their interrelationship.

‘Reinforcement Strategies’ deal with all initiatives to boost program outcomes or to overcome organizational resistance: incentives, transparent communication, risk taking and integration of rules. As incentives, training, salary increases, bonuses, performance recognition and any form of praise that may encourage program success. On the other hand, overwork, little pay, lack of organizational structure and support may burden staff and discourage performance. Transparent communication helps sustainability by paving a common pathway of exchange between coordinators and program staff and fostering an open environment, avoiding misunderstandings. Risk-taking is also a reinforcement strategy, in the sense that it involves innovation, and because it mobilizes group attitudes and behavior in order to learn, adapt and adopt. By integrating rules, organization and program will ‘work on the same page’. Investigation focused on mapping principal-agent relations, especially those concerned with adhesion to program activities, such as incentive policies, career plans, communication channels and development of program guidelines.

Finally, ‘Context and Organizational Culture’ examine if program rationale coincides with cultural values and beliefs of implementation agents, and can be best described by two events: adjustment of goals and sharing of cultural artifacts. Adjustment of goals deals with the perceived missions of the organization and of the program – identity of objectives and activities. Coordination between program and organization are considered to be necessary for program sustainability. Sharing of cultural artifacts involves identity of values between parent organization and program, embedded into daily routines and activities. Interviews examined the existence of specific access to medicines mechanisms for HIV/Aids, care patterns for vulnerable groups, values and attitudes associated with those patterns.

Country experiences in sustainability of ARV provision

The framework was challenged in Bolivia, Mozambique and Peru. Chart 2 describes critical events identified in program history, and according to the framework dimensions.

Few events marked resource investment in Bolivia. Allocation of resources by the Ministry of Health (MoH) was small and maintenance of resources depended on renewal of grants from The Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM). Changes in program coordination and management, and litigation for access to ARVs were also observed. In Mozambique, although flow of resources for AIDS had varied over the years, the government planned to maintain a steady rise in number of people on treatment. Events that have been associated with this are the existence of a common health fund, the involvement of the Clinton Health Access Initiative (CHAI) and the training of health professionals. In Peru we observed a triple phenomenon: increasingly autonomous ARV financing; renewable employment contract schemes; and gradual increase in ARV coverage (Chart 2).

In Bolivia ARV procurement activities had been historically performed by organizations outside the MoH and their intermittent change was observed. In Mozambique government organizations dealt with all medicines purchases, including ARVs. When one organization was privatized, another absorbed its activities immediately without observed disruption. On the other hand, expansion and decentralization of second-line treatment coverage revealed problems in provincial stock management. In Peru few changes regarding ARV provision activities were observed in program history. These activities were carried out by government organizations, which were also responsible for provision of all other medicines (Chart 2).

In relation to Reinforcement strategies, an innovative effort was made in Bolivia in regard to customs clearance procedures in order to speed up availability of imported ARVs. In Mozambique, treatment standards and guidelines were implemented for all health professionals in the country. Selective hiring and better job conditions for health professionals working specifically with HIV/AIDS were observed, before 2008. After this, despite difficulties, HIV/AIDS care was integrated with general healthcare in the country. In Peru, meetings involving different ARV provision organizations provided opportunities for the sharing of information, and career benefits fostered the desire to work for the program. Innovation was observed by the country’s participation in joint ARV price negotiations rounds in Latin America. We also observed an effort in the passing of legislation establishing parameters for the program to follow (Chart 2).

No events regarding Context and Organizational Culture were identified in Bolivia. Reorganization in the process of care for people living with HIV (PLHIV) in Mozambique led to the closing of Day Hospitals, integrating HIV/AIDS care with that of other diseases. PLHIV under treatment and care providers perceived it as a letdown that damaged the bond between PLHIV and the health system, making way for patient discrimination. In Peru, ARV provision duties were absorbed by organizations that presented institutional missions synchronic with program requirements (Chart 2).

Discussion

The main goal of this study was to challenge a sustainability assessment framework, which involved the identification and hierarchization of key conceptual elements, the building of an operational definition of sustainability, and the translation of those theoretical developments into assessment tools. Guidance by an explicit theoretical framework, adoption of clear working definitions, and use of adequate coherent-to-theory tools are pointed out as critical to further develop sustainability research and evaluation¹³13. Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, Bunger AC. Public health program capacity for sustainability: a new framework. Implementation Science 2013; 8(15):1-9.

14. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implementation Science 2012; 7(17):1-19.-¹⁵15. Scheirer MA. Linking Sustainability Research to Intervention Types. Am J Public Health 2013; 103(4):e73-80..

The choice for research strategy and the selection of factors to be investigated should be determined by the type of intervention. Scheirer¹⁵15. Scheirer MA. Linking Sustainability Research to Intervention Types. Am J Public Health 2013; 103(4):e73-80. proposes six different intervention types, including Interventions Requiring Coordination Among Multiple Staff. Aspects pertaining to the sustainability of this type of intervention comprise: administrative support; the role of program champions; culture and mission compatibility between the intervention and the organization; consonance of specific tasks related to the intervention with other organizational procedures; and continuity of financial resources. This type of intervention as well as the above-mentioned aspects may be recognized in Pluye’s proposal²⁴24. Pluye P, Potvin L, Denis J-L, Pelletier J, Mannoni C. Program sustainability begins with the first events. Evaluation and Program Planning 2005; 28(2):123-37..

Our study’s object was ARV provision, an intervention that requires coordination between multiple organizations, stakeholders, staff etc. As such, our choice of Pluye’s proposal as a reference for building our framework is also supported by Sheirer’s analysis.

The multiple components involved in ARV provision may or may not be recurrent in different scenarios over time. The three countries in which provision was studied presented different income levels, different health system structure, very different levels of HIV prevalence, different forms of organizing HIV/AIDS initiatives, and therefore of ARV provision³3. Joint United Nations Programme on HIV/AIDS (UNAIDS). World AIDS Day Report 2012 - Results. Geneva: UNAIDS; 2012.,³⁰30. Giovanella L, Feo O, Faria M, Tobar S, editors. Sistemas de Salud en Suramérica - desafíos para la universalidad, la integralidad y la equidad. Rio de Janeiro: ISAGS; 2012.

31. The World Bank. The World Development Report online - Income Levels Washington, DC2013 [cited 2013 Nov 18]. Available from: http://wdronline.worldbank.org/worldbank/a/incomelevel.
http://wdronline.worldbank.org/worldbank...

32. Pfeiffer J, Montoya P, Baptista AJ, Karagianis M, Pugas MdM, Micek M, Johnson W, Sherr K, Gimbel S, Baird S, Lambdin B, Gloyd S. Integration of HIV/AIDS services into African primary health care: lessons learned for health system strengthening in Mozambique - a case study. J Int AIDS Soc 2010; 13:3.

33. WHO Regional Office for Africa (WHO/AFRO). WHO Country Cooperation Strategy, 2009-2013 - Mozambique. Brazzaville: WHO/AFRO; 2009.-³⁴34. Bastos FI, Cáceres C, Galvão J, Veras MA, Castilho EA. AIDS in Latin America: assessing the current status of the epidemic and the ongoing response. Int J Epidemiol 2008; 37(4):729-737..

This variability hinders the use of classic theory-driven evaluation standardized logic models for programs¹⁶16. Chen HT. Theory-driven evaluations. Newbury: Park Sage Publications; 1990.,³⁵35. Hartz ZMdA, Vieira-da-Silva LM. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Salvador, Rio de Janeiro: EDUFBA, Fiocruz; 2005.,³⁶36. Chen H-T. Practica program evaluation: Assessing and improving planning, implementation and effectiveness. Thousand Oaks, London, New Delhi: Sage; 2005.. We preferred to adopt the idea that the program is “a set of resources and activities directed toward one or more common goals”¹⁸18. Scheirer MA. Is Sustainability Possible? A Review and Commentary on Empirical Studies of Program Sustainability. Am J Eval 2005; 26(3):320-347., the goal in this case being ARV provision. Furthermore, such variability also requires the need for an appropriate data collection approach¹⁵15. Scheirer MA. Linking Sustainability Research to Intervention Types. Am J Public Health 2013; 103(4):e73-80.. We believe that in focusing on program history²⁴24. Pluye P, Potvin L, Denis J-L, Pelletier J, Mannoni C. Program sustainability begins with the first events. Evaluation and Program Planning 2005; 28(2):123-37. we were able to better grasp and understand differing internal developments, rather than using a one-fits-all logic model.

In a literature review on the sustainability of new programs and innovations, which included 125 health related studies, Stirman et al.¹⁴14. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implementation Science 2012; 7(17):1-19. point out that the majority of studies neither explore the nature and reasons for changes, nor the process by which adaptations and key decisions are made. By focusing on critical events that build program history, this approach specifically searches for patterns of change, their reasons – identified through key-stakeholders’ perspectives – and aim at analyzing underlying processes that help to explain program development and trends, i.e. process tracing²⁵25. Steel D. Social Mechanisms and Causal Inference. Philosophy of the Social Sciences 2004; 34(1):55-78..

By assuming implementation as a general process we strove not only to open interventions’ black boxes¹⁵15. Scheirer MA. Linking Sustainability Research to Intervention Types. Am J Public Health 2013; 103(4):e73-80.,¹⁶16. Chen HT. Theory-driven evaluations. Newbury: Park Sage Publications; 1990.,¹⁸18. Scheirer MA. Is Sustainability Possible? A Review and Commentary on Empirical Studies of Program Sustainability. Am J Eval 2005; 26(3):320-347.,³⁶36. Chen H-T. Practica program evaluation: Assessing and improving planning, implementation and effectiveness. Thousand Oaks, London, New Delhi: Sage; 2005., but also to analyze which program components and processes of change had in fact been mobilized³⁷37. Potvin L, Bisset S. There is more to methodology than method. In: Potvin L, McQueen D, editors. Health Promotion Evaluation Practices in the Americas. New york: Springer; 2008. p. 63-80.,³⁸38. Bisset S, Potvin L. Expanding our conceptualization of program implementation: lessons from the genealogy of a school-based nutrition program. Health Education Research 2007; 22(5):737-746.. By the adoption of Pluye’s focus on organizational routines³⁹39. Pluye P, Potvin L, Denis JL, Pelletier J. Program sustainability: focus on organizational routines. Health Promot Int 2004; 19(4):489-500., we were able to assess whether or not program pathways were put in place in a manner that favored program continuation, i.e. building program capacity for sustainability¹³13. Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, Bunger AC. Public health program capacity for sustainability: a new framework. Implementation Science 2013; 8(15):1-9.,¹⁷17. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education Research 1998; 13(1):87-108..

The framework also inserts sustainability-related conceptual events – implementation and routinization events²⁴24. Pluye P, Potvin L, Denis J-L, Pelletier J, Mannoni C. Program sustainability begins with the first events. Evaluation and Program Planning 2005; 28(2):123-37. – into an identifiable interface, which can be more easily perceived at data collection.

The interview process stimulated key informants to retell program history emphasizing each of the four dimensions: Program Resources; Activities; Reinforcement Strategies; and, Context and Organizational Culture. By revisiting their own narratives, informants were able to detail events and reflect upon the way in which context shaped program history. This strategy was enhanced by confrontation with the supplied documentation. As such we were able to detect a considerable number of critical events in each country, which in turn led to greater data consistency and internal validity. Moreover, the detection of more events enhanced the potential to explain sustainability in different scenarios. The exhaustive nature of the framework helped overcome the relative difficulty to identify critical events reported by Pluye²⁴24. Pluye P, Potvin L, Denis J-L, Pelletier J, Mannoni C. Program sustainability begins with the first events. Evaluation and Program Planning 2005; 28(2):123-37..

Informant selection was based on the best possible information retrieval when process tracing is required⁴⁰40. Tansey O. Process Tracing and Elite Interviewing: A Case for Non-probability Sampling. PS: Political Science and Politics 2007; 40(4):765-772.. The key process being traced was the carrying out of activities and flow of resources aimed at ARV provision, i.e. the implementation of ARV provision. Key informants were, or had been, very much involved in ARV provision with central participation in program history.

The framework allowed us to observe the differing trends towards resource sustainability. In Bolivia, the dependence on external grants and various changes in program coordination may have resulted in lack of access to ARVs⁴¹41. World Health Organization (WHO). Global update on HIV treatment 2013: results, impact and opportunities: WHO report in partnership with UNICEF and UNAIDS. Geneva: WHO; 2013.. These aspects show insufficient investment and lack of resources stability. In Mozambique, heavy dependence on foreign aid⁴²42. Ooms G. Health Development versus Medical Relief: The Illusion versus the Irrelevance of Sustainability. PLoS Medicine 2006; 3(8):1202-1205. put resource stability at risk. Notwithstanding, government agents planned an increase in treatment numbers over the years, avoiding large leaps, which in turn has favored continuity in mobilization of resources from external donors⁴³43. Ooms G, Damme WV, Baker BK, Zeitz P, Schrecker T. The ‘diagonal’ approach to Global Fund financing: a cure for the broader malaise of health systems? Global Health 2008; 4:6.. In five years (2008-2013) coverage has doubled⁴¹41. World Health Organization (WHO). Global update on HIV treatment 2013: results, impact and opportunities: WHO report in partnership with UNICEF and UNAIDS. Geneva: WHO; 2013.,⁴⁴44. Aidspan. Countries most in need of scaling-up ARV provision. 2008 [cited 2013 19 December]. Available from: www.aidspan.org/gfo_article/countries-most-need-scaling-arv-provision .
www.aidspan.org/gfo_article/countries-mo... ,⁴⁵45. The World Bank. Antiretroviral therapy coverage 2013 [cited 2013 19 December]. Available from: data.worldbank.org/indicator/SH.HIV.ARTC.ZS.
data.worldbank.org/indicator/SH.HIV.ARTC... . The involvement of the Clinton Health Access Initiative (CHAI) has been tantamount in helping to better ARV provision, by more efficient procurement schemes, resulting in better resource investment. Initiatives in continuous training of health professionals illustrate constant investment in human resources. In Peru a gradual but constant increase in ARV coverage was observed⁴⁵45. The World Bank. Antiretroviral therapy coverage 2013 [cited 2013 19 December]. Available from: data.worldbank.org/indicator/SH.HIV.ARTC.ZS.
data.worldbank.org/indicator/SH.HIV.ARTC... , and this was associated with a progressive scaling-up of resource investment over the years⁴⁶46. The Global Fund to Fight Aids, Tuberculosis and Malaria (GFATM). Grant Portfolio - Peru 2013 [cited 2013 December 19]. Available from: http://portfolio.theglobalfund.org/es/Grant/List/PER.
http://portfolio.theglobalfund.org/es/Gr... . Also, changes in employment contract schemes and the building of autonomous ARV financing have resulted in greater stability of government staff and of flow of resources⁴⁶46. The Global Fund to Fight Aids, Tuberculosis and Malaria (GFATM). Grant Portfolio - Peru 2013 [cited 2013 December 19]. Available from: http://portfolio.theglobalfund.org/es/Grant/List/PER.
http://portfolio.theglobalfund.org/es/Gr... ,⁴⁷47. Amaya AB. Blood, aid & sustainability: The case of HIV in Peru. International Health Policies 2013..

Low-income countries are more prone to rely on donations and international aid¹⁹19. Bossert TJ. Can they get along without us? Sustainability of donor-supported health projects in Central America and Africa. Soc Sci Med 1990; 30(9):1015-1023.,⁴²42. Ooms G. Health Development versus Medical Relief: The Illusion versus the Irrelevance of Sustainability. PLoS Medicine 2006; 3(8):1202-1205.. It is noteworthy that even though the dependence of external aid means a risk to resource stability⁴4. Eholié S-P, Aoussi FE, Ouattara IS, Bissagnéné E, Anglaret X. HIV treatment and care in resource-constrained environments: challenges for the next decade. J Int AIDS Soc 2012; 15:17334., the classical teach a man to fish doctrine of external donor aid¹⁹19. Bossert TJ. Can they get along without us? Sustainability of donor-supported health projects in Central America and Africa. Soc Sci Med 1990; 30(9):1015-1023. has suffered severe criticism⁴²42. Ooms G. Health Development versus Medical Relief: The Illusion versus the Irrelevance of Sustainability. PLoS Medicine 2006; 3(8):1202-1205.,⁴⁸48. Swidler A, Watkins SC. ‘‘Teach a Man to Fish”: The Sustainability Doctrine and Its Social Consequences. World Development 2009; 37(7):1182-1196. and seems to be changing⁴³43. Ooms G, Damme WV, Baker BK, Zeitz P, Schrecker T. The ‘diagonal’ approach to Global Fund financing: a cure for the broader malaise of health systems? Global Health 2008; 4:6.. The GFATM has adhered to “a new form of sustainability that relies on a combination of domestic resources and predictable, open-ended foreign assistance”⁴³43. Ooms G, Damme WV, Baker BK, Zeitz P, Schrecker T. The ‘diagonal’ approach to Global Fund financing: a cure for the broader malaise of health systems? Global Health 2008; 4:6.. The increase in demand for all aspects of HIV/AIDS care is bound to produce greater competition for international funds⁴4. Eholié S-P, Aoussi FE, Ouattara IS, Bissagnéné E, Anglaret X. HIV treatment and care in resource-constrained environments: challenges for the next decade. J Int AIDS Soc 2012; 15:17334..

The existence of external agents involved in ARV provision in Bolivia indicates that the activities required for ARV provision were not compatible with the MoH activities. Many changes in management of ARV provision throughout the years indicate successive activity failures. A 2012 UNDP audit of GFATM grants reports persisting difficulties in cooperation and coordination and partial compliance to procurement and supply management of health products⁴⁹49. United Nations Development Program (UNDP). Audit of UNDP Bolivia: Grants from The Global Fund to Fight AIDS, Tuberculosis ans Malaria. Report No. 901. Office of Audit and Investigations 15 August 2012. 2012.. Initially, the introduction of ARV purchases into Mozambican MoH procurement organizations didn’t change their activity profile (compatibility). In spite of changes in procurement responsibilities, observed continuity in work processes showed successful adaptation of activities. Provincial storage and distribution systems failed to adapt to rapid expansion and decentralization of treatment coverage⁵⁰50. Moçambique. Ministério da Saúde (MS). Desempenho do sector da Saúde. Oitava avaliação conjunta anual. Maputo: MS; 2009.. In Peru, no adaptation of activities seemed to be necessary. Organizations involved in ARV provision had been working in partnership and kept true to their original activities profiles, indicating compatibility⁵¹51. Peru. Ministério de Salud. Medición del Gasto en SIDA - MEGAS: Informe final. Lima: Minitério de Salud; 2011..

In Bolivia, risk taking procedures were established by new activities regarding customs clearance, which were enforced by means of new legislation⁵²52. Bolivia. Ley nº 3729, de 8 de agosto de 2007. Ley para la prevencion del vih-sida, proteccion de los derechos humanos y asistencia integral multidisciplinaria para la personas que viven con el VIH-SIDA. La Paz; 2007.. In Mozambique, integration of rules may be perceived by heavily centralized treatment standards and guidelines. Health professionals perceived the recruitment policies enforced by the government as incentives. This was considered an important strategy to overcome a health workforce gap in Mozambique⁵³53. Ooms G, Damme WV, Temmerman M. Medicines without Doctors: Why the Global Fund Must Fund Salaries of Health Workers to Expand AIDS Treatment. PLoS Medicine 2007; 4(4):e128.. These incentives were abandoned in 2008 when Day Hospitals were discontinued. From that time on, the government took risks by integrating HIV/AIDS and Primary Health Care. Pfeiffer et al.³²32. Pfeiffer J, Montoya P, Baptista AJ, Karagianis M, Pugas MdM, Micek M, Johnson W, Sherr K, Gimbel S, Baird S, Lambdin B, Gloyd S. Integration of HIV/AIDS services into African primary health care: lessons learned for health system strengthening in Mozambique - a case study. J Int AIDS Soc 2010; 13:3., defend this step as a positive measure to enhance all aspects of HIV AIDS care and ARV provision.

In Peru, an adequate level of communication was achieved by cooperation between MoH and other organizations involved in ARV provision⁵¹51. Peru. Ministério de Salud. Medición del Gasto en SIDA - MEGAS: Informe final. Lima: Minitério de Salud; 2011.. Career benefits worked as incentives, fostering program routines. Engagement in multinational price negotiations was an innovation that resulted in the definition of ceiling prices for future purchases, which may explain lower prices of ARV procured by MoH⁵⁴54. Mucha SM. Compras públicas de antirretrovirales en Perú - Octubre 2008- Julio 2009. Lima: Acción Internacional para la Salud, Oficina de Coordinación América Latina y el Caribe; 2010.. The success of these price negotiations, however, depend upon coordination of pharmaceutical regulations and policies, and/or pooled procurement schemes⁵⁵55. Osorio-de-Castro CGS, Crisante M, Miranda ES, Oliveira EA, Oliveira MA. Proposed methodology for monitoring antiretroviral drugs price negotiations in Latin America and the Caribbean. Pan Am J Public Health 2009; 26(2):137-147.,⁵⁶56. Vazquez ES, Monguio RR. Negotiating antiretroviral drug prices: the experience of the Andean countries. Health Policy Plan 2007; 22(2):63-72.. Additionally, the enactment of legislation illustrates initiatives to standardize HIV/Aids response in the country (integration of rules).

The end of Day Hospitals in Mozambique resulted in reduction of professionals’ time for HIV/Aids-related activities. This program reorientation also led to dissatisfaction from local-level professionals and PLHIV⁵⁷57. Couto Z. Encerramento dos Hospitais Dia: doentes perdem espaços onde se sentiam dignificados e não discriminados. Vertical 2009; 25:1-2.. In Peru, programmatic objectives were adjusted to MoH organizations’ institutional missions.

By selecting representatives of high level bureaucracy as key informants, local dynamics and their feedback effects, that are important in implementation analysis³⁸38. Bisset S, Potvin L. Expanding our conceptualization of program implementation: lessons from the genealogy of a school-based nutrition program. Health Education Research 2007; 22(5):737-746.,⁵⁸58. Matland RE. Synthesizing the Implementation Literature: The Ambiguity-Conflict Model of Policy Implementation. Journal of Public Administration Research and Theory 1995; 5(2):145-174.

59. Lipsky M. Street-Level Bureaucracy: Dilemmas of the individual in public services. New York: Russel Sage Foundation; 1980.

60. Elmore RF. Backward mapping. Implementation research and policy decisions. Political Science Quaterly 1980; 94(4):601-616.-⁶¹61. Barrett SM. Implementation studies: time for a revival? Personal reflections on 20 years of implementation studies. Public Administration 2004; 82(2):249-262., may have become underrepresented in program history accounts. Some particular elements that arise from vertical relations, such as the existence and use of communication channels, may have not been well detected. Additionally, the focus on organizational processes shift attention from the broader context, such as political support, that is important to explain sustainability¹³13. Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, Bunger AC. Public health program capacity for sustainability: a new framework. Implementation Science 2013; 8(15):1-9.,¹⁷17. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education Research 1998; 13(1):87-108..

By contrasting the three program histories it was possible to infer relative trends of sustainability. Overall, stronger routines and more established standards found in Peru put this country in a good position regarding sustainability. Distribution of events over time may be interpreted as a trend of strengthening ARV provision sustainability in that country. Many events identified in Mozambique indicate the development of routines and standards, but there are some bottlenecks and difficulties, specially regarding availability of resources and health system structure, which challenge program sustainability. Considering the great deal of effort in program planning, scaling-up and adapting to an ever-changing international donor context, trends over time seem to be positive. ARV provision in Bolivia has been going through many organizational and institutional changes, which has led to unstable routines, few standards and, therefore, weaker capacity for sustainability.

Conclusions

The conceptual framework was useful in recognizing events that may influence sustainable ARV provision in assessed countries. The application of our framework and its tools in the assessment of ARV provision resulted in three thorough accounts of program history, in which country specificities and internal development explanatory elements were explored. This approach was successful in capturing relevant data (critical events). The analytical matrix provided by the framework made it easier to organize and categorize data in a way that country-specific narratives could be translated into explanatory accounts. Processes and mechanisms leading to development of routines and establishment of standards could be, then, identified.

Our objective was not to propose a synthetic judgment of sustainability levels. We believe that the analysis of each program’s dynamics allows for the identification of strengths, challenges and opportunities, which, in turn, can contribute for better planning for sustainable ARV provision.

References

Publication Dates

History