ORIGINAL RESEARCH INVESTIGACIÓN ORIGINAL
A methodological approach for the evaluation of preparedness of pharmaceutical services
Un enfoque metodológico para evaluar el estado de preparación de los servicios farmacéuticos
Elaine S. MirandaI; James F. FitzgeraldII; Claudia G. S. Osorio-de-CastroIII
IDepartamento de Farmácia e Administração Farmacêutica, Faculdade de Farmácia, Universidade Federal Fluminense, Niterói, Rio de Janeiro, Brazil. Send correspondence to: Elaine S. Miranda, firstname.lastname@example.org
IIHealth Systems and Services, Pan American Health Organization, Washington D.C., United States of America
IIINúcleo de Assistência Farmacêutica, Escola Nacional de Saúde Pública Sérgio Arouca, Fundação Oswaldo Cruz, Rio de Janeiro, RJ, Brazil
OBJECTIVE: The aim of this article is to provide an outcome evaluation model for preparedness of pharmaceutical services (PS) in disaster situations.
METHODS: A five-step evaluation model development was conducted. The first step was a search of disaster preparedness and PS literature. The second step consisted of a description of the political and organizational aspects, external context, implementation, and performance of PS in disaster preparedness. A theoretical model on PS preparedness in disaster situations, encompassing pharmaceutical services variables and measures of preparedness, was developed as the third step. The fourth step produced a comprehensive theoretical model for evaluating PS preparedness, combining the two approaches used in steps two and three. The fifth and final step examined the development of the indicator framework.
RESULTS: The results of this methodological approach are presented in the logic model of PS preparedness and the indicator framework, both of which were developed based on the disaster preparedness and PS literature and organized to provide a structured evaluation approach.
CONCLUSIONS: PS was conceptualized as a program that can be evaluated by measurable effects. These effects can only be measured based on documented, on-site conditions before and after an event recognized as a disaster situation. This conceptual approach is context-modulated and therefore only applicable where the logistic cycle has been adopted as the rationale for PS.
Key words: Pharmaceutical services; preparedness; evaluation; methodology; Brazil
OBJETIVO: Proporcionar un modelo de evaluación de resultados relativos al estado de preparación de los servicios farmacéuticos (SF) para situaciones de desastre.
MÉTODOS: Se elaboró un modelo de evaluación en cinco pasos. El primer paso fue una búsqueda bibliográfica sobre preparativos para situaciones de desastre y SF. El segundo consistió en una descripción de los aspectos políticos y técnicos, el contexto externo, la implementación y el desempeño de los SF en la preparación para situaciones de desastre. El tercer paso fue la elaboración de un modelo teórico sobre el estado de preparación de los SF para situaciones de desastre, que comprendía variables relativas a los SF y medidas del estado de preparación. El cuarto paso consistió en la creación de un modelo teórico integral para evaluar el estado de preparación de los SF, mediante la combinación de los enfoques usados en los pasos dos y tres. El quinto y último paso fue el análisis de la elaboración del marco de indicadores.
RESULTADOS: Se presentan los resultados de este enfoque metodológico en el modelo lógico del estado de preparación de los SF y el marco de indicadores, ambos elaborados con base en la bibliografía sobre preparativos para casos de desastre y SF, y organizados para poder proporcionar un método estructurado de evaluación.
CONCLUSIONES: A efectos de la elaboración de este marco conceptual se consideró a los SF como un programa que podría ser evaluado a partir de sus efectos cuantificables. Estos efectos solo pueden ser medidos sobre la base de condiciones documentadas en el lugar de los hechos antes y después del evento considerado una situación de desastre. Este enfoque conceptual está modulado en función del contexto y por lo tanto se considera aplicable solamente donde se haya adoptado el ciclo logístico como fundamento de los SF.
Palabras clave: Servicios farmacéuticos; preparación; evaluación; metodología; Brasil
The increase in world population and modification in population dynamics that has occurred in the last 100 years has changed the odds in favor of disaster occurrence. Moreover, disaster detection has increased, producing a worldwide rise in the number of both notified disasters and people affected by them (1, 2).
Establishing the response capacity of health services would help minimize the consequences of disasters. Health services response can be enhanced through health services preparedness. According to the World Health Organization (WHO), disaster preparedness should extend to all of the most frequently identified hazards in a given country regardless of type, through the adoption of the "all hazard/whole health" approach (3).
Disaster response systems should include a variety of functions from distinct fields of knowledge (4). Perry establishes six generic functions for disaster response: alertness, evacuation, shelter, emergency health care, search, and rescue (5). Quarantelli suggests four more: human resource mobilization, damage evaluation, coordination of response activities, and essential public services restructure. Some of these functions are linked to health services provision (6).
Some disaster response functions, such as emergency health care, can be broken down into specific, independent processes such as the provision of pharmaceutical services (PS) in emergency situations (7). These processes can be studied in terms of their functions as health-related programs. Hartz defines a program as a group of organized actions, put into practice to provide a given population with goods and services (8). A variety of approaches can be used to evaluate program performance, such as the setting of benchmarks or the use of outcomes evaluation (9, 10).
Because PS is a crucial element in the provision of disaster response, evaluating PS preparedness, as a health-related program, may help identify vulnerabilities in the medicines supply chain and thus improve the quality and utility of the health sector response. The aim of this article is to provide an outcome evaluation model for PS preparedness in disaster situations.
MATERIALS AND METHODS
The development of the evaluation model was conducted in five steps. The first was a search of the literature. Scopus, Medline (PubMed), Web of Science, and Scielo databases were searched, for the 19902012 period, for articles using the following search terms (separately and cross-referenced): disaster preparedness, pharmaceutical services, program evaluation, and theoretical models. Several documents from key information sources such as the Pan American Health Organization (PAHO), the World Association for Disaster and Emergency Medicine (WADEM), and WHO were also added to the search results.
The second step consisted of describing the political and organizational aspects, external context, implementation, and performance of PS in disaster preparedness. Possible approaches for developing PS preparedness strategies, according to the literature, including all intermediary aspects or components, were tested in a theoretical or logic model on PS preparedness in disasters encompassing PS variables and measures of preparedness. The fourth step produced a comprehensive theoretical or logic model for evaluation of PS preparedness, combining the two approaches used in steps two and three (e.g., PS preparedness and its relationship to local context).
The fifth and final step examined the development of the indicator framework. To better illustrate the concepts or components tested in the logic model, the indicators were translated into specific, objective, and measurable variables. During the model development, individual indicators were added to improve model clarity, utility, reliability, measurability, and validity, but the indicator framework was designed to allow for objective analysis of the value of the program (PS preparedness) (9). Therefore, the indicators were organized by PS component. When possible, previously validated PS indicators were extracted from the literature and adapted. All indicators were described in detail, along with information sources and standard evaluation criteria (for all possible indicator outcomes).
Both the logic model of PS preparedness and the indicator framework were developed based on disaster preparedness and PS literature and organized to provide a structured evaluation approach.
Evaluating PS preparedness: implementation and performance
The literature on program evaluation showed two dimensions from which to structure models: implementation and performance (11, 12) and two aspects that should be taken into account: the political and organizational environment in which programs are developed, and the external context of the programs (13). Political and organizational environment included technical, managerial, financial, and political autonomy. External context included political will, potential hazards and vulnerabilities, existing social/health resources, and strategies (health sector preparedness, management of humanitarian aid, other community health programs, academic and professional associations, and actions of civil society) (4, 9, 13, 14).
The effects of a program can be measured by the interaction between the program environment and external context and program implementation and performance. The program's level of implementation can be evaluated by measuring its comprehensiveness and accessibility as well as the quality of its actions. Performance can be measured by assessing risk and damage control and user satisfaction, as well as program results and impacts (8, 11, 13).
Logic model for evaluating PS preparedness
As illustrated in the simplified version shown in Figure 2, the logic model included implementation and performance dimensions (organized by row), plus various components of PS preparedness identified in the literature (organized by column). All of these variables were included in the logic model, which comprised the basic activities of the PS cycle (defined below): medicines selection, forecasting, procurement, storage, distribution, and medicines utilization (15). Additional components, such as training and deployment of human resources, and specific activities related to disaster preparedness (e.g., production capacity for surpluses, and management systems for humanitarian aid) were also added. The final performance evaluation model included the results and impacts of PS preparedness, with a focus on performance components.
Implementation evaluation. As shown in Figure 2, the implementation dimension is composed of elements (shown in the boxes) pertaining to PS activities: medicine kits for disaster situations; consumption data; forecasting and procurement systems; procedures for receiving humanitarian aid; production of medicines and diagnostics; good storage practices (GSP) and infrastructure; adequately planned distribution and transportation systems; good distribution and transportation practices (GDTP); facilities for diagnosis, and for prescribing and dispensing pharmaceuticals; and adequately trained human resource teams.
Performance evaluation. PS performance is measured based on whether or not PS resources produce favorable outcomes in response to challenges often faced in disaster situations (16). Therefore, the performance dimension includes the following elements, placed under the same PS cycle: a list to guide the purchase and prescription of medicines (including appropriate procurement quantities), presented in an operational spreadsheet; compliance of medicine donations with donation protocol and medicine regulations; emergency procurement procedures for humanitarian aid; availability of production lines ready to meet surplus needs; properly stored medicines for rapid distribution according to local needs and capacity; adherence to medicines guidelines by health professionals and users; in-place systems to accommodate and care for patients; adequate disposal systems for medicines; and rapid deployment of trained personnel.
The evaluation process examines both implementation and performance to diagnose possible problems and discrepancies and to assess the overall comprehensiveness and quality of PS preparedness.
Program results and impacts are part of the performance dimension and include the entire array of performance components, which are designed to provide a comprehensive picture of PS preparedness. However, actual results can only be assessed after the onset of a recognized disaster (i.e., the performance components related to results can only be measured on site, during or after the response to a recognized disaster). The same limitations apply to evaluation of program impacts, to an even further degree, as they occur after both the event and the response.
It is important to recognize the relationship between the political and organizational environment and the external context, and the potential advantages of private sector participation, particularly from the following industries/institutions: pharmaceutical manufacturing (in relation to the production of medicines); academic institutions and professional associations (for human resource capacity building for preparedness); and civil society (for reducing vulnerabilities).
The 38-indicator framework based on PS evaluation literature and on preparedness followed previously established context variables, dimensions, components, and elements, presented in the comprehensive logic model. This conceptual benchmark was developed as a practical link to data collection instruments and to field data collection guidelines. The framework included identification of key sources of information (Table 1).
The first set of indicatorsexternal contextfocused on system vulnerabilities and resilience, which, in the disaster field, are directly related to the vulnerabilities and resilience of a given community (16). The 10 external context indicators shown in Table 1 include socioeconomic data; existing hazards; number of health professionals in the country/community; existing infrastructure to manage disaster response (emergencies management committee, humanitarian aid management, "safe hospitals"1); and known resilience factors such as community health worker programs and nongovernmental organizations (NGOs) dedicated to reducing vulnerabilities (4, 14, 17).
The second set included eight indicators related to the political and organizational environment that pertain to disaster-situation political and regulatory issues in the pharmaceutical field, such as medicines and disasters policies, protocols, and country regulatory structure.
The 20 indicators listed for pharmaceutical services focus on PS cycle activities (selection, forecasting, procurement/ production/donation, stock management, distribution, and medicines utilization) and human resources. Because PS preparedness involves both implementation and performance, some indicators for some of the PS cycle activities correspond to both dimensions. The selection activity involves listing priority medicines for disaster situations, and forecasting the quantities that will be acquired through purchase (procurement), production, or donations from humanitarian aid. The stock management and distribution activities require infrastructure for implementation, and performance is evaluated according to best practices. Medicines utilization involves sub-activities such as prescribing, dispensing, and use of pharmaceuticals. Human resource indicators stress the training and deployment of health professionals and health workers involved in PS.
The relationships expressed in the logic model are supported by evidence from both the literature review and specialists in the field (3, 1520), which describes standard aspects of PS as 1) the supply of medicines, storage, and quality assurance, and the safety and therapeutic efficacy of medicines use, as well as its monitoring and evaluation; 2) the collection and dissemination of information on medicines; and 3) education of health professionals, patients, and the community to ensure its rational use (21).
Disaster preparedness is characterized by the ability to mitigate the immediate impact of a disaster event and the potential to alleviate suffering and accelerate recovery. To achieve a good level of performance, preparedness must be carried out in an organized manner, within a specific sector, or through a multi-sector approach, as is often the case due to the complexity of disaster situations (22). For both individual- and multi-sector preparedness, all details related to all elements in the process must be clearly depicted and understood.
PS must be included in the various technical health sector activities that should be included in the development of a preparedness plan. Both disaster preparedness and PS comprise heavily technical aspects, albeit originating in different fields of knowledge (disaster management and health systems, respectively). Therefore, from a technical perspective, because both preparedness and PS may be assessed and measured via the same evaluation techniques, it is theoretically possible to use a joint approach, measuring implementation and performance of PS preparedness as outcome measures.
The challenge of combining these two approaches into one performance-oriented model, including elements of both PS and preparedness, was undertaken to provide insight into the possibility of providing adequate PS in disaster situations. Adequacy of services is assessed based on technical performance, and is quality-oriented. Because the quality perspective was central to the approach, PS preparedness was treated as a "program" for which implementation and performance could be measured (11). This type of evaluation demands systematic collection of data and provides information useful to decision-making to provide a performance diagnosis and, ultimately, improvement of the intervention (23).
Denis & Champagne recommend that program evaluation design be based on a theoretical modela "theory-driven" evaluation (TDE) scheme. The TDE approach requires the construction of a logic model (24). Hartz establishes that the logical construction should include the "problem" (what is expected from a program); the target population and context; and the program content (the minimum program attributes required for producing isolated effects and/or comprehensive expected effects) (8). According to Reynolds, the construction of a logic model should be concise and should originate from previous research results, social science theories, and experiences from managers and evaluators (25).
The logic model for PS preparedness used in this study was structured according to the organization of PS recognized by PAHO and adopted in many Latin American countries where comprehensive PS are mostly government subsidized and occur as consecutive activities known as the "PS cycle" or the logistics cycle (26). According to this perspective, PS is a multidisciplinary and systemic process, not restricted to the simple supply of medicines (15, 21). Thus, the products obtained in this process depend on management capacity, adherence to guidelines, and the translation of guidelines into concrete actions, all of which may be assessed via value judgments, translated into performance evaluation (27).
The authors have applied these indicators to more than 20 municipalities in Brazil and expect to produce synthesized results for assessment of the logic model. It is hoped that dissemination of this model will increase awareness of PS disaster preparedness in other countries in Latin America.
Adini et al. recommend that in the field of preparedness, evaluations assess elements of communication, planning, and emergency coordination activities, as well as training of personnel, health sector capacity, and availability of resources (28). Because PS preparedness involves the deployment of resources, through specific actions, its performance can be assessed by the products generated in the process (8, 13, 16).
The literature highlights two main elements that should be considered in program evaluation: the political and organizational environment, and the external context (8, 9, 13). If this concept is applied to the field of disasters, some important issues arise that are bound to influence PS preparedness, such as aspects related to in-country disasters policy, medicines policy, and medicines and health regulations, as well as the structures that provide support to policy and regulation. Consideration of the environment and context also provides depth to case analyses, helping researchers to better understand the subject of the study, and strengthening internal validity (29).
The need for health system preparedness for disaster situations is growing. Therefore, a structured methodology for evaluating preparedness is a useful tool. In case studies, generalization (external validity) is achieved through a logic model and not through the sample (30). Nevertheless, it is important to focus on the external validity of the model. When developing the model described here, external context was translated into three main aspects: resources and infrastructure (physical and financial means and in-place coordination and communication networks); vulnerabilities (hazards and socioeconomic standing of the community, as well as its level of development); and resilience (mainly the structure of civil society and health sector support programs). These elements of the model were also included when the indicators were formulated.
This model was not designed to involve the private sector per se because disaster preparedness in Latin American countries is mainly seen as the responsibility of the public sector (at least in recent decades). A study conducted in the state of Minas Gerais, Brazil, showed that although various private sector stakeholders are included in the Brazilian National (and State) Civil Defense Plan, only the public sector had any knowledge of its role, and this knowledge existed only among some public stakeholders.2 This lack of awareness must change.
The academic community is an important stakeholder in disaster response, not only for capacity building but also for involvement in the response itself, as a qualified workforce that can be deployed (31).
The literature reviewed in this study also supported the indicator framework. Each indicator was defined according to a series of published evaluation studies (9, 11, 12, 15, 16, 1820, 28, 32, 33). All indicators generated from the model were previously validated in the literature for exactly the same components used in the logic model.
While program implementation and performance are measurable, limitations arise when the measurable outcome is "effects"which encompasses program results and impacts (9, 12). Measuring results and impacts of PS preparedness is difficult due to the unpredictable nature of disasters, and the fact that this type of assessment may only be done in loco, during or after the event has taken place and been recognized as a disaster situation based on predefined criteria.
When disaster strikes, mayhem is sometimes the result, due to a lack of central coordination or conflicts among various actors and managers from different response sectors (34). Because the focus of this study was on preparedness (i.e., pre-response), and because only PS was considered, these outcomes can not be addressed by the model. Although all components of disaster response should interact in a positive manner, PS must be prepared independently and regardless of conflicts and problems in other sectors involved in the response.
Another limitation of the logic model used in this study involves its applicability to different types of PS activities. Because the model assumes PS activities are organized according to the logistics cycle or "PS cycle," preparedness implementation and performance must be carefully analyzed to verify the model's goodness of fit for evaluation of any PS activities that are not organized in the same manner. In addition, although external validity is assumed when the external context is associated with the results of the program, it has not been proven. The indicator framework is currently being adopted for evaluation of PS preparedness in different contexts in Brazil, which will help to provide evidence for this important attribute.
Health systems must be increasingly aware of the risk of disasters. WHO recommends the "all hazards/whole-health" approach for strategic health sector and community preparedness (3). The fundamental perspective of this approach is that countries should not use hazard-specific preparedness, mitigation, and response procedures due to the high costs and increased vulnerabilities inherent in customized responses. Use of the "all hazards/whole-health" approach makes good sense, as planning and resource tools for all types of disaster response are similar regardless of the type of hazard. For these same reasons, a methodologically sound, literature-based approach for objective evaluation of PS preparedness seems justified.
A conceptual approach for evaluating PS preparedness in disasters was proposed, specifically regarding PS implementation and performance. The approach involves a logic model for the evaluation and an indicator framework. For the development of this conceptual benchmark, and based on the literature, PS was treated as a program that could be evaluated by measurable effects. However, actual effects can only be examined after the event has taken place. This conceptual approach is context-modulated and is deemed applicable where the logistic cycle has been adopted as the rationale for PS. Applicability of the model in contexts that adopt alternative PS structures requires further investigation.
Conflicts of interest. None.
1. Centre for Research on the Epidemiology of Disasters. EMDAT: the international disaster database [Internet]. Natural disasters reported 19002011. Brussels: CRED; 2012. Available from: http://www.emdat.be/sites/default/files/Trends/natural/world_1900_2011/eveyr2.pdf Accessed 1 May 2012.
2. Centre for Research on the Epidemiology of Disasters. EMDAT: the international disaster database [Internet]. Number of people reported affected by natural disasters 19002011. Brussels: CRED; 2012. Available from: http://www.emdat.be/sites/default/files/Trends/natural/world_1900_2011/affyr1.pdf Accessed 1 May 2012.
3. World Health Organization. Risk reduction and emergency preparedness: WHO six-year strategy for the health sector and community capacity development. Geneva: WHO; 2007. Available from: http://www.who.int/hac/techguidance/preparedness/emergency_preparedness_eng.pdf Accessed 1 May 2012.
4. United Nations Office for Disaster Risk Reduction; United Nations Office for the Coordination of Humanitarian Affairs. Disaster preparedness for effective response: guidance and indicator package for implementing priority five of the Hyogo Framework. Geneva: UNISDR / OCHA; 2008. 51 pp.
5. Perry RW. Managing disaster response operations. In: Drabek TE, Hoetmer GJ, editors. Emergency management: principles and practice for local government. Washington: International City Management Association; 1991. Pp. 20123.
6. Quarantelli EL. Ten criteria for evaluating the management of community disasters. Disasters. 1997;21(1):3956.
7. Birnbaum ML. Inputs, processes, functions, and systemslet us be clear. Prehosp Disaster Med. 2011;26(3):1467.
8. Hartz ZMA. Avaliação dos programas de saúde: perspectivas teórico-metodológicas e político-institucionais. Cien Saude Colet. 1999;4(2):34153.
9. Framework for program evaluation in public health. MMWR Recomm Rep. 1999;48(RR-11):140.
10. Evaluation Research Society Standards Committee. Evaluation Research Society standards for program evaluation. In: Rossi PH, editor. Standards for evaluation practice. New directions for program evaluation. No. 15, 719. San Francisco: Jossey-Bass; 1982.
11. Cosendey MA, Hartz ZM, Bermudez JA. Validation of a tool for assessing the quality of pharmaceutical services. Cad Saude Publica. 2003;19(2):395406.
12. Global AIDS Program. Monitoring and evaluation capacity building for program improvement field guide, version 1. Atlanta: U.S. Centers for Disease Control and Prevention; 2003.
13. Santos EM, Natal S. Dimensão técnico-operacional: unidade didático-pedagógica: modelo teórico da avaliação. Rio de Janeiro: Abrasco; 2006.
14. Redwood-Campbell L, Abrahams J. Primary health care and disastersthe current state of the literature: what we know, gaps and next steps. Prehosp Disaster Med. 2011;26(3):18491.
15. Management Sciences for Health. Managing for rational drug use. In: Quick JD, Hogerzeil HV, Rankin JR, Dukes MN, Laing R, Garnett A, et al., editors. Managing drug supply: the selection, procurement, distribution, and use of pharmaceuticals. 2nd ed. West Hartford, CT: Kumarian Press; 1997. Pp. 4219.
16. Task Force on Quality Control of Disaster Management; World Association for Disaster and Emergency Medicine; Nordic Society for Disaster Medicine. Health disaster management: guidelines for evaluation and research in the Utstein Style. Overview and concepts. Prehosp Disaster Med. 2003;17 Suppl 3:3155.
17. Barrantes M, Basabe P, Basher R, editors. Words into action: a guide for implementing the Hyogo Framework. Geneva: United Nations International Strategy for Disaster Reduction; 2007. Available from: http://www.unisdr.org/files/594_10382.pdf Accessed 19 March 2012.
18. Management Sciences for Health Rational Pharmaceutical Management Project. Rapid pharmaceutical management assessment: an indicator-based approach. Arlington, VA: MSH; 1995.
19. World Health Organization. How to investigate drug use in health facilitiesselected drug use indicators. EDM Research Series No. 7. Geneva: WHO; 1993. (WHO/DAP/93.1).
20. Clark M. Drug management for malaria manual. Arlington, VA: Management Sciences for Health; 2000. 160 pp.
21. Marin N, Luiza VL, Osorio-de-Castro CGS, Machado-dos-Santos S, editors. Assistência farmacêutica para gerentes municipais. 1st ed. Rio de Janeiro: Pan American Health Organization; 2003. Pp. 1204.
22. Schutz GE, Freitas CM. Enfoque desde la ciencia post-normal de la epizootia fiebre aftosa. Hist Cienc Saude-Manguinhos. 2003;10(2):63755.
23. Schalock RL. Outcome-based evaluation. 1st ed. New York: Plenum Press; 1995. P. 242.
24. Denis JL, Champagne F. Analise da implantação. In: ZMA Hartz, editor. Avaliação em saúde: dos modelos conceituais à prática na análise da implantação de programas. Rio de Janeiro: Fiocruz; 1997. Pp. 4988.
25. Reynolds AJ. Confirmatory program evaluation: a method for strengthening causal inference. Am J Evaluation. 1998;19(2):20321.
26. Pan American Health Organization. Public policies and health systems and services. In: Health in the Americas 2007. Washington: PAHO; 2007. Pp. 298381.
27. Chen HT. Theory-driven evaluations. Newbury Park, CA: Sage; 1990.
28. Adini B, Goldberg A, Laor D, Cohen R, Zadok R, Bar-Dayan Y. Assessing levels of hospital emergency preparedness. Prehosp Disaster Med. 2006;21(6):4517.
29. Mohr LB. The qualitative method of impact analysis. Am J Evaluation. 1999;20(1):6984.
30. Yin RK. Case study research: design and methods. 2nd ed. Thousand Oaks, CA: Sage; 2009.
31. Hogue MD, Hogue HB, Lander RD, Avent K, Fleenor M. The nontraditional role of pharmacists after hurricane Katrina: process description and lessons learned. Public Health Rep. 2009;124(2):21723.
32. Osório-de-Castro CG, Chaves GC, Ruiz AM, Miranda ES, Campos MR, Suárez-Mútis MC. A proposal for an evaluation model of pharmaceutical services for malaria. Cad Saude Publica. 2009;25(9):207582.
33. Asch SM, Stoto M, Mendes M, Valdez RB, Gallagher ME, Halverson P, et al. A review of instruments assessing public health preparedness. Public Health Rep. 2005;120(5):53242.
34. Poncelet JL, de Ville de Goyet C. Disaster preparedness: institutional capacity building in the Americas. World Health Stat Q. 1996;49(34):1959.
35. McConnan I, editor. Humanitarian charter and minimum standards in disaster response. Geneva: Sphere Project; 2000.
36. United Nations International Strategy for Disaster Reduction. Report of the World Conference on Disaster Reduction, Kobe, Hyugo, Japan, 1822 January 2005. New York: UNISDR; 2005. Available from: www.unisdr.org/2005/wcdr
37. World Health Organization. Guidelines for developing national drug policies. Geneva: WHO; 1999.
38. World Health Organization. Guidelines for safe disposal of unwanted pharmaceuticals in and after emergencies. Geneva: WHO; 1999.
39. Organização Pan-Americana da Saúde; Ministério da Saúde (BR). Avaliação da Assistência Farmacêutica no Brasil: estrutura, processo e resultados. Brasília, DF: OPAS Brasil, MS; 2005.
40. Osorio-de-Castro CGS, Castilho SR. Diagnóstico da Farmácia Hospitalar no Brasil. Rio de Janeiro: Fiocruz; 2004. 150 p.
Manuscript received on 13 November 2012.
Revised version accepted for publication on 7 October 2013.
1 Facilities that have the capacity to remain functional during and after a disaster.
2 Nascimento RP. A gestão dos desastres na perspectiva dos órgãos setoriais e de apoio: uma análise da estrutura da defesa civil instalada na capital mineira [dissertação]. Niterói, RJ: Programa de Defesa e Segurança Civil, Universidade Federal Fluminense; 2011.