RESEARCH

Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions

Congélation du vaccin anti-hépatite B dans la chaîne du froid indonésienne : observations et solutions

Congelación de vacunas anti-hepatitis B en la cadena de frío en Indonesia: evidencia y soluciones

Carib M. Nelson^{I, 1}; Hariadi Wibisono^II; Hary Purwanto^III; Isa Mansyur^IV; Vanda Moniaga^V; Anton Widjaya^VI

^IProgram Officer, Program for Appropriate Technology in Health (PATH), Seattle, WA, USA
^IIHead, Subdirectorate for Expanded Programme on Immunization, Ministry of Health, Indonesia
^IIISubdirectorate for Expanded Programme on Immunization, Ministry of Health, Indonesia
^IVDirector for Production, PT Bio Farma, Indonesia
^VAssociate Technical Officer, Program for Appropriate Technology in Health (PATH), Jakarta, Indonesia
^VISenior Technical Director, Program for Appropriate Technology in Health (PATH), Jakarta, Indonesia

ABSTRACT

OBJECTIVES: To document and characterize freezing temperatures in the Indonesian vaccine cold chain and to evaluate the feasibility of changes designed to reduce the occurrence of freezing.
METHODS: Data loggers were used to measure temperatures of shipments of hepatitis B vaccine from manufacturer to point of use. Baseline conditions and three intervention phases were monitored. During each of the intervention phases, vaccines were removed progressively from the standard 2–8 °C cold chain.
FINDINGS: Freezing temperatures were recorded in 75% of baseline shipments. The highest rates of freezing occurred during transport from province to district, storage in district-level ice-lined refrigerators, and storage in refrigerators in health centres. Interventions reduced freezing, without excessive heat exposure.
CONCLUSIONS: Inadvertent freezing of freeze-sensitive vaccines is widespread in Indonesia. Simple strategies exist to reduce freezing — for example, selective transport and storage of vaccines at ambient temperatures. The use of vaccine vial monitors reduces the risk associated with heat-damaged vaccines in these scenarios. Policy changes that allow limited storage of freeze-sensitive vaccines at temperatures >2–8 °C would enable flexible vaccine distribution strategies that could reduce vaccine freezing, reduce costs, and increase capacity.

Keywords: Hepatitis B vaccines; Freezing; Refrigeration; Drug storage/methods; Transportation; Drug stability; Child health services; Indonesia (source: MeSH, NLM).

RÉSUMÉ

OBJECTIF: Documenter et caractériser les températures de congélation dans la chaîne du froid indonésienne utilisée pour les vaccins et évaluer la faisabilité de modifications destinées à réduire la fréquence de la congélation.
MÉTHODES: Des enregistreurs séquentiels de données ont été utilisés pour mesurer la température du vaccin anti-hépatite B entre le départ de chez le fabricant et le point d'utilisation. Les conditions initiales et les conditions au cours des trois phases de l'intervention ont été mesurées. A chacune des phases de l'intervention, les vaccins ont été progressivement retirés de la chaîne du froid dont la température normale est de 2-8°C.
RÉSULTATS: La température de congélation a été notée pour 75 % des envois dans les conditions initiales. La fréquence de la congélation était maximale au cours du transport de la province vers le district et de la conservation dans les réfrigérateurs givrés du district et dans ceux des centres de santé. Les interventions ont diminué la congélation sans exposition thermique excessive des vaccins.
CONCLUSION: La congélation involontaire des vaccins sensibles est fréquente en Indonésie. Il existe des stratégies simples pour la diminuer – par exemple transporter et conserver sélectivement ces vaccins à température ambiante. Dans de tels scénarios, l'utilisation des pastilles de contrôle des vaccins diminue le risque de dégradation thermique. Une modification des politiques permettant une conservation limitée des vaccins sensibles à des températures >2-8°C déboucherait sur des stratégies de distribution des vaccins plus souples et susceptibles de réduire la congélation des vaccins et les coûts et d'augmenter la capacité.

Mots clés: Vaccin antihépatite B; Congélation; Réfrigération; Conservation médicament/méthodes; Transports; Stabilité médicament; Service santé infantile; Indonésie (source: MeSH, INSERM).

RESUMEN

OBJETIVO: Documentar y caracterizar las temperaturas de congelación en la cadena de frío de las vacunas en Indonesia y evaluar la viabilidad de los cambios planeados para reducir el riesgo de congelación.
MÉTODOS: Se usaron dispositivos de registro de datos para medir la temperatura de las remesas de vacuna anti-hepatitis B entre el fabricante y el punto de uso. Se vigilaron las condiciones durante una fase de referencia y en tres fases de intervención. Durante cada una de las fases de intervención se retiraron vacunas progresivamente de la cadena de frío estándar a 2–8 °C.
RESULTADOS: Se registraron temperaturas de congelación en un 75% de las remesas de referencia. Las tasas más altas de congelación se dieron durante el transporte de la provincia a los distritos, durante el almacenamiento en frigoríficos de hielo en los distritos, y durante el almacenamiento en los frigoríficos de los centros de salud. Las intervenciones redujeron los casos de congelación, sin que ello entrañase la exposición a un calor excesivo.
CONCLUSIÓN: La congelación accidental de vacunas sensibles a las bajas temperaturas es un fenómeno generalizado en Indonesia. Existen estrategias sencillas para reducir la congelación, como por ejemplo el transporte y almacenamiento selectivo de vacunas a temperatura ambiente. El uso de sensores de control permite reducir el riesgo asociado a las vacunas dañadas por el calor en esas circunstancias. Todo cambio normativo que haga posible un almacenamiento limitado de las vacunas sensibles a congelación a temperaturas > 2–8°C permitiría adoptar estrategias flexibles de distribución de vacunas que podrían reducir los casos de congelación, abaratar los costos y aumentar la capacidad.

Palabras clave: Vacunas contra hepatitis B; Congelamiento; Refrigeración; Almacenaje de medicamentos/métodos; Transportes; Estabilidad de medicamentos; Servicios de salud infantil; Indonesia (fuente: DeCS, BIREME).

Introduction

Immunization has been identified as one of the most cost-effective health interventions (1). The Global Alliance for Vaccines and Immunizations and the Vaccine Fund have stimulated international support of vaccination programmes by providing nearly US$1 billion to strengthen immunization systems and accelerate the introduction of new vaccines. Adequate cold-chain infrastructure and compliance are paramount for preserving the quality of these vaccines as they are distributed. Emphasis has long been placed on avoiding high temperatures during vaccine shipments, but, until recently, little attention has been focused on freezing of vaccines.

WHO guidelines for the use of hepatitis B, diphtheria–tetanus–pertussis, diphtheria–tetanus, and tetanus toxoid vaccines specify that they must not be frozen and should not be used if thought to have been frozen (2). The freezing of vaccines that contain liquid formulations of diphtheria, tetanus, pertussis, and hepatitis B vaccines leads to a loss of potency and compromised protective immunogenicity in recipients (3–6).

Although WHO guidelines and vaccine labels state that these vaccines should be stored at temperatures of 2–8 °C, cold-chain temperatures <0 °C are common. Recent studies in Australia (7–9), Canada (6), Hungary (17), Malaysia (16), Mongolia (18), Pakistan (15), the United Kingdom (10–12), the United States (13, 14) and other countries have found widespread freezing at many stages of the vaccine distribution system. Improperly adjusted refrigeration equipment, poor compliance with cold chain procedures, inadequate monitoring, and poor understanding of the dangers of freezing result in frequent cold-chain freezing. Studies in China and Indonesia showed that hepatitis B and tetanus toxoid vaccines can be stored and used at temperatures >2–8 °C while retaining full immunogenicity (19, 20). Indonesia recently implemented a policy of storing hepatitis B vaccine in midwives' homes at ambient temperatures (21). Although the impetus for ambient temperature storage of these vaccines was for programmatic benefit, reduced freezing was a probable side benefit. Few data have been published on removing vaccines from the cold chain to prevent freezing.

This study aimed to characterize the frequency of freezing temperatures that occur in the Indonesian vaccine cold chain and to evaluate the feasibility of several modifications to the cold chain that were designed to reduce freezing.

Methods and materials

Study sites

The study took place in the Indonesian provinces of East Java and Nusa Tenggara Barat from September 2001 to June 2002. Two districts in each province and two health centres in each district were chosen, which gave a total of eight health centres. Although sites were not selected randomly, a wide range of transport distances, accessibility, and infrastructure situations were sought.

The ambient temperatures in the study districts and along transport routes ranged from 19–36 °C. Typically the minimum and maximum temperature lay in the range of 22–32 °C. Average temperatures were 26–29 °C.

Equipment and supplies

Data loggers

Tiny TTM data loggers (Remonsys Limited, Witney, UK) were programmed to take temperature readings at two-hour intervals.

Uniject devices

Standard 0.5-ml Uniject prefill injection devices (BD Pharmaceutical System, Franklin Lakes, NJ, USA) were filled with recombinant hepatitis B vaccine through standard filling procedures by PT Bio Farma (Bandung, Indonesia).

Vaccine vial monitors

Type VVM30 vaccine vial monitors (LifeLines Technology, Morris Plains, NJ, USA) were shipped to PT Bio Farma and applied to Uniject pouches just before shipment to the provinces. The vaccine vial monitors underwent standard quality assurance procedures before they were attached to the Uniject pouches.

Freeze indicators

Freeze Watch (0 °C freeze) indicators (3M, St. Paul, Minnesota, USA) were included in each shipment.

Refrigerators

The study monitored existing cold-chain equipment commonly used in the Indonesian cold chain. Ice-lined refrigerators were used in district storage. Refrigerators in health centers were a combination of chest-type absorption refrigerators that operated on electrical power, chest-type compression refrigerators, and domestic refrigerators.

Definitions

Freezing

Shipments were considered subject to freezing temperatures when one or more temperatures reading <0 °C were recorded by the data logger. In two instances when Freeze Watch indicators activated, but corresponding subzero temperatures were not recorded by the data logger (Freeze Watch sensitivity 92% (95% confidence interval (CI) = 73–99%) and specificity 94% (CI = 79–100%)), shipments were not considered frozen, although temperatures could have dropped below freezing during the two-hour sampling interval of the data logger. The reported freezing point of hepatitis B vaccines is –0.5 °C and the effect of supercooling may result in temperatures several degrees below –0.5 °C before actual freezing occurs. The freezing of hepatitis B vaccine to a solid state thus may not have occurred during all instances of recorded subzero temperatures.

Ambient temperatures

"Ambient-temperature" storage of vaccines was used to describe situations in which vaccines were transported or stored without ice, refrigeration, or air conditioning, and thus were exposed to ambient daily temperatures.

Study design

Shipment monitoring

Within each shipment destined for each study site, midwife "monitoring packets" were prepared by combining a data logger, a vaccine vial monitor on a card, and a freeze indicator in a sealed plastic bag. These packets were placed inside boxes containing 100 Uniject devices. Boxes that contained monitoring packets were marked to indicate the site for delivery. Monitoring packets were prepared immediately before the vaccines were packed into shipping containers. At each stage of the cold chain, staff from the ministry of health recorded the time and date of arrival and departure of the vaccine shipments, as well as the status of the vaccine vial monitor and the condition of the Freeze Watch device. Staff at each level received training on completion of forms and study procedures.

Indonesia's existing (baseline) cold chain

The Indonesian cold chain for distribution of hepatitis B vaccine in the Uniject device (HB-Uniject) was slightly different from the standard WHOrecommended cold chain for vaccines. To facilitate delivery of hepatitis B vaccine to infants born at home, Indonesia has adopted an "out-of-the-cold-chain" storage and delivery policy for midwives who deliver the birth dose. Midwives are allowed to store HBUnijects in their homes without refrigeration until the endpoint of the vaccine vial monitor or expiry date is reached. In all other aspects, the Indonesian cold chain represents a standard WHO-recommended cold chain, in which a series of insulated boxes, vaccine carriers, cold rooms, and refrigerators were used to transport and store the vaccines. Recommended temperatures for these stages of the cold chain are 2–8 °C.

Cold-chain interventions to reduce freezing

To reduce overall cold-chain freezing, the study targeted distribution stages expected to show the highest levels of freezing. Alternatives to existing cold-chain procedures and WHO-recommended temperature limits of 2–8 °C were introduced. During each of the three intervention phases, HB-Uniject distribution was removed progressively from the existing Indonesian cold chain. During the first phase, "no-ice" transport was introduced in all transportation stages. HB-Unijects were transported in standard cold boxes and vaccine carriers but without any ice or ice packs. During the second phase, no-ice transport continued and HBUnijects were stored in air-conditioned rooms at the district level. During the third phase, in addition to no-ice transport and air-conditioned district storage, HB-Unijects were stored at ambient temperatures in the health centres. Table 1 summarizes distribution conditions.

Results

Baseline freezing: existing cold chain

Freezing occurred during all legs of the existing vaccine distribution system, except for during transport to provinces and during ambient-temperature storage by midwives. The highest incidence of freezing occurred in three distinct distribution stages: transport from province to district, refrigerated storage at the district level, and refrigerated storage at health centres (Table 2). Overall, 12/16 (75%) shipments were exposed to freezing temperatures (Table 3).

Phase I intervention: no-ice transport

In Phase I, vaccines were transported in vaccine carriers and cold boxes but without ice. Under this intervention, 67% of shipments were subject to freezing temperatures (Table 3). Transport of vaccines without ice eliminated freezing temperatures during the transport legs, but it did not significantly reduce overall freezing. This was because high levels of freezing continued to occur in province, district, and health centre refrigerators.

Phase II interventions: no-ice transport and air-conditioned storage in districts

In addition to no-ice transport, air-conditioned rooms instead of ice-lined refrigerators were used to store vaccines at district health offices. These interventions reduced overall freezing of 29% of shipments (Table 3). Elimination of freezing during transport and in the districts' ice-lined refrigerators resulted in a significant reduction in overall freezing.

Phase III interventions: no-ice transport, air-conditioned storage in districts, and ambient-temperature storage at health centres

In addition to the previous interventions, HB-Unijects were stored at ambient temperatures at health centres, without refrigeration. This combination of interventions completely eliminated freezing temperatures during distribution of the eight shipments monitored during this phase (Table 3).

Heat exposure in modified cold-chain scenarios

Selective transport and storage of vaccine outside the cold chain did not result in exposure to excessive temperatures. During storage by midwives (7–42 (average 32) days) the highest temperature recorded was 34.1 °C and the average temperature was 27 °C. During no-ice transport (one hour to six days), the highest temperature recorded was 33 °C and the average was 28 °C. During air-conditioned storage at district health offices (18–42 days), the highest temperature recorded was 32.2 °C and the average was 25 °C. During ambient-temperature storage at health centres (1–27 (average 23) days), the maximum temperature recorded was 34.5 °C, and the average was 28 °C.

Temperature readings from data loggers were analysed to estimate the effects of heat exposure on vaccine stability during each modified cold-chain scenario (Table 4). The calculated endpoint of the VVM30 (based on 30-day limit at 37 °C) was considered the limit of the hepatitis B vaccine stability "safety margin". Temperature data were used to calculate the reduction of the safety margin in each scenario: "0% reduction" indicated a vaccine with no heat exposure and "100% safety margin used" a vaccine at the vaccine vial monitor's endpoint, which should be discarded. In none of the 48 monitored shipments did any vaccines receive excessive heat exposure.

The calculations used to determine the stability safety margin were extrapolated to estimate the number of additional days midwives could have stored hepatitis B vaccine at ambient temperatures before they would have reached the vaccine vial monitor's endpoint (Table 4). In even the most extreme out-of-cold chain scenario modelled in this study (Phase III: no-ice transport, air-conditioned district storage, ambient-temperature storage at health centres, and midwife ambient storage), hepatitis B vaccines could have been stored for an additional 36 days at ambient temperatures before reaching the vaccine vial monitor's endpoint.

Discussion

Recent publications show that the high level of freezing recorded in this study is not limited to the Indonesian provinces. The finding that 75% of hepatitis B vaccines were exposed to subzero temperatures supports evidence of serious freezing problems in other parts of the world, including several developed countries (7–14, 16–18, 22–24). In light of this growing body of evidence of widespread freezing in the cold chain, steps must be taken to reduce the exposure of freeze-sensitive vaccines to freezing temperatures. Vaccine freezing could undermine recent global efforts devoted to improving immunization coverage and supplying new vaccines to developing countries. To ensure that vaccines reach their targets with potency intact, new strategies and systems for vaccine distribution must be implemented. If not, the influx of new vaccines may exacerbate cold-chain problems as capacity limits are reached.

Concurrent with the heightened concern about vaccine freezing is an increased understanding of the heat stability of freeze-sensitive vaccines. Sensitivity to freeze damage, combined with demonstrated heat stability, supports limited storage of hepatitis B, tetanus, diphtheria, and pertussis vaccines at temperatures higher than the conventional 2–8 °C range. At the same time, new monitoring tools reduce concerns that exposure to higher temperatures might result in use of damaged vaccines. Vaccine vial monitors help ensure that vaccines exposed to excessive heat can be identified and discarded (25), thus enabling freeze-sensitive vaccines to be stored at higher than traditional cold-chain temperatures. When used in flexible cold-chain approaches, such as those presented in this study, vaccine vial monitors can help reduce vaccine freezing and control the negative consequences of heat exposure.

The limited storage of vaccines above the 2–8 °C temperature range is not entirely experimental. Since 1999, Indonesia has introduced an innovative policy of storage of HB-Uniject in village midwives' homes at ambient temperatures up to the limits indicated by the vaccine vial monitor and expiry date. This approach has greatly facilitated neonatal home visits and allowed the country to pursue a birth-dose programme for hepatitis B vaccine (21). Indonesia's flexible cold-chain birth dose approach has improved hepatitis B vaccine coverage and reduced hepatitis B carrier rates (26). The flexible cold-chain strategy is being tested in global programmes, such as tetanus elimination, which strive to reach women and children at the periphery of the health care system, where the infrastructure is limited. Even delivery of oral polio vaccine — the least heat-stable vaccine — has been expanded successfully beyond cold-chain limits when delivered to and used at health posts without ice in Sudan (27).

In Indonesia, midwives typically store hepatitis B vaccine at ambient temperature for an average of 32 days. Table 4 shows the additional days of ambient temperature accumulated during each of the modified cold-chain intervention phases. In the third phase, vaccines were stored for an average of 83 days outside the cold chain without exceeding heat exposure limits. This level of heat stability allows significant opportunities for more flexible vaccine distribution programmes tailored to overcome individual countries' cold-chain weaknesses.

This study showed several simple and appropriate interventions to reduce vaccine freezing by allowing higher temperatures in several stages of the vaccine distribution system. Of the modified cold-chain scenarios explored, transport of hepatitis B vaccine in cold boxes but without ice packs, was the simplest approach. Vaccines remained within the physical and managerial structure of the cold chain, thus minimizing fears that cold-chain discipline would be undermined. Higher temperature allowances were restricted to discrete, short-term transport stages of the cold chain to reduce concerns about prolonged heat exposure. Additional advantages of this approach are the potential to increase capacity of vaccine transport while reducing costs and logistics associated with the freezing and handling of ice packs.

Unfortunately, frequent freezing in district ice-lined refrigerators and health centre refrigerators suggests that no-ice transport alone may not be enough to overcome cold-chain freezing. Any programme designed to reduce vaccine freezing must include steps to reduce freezing in refrigerators. Our study addressed this issue by storing vaccines in air-conditioned rooms and at ambient temperatures. Although these approaches together were successful in eliminating vaccine freezing and did not cause excessive vaccine heat exposure, they represent greater deviations from cold-chain infrastructure and standard procedures. Their adoption would require careful management.

We do not intend to suggest that the strategies evaluated would be universally applicable or to imply that possible cold-chain modifications are limited to those presented here. Careful consideration of local conditions and cold-chain weaknesses are needed before new approaches to vaccine distribution are introduced. Whether flexible distribution approaches, such as those presented in this study, are feasible in other countries depends on several factors (Box 1).

When strategies for flexible cold chains are considered, the probability of vaccine inactivation because of freezing must be weighed against the likelihood of inactivation because of heat exposure. A wellmanaged, flexible distribution system for vaccines, in which vaccine vial monitors are used to identify and reject damaged vaccines, could tolerate occasional vaccine discard due to heat exposure. Such a trade-off might be preferable to current levels of vaccine freezing.

Although action to reduce vaccine freezing clearly is needed, individual countries are unlikely to implement flexible cold-chain procedures without additional evidence and higher-level endorsement. More modelling of flexible cold-chain scenarios, along with international policy discussions, are urgently needed to facilitate the refinement and adoption of "freeze-proof", vaccine-distribution strategies. The added benefits of such strategies might include reduced distribution costs and expanded distribution capacity.

Acknowledgements

We thank Dr Indriyono Tantoro, the Indonesian Ministry of Health; the production and marketing divisions at PT Bio Farma; the Indonesian Expanded Programme on Immunization; East Java and Nusa Teggara Barat Province (NTB) Provincial Health Services; Pasuruan, Sumenep, West Lombok and Bima District Health Services; Kraton, Purwodadi, Pandian, Arjasa, Lingsar, Bayan, Rasanae Timur, and Wera Timur health centres; Sophie Newland; Fred Grabiner, Juhartini, and Ika Puspita Sari for their assistance during the study.

Funding: This study was conducted under the HealthTech project funded by the United States Agency for International Development, Cooperative Agreement No. GPH-A-00-01-00005-00, and the Affordable Technologies for Health Project and Children's Vaccine Program at PATH, both funded by the Bill & Melinda Gates Foundation.

Conflicts of interest: none declared.

References

1. World Bank. World development report: investing in health. New York: Oxford University Press; 1993.        

2. Galazka A, Milstien J, Zaffran M. Thermostability of vaccines. Geneva: World Health Organization; 1998. WHO document WHO/GPV/98.07.        

3. World Health Organization. The effects of freezing on the appearance, potency, and toxicity of adsorbed and unadsorbed DTP vaccines. Weekly Epidemiological Record 1980;55:385-92.        

4. Dietz V, Galazka A, van Loon F, Cochi S. Factors affecting the immunogenicity and potency of tetanus toxoid: implications for the elimination of neonatal and non-neonatal tetanus as public health problems. Bulletin of the World Health Organization 1997;75:81-93.        

5. Klotz SA, Normand R, Silberman R. Hepatitis B vaccine in healthy hospital employees. Infection Control 1986;7:365-9.        

6. Milhomme P. Cold chain study: danger of freezing vaccines. Canada Commununicable Disease Report 1993;19:33-8.        

7. Gold MS, Martin L, Nayda CL, Kempe AE. Electronic temperature monitoring and feedback to correct adverse vaccine storage in general practice. Medical Journal of Australia 1999;171:83-4.        

8. Burgess MA, McIntyre PB. Vaccines and the cold chain: is it too hot...or too cold? Medical Journal of Australia 1999;171:82.        

9. Guthridge SL, Miller NC. Cold chain in a hot climate. Australian and New Zealand Journal of Public Health 1996;20:657-60.        

10. Jeremijenko A, Kelly H, Sibthorpe B, Attewell R. Improving vaccine storage in general practice refrigerators. BMJ 1996;312:1651-2.        

11. Thakker Y, Woods S. Storage of vaccines in the community: weak link in the cold chain? BMJ 1992;304:756-8.        

12. Wawryk A, Mavromatis C, Gold M. Electronic monitoring of vaccine cold chain in a metropolitan area. BMJ 1997;315:518.        

13. Bishai DM, Bhatt S, Miller LT, Hayden GF. Vaccine storage practices in pediatric offices. Pediatrics 1992;89:193-6.        

14. Bell KN, Hogue CJ, Manning C, Kendal AP. Risk factors for improper vaccine storage and handling in private provider offices. Pediatrics 2001;107:E100.        

15. World Health Organization. TT vaccine — safer out of the cold chain? Geneva: World Health Organization; 1990.        

16. Hanjeet K, Lye MS, Sinniah M, Schnur A. Evaluation of cold chain monitoring in Kelantan, Malaysia. Bulletin of the World Health Organization 1996;74:391-7.        

17. Lugosi L, Battersby A. Transport and storage of vaccines in Hungary: the first cold chain monitor study in Europe. Bulletin of the World Health Organization 1990;68:431-9.        

18. Edstam JS, Dulmaa N, Nymadawa P, Rinchin A, Khulan J, Kimball AM. Comparison of hepatitis B vaccine coverage and effectiveness among urban and rural Mongolian 2-year-olds. Preventive Medicine 2002;34:207-14.        

19. Hepatitis B vaccine delivery outside the cold chain: the Long-An County, China, example. Global Perspective on Hepatitis 1991;2:3 (Newsletter of the International Task Force on Hepatitis B and the Programme for Appropriate Technology in Health [PATH]), copy available on request from the Global Programme for Vaccines and Immunization, World Health Organization, 1211 Geneva 27, Switzerland.        

20. Otto BF, Suarnawa IM, Stewart T, Nelson C, Ruff TA, Widjaya A, et al. At-birth immunisation against hepatitis B using a novel pre-filled immunisation device stored outside the cold chain. Vaccine 1999;18:498-502.        

21. Sutanto A, Suarnawa IM, Nelson CM, Stewart T, Soewarso TI. Home delivery of heat-stable vaccines in Indonesia: outreach immunization with a prefilled, single-use injection device. Bulletin of the World Health Organization 1999;77:119-26.        

22. Beauchamp J, Mansoor O. Temperature and the storage of vaccines. New Zealand Medical Journal 1992;105:135.        

23. Woodyard E, Woodyard L, Alto WA. Vaccine storage in the physician's office: a community study. Journal of the American Board of Family Practice 1995;8:91-4.        

24. Yuan L, Daniels S, Naus M, Brcic B. Vaccine storage and handling. Knowledge and practice in primary care physicians' offices. Canadian Family Physician 1995;41:1169-76.        

25. World Health Organization. Quality of the cold chain: WHO-UNICEF policy statement on the use of vaccine vial monitors in immunization services. Geneva: World Health Organization; 1999.        

26. Ruff TA, Gertig DM, Otto BF, Gust ID, Sutanto A, Soewarso TI, et al. Lombok hepatitis B model immunization project: toward universal infant hepatitis B immunization in Indonesia. Journal of Infectious Disease 1995;171:290-6.        

27. Everts J. Report of National Immunization Days conducted in February 1998, South Sudan. Consultancy Report. Geneva: World Health Organization; 1998.        

Submitted: 4 December 02 – Final revised version received: 13 February 03 – Accepted: 16 June 03

1 Correspondence should be sent to Mr Nelson at 1455 NW Leary Way, Seattle, WA 98107, USA (email: cnelson@path.org).