Using lot quality assurance sampling to improve immunization coverage in Bangladesh

Youssef Tawfik,¹ Shamsul Hoque,² & Mizan Siddiqi³

OBJECTIVE: To determine areas of low vaccination coverage in five cities in Bangladesh (Chittagong, Dhaka, Khulna, Rajshahi, and Syedpur).
METHODS: Six studies using lot quality assurance sampling were conducted between 1995 and 1997 by Basic Support for Institutionalizing Child Survival and the Bangladesh National Expanded Programme on Immunization.
FINDINGS: BCG vaccination coverage was acceptable in all lots studied; however, the proportion of lots rejected because coverage of measles vaccination was low ranged from 0% of lots in Syedpur to 12% in Chittagong and 20% in Dhaka’s zones 7 and 8. The proportion of lots rejected because an inadequate number of children in the sample had been fully vaccinated varied from 11% in Syedpur to 30% in Dhaka. Additionally, analysis of aggregated, weighted immunization coverage showed that there was a high BCG vaccination coverage (the first administered vaccine) and a low measles vaccination coverage (the last administered vaccine) indicating a high drop-out rate, ranging from 14% in Syedpur to 36% in Dhaka’s zone 8.
CONCLUSION: In Bangladesh, where resources are limited, results from surveys using lot quality assurance sampling enabled managers of the National Expanded Programme on Immunization to identify areas with poor vaccination coverage. Those areas were targeted to receive focused interventions to improve coverage. Since this sampling method requires only a small sample size and was easy for staff to use, it is feasible for routine monitoring of vaccination coverage.

Keywords Immunization programs; Poverty areas; Quality control; Sampling studies; Bangladesh (source: MeSH).

Mots clés Programmes de vaccination; Zone pauvreté; Contrôle qualité; Etude échantillon; Bangladesh (source: INSERM).

Palabras clave Programas de inmunización; Areas de pobreza; Control de calidad; Muestreo; Bangladesh (fuente: BIREME).

Introduction

A quarter of Bangladesh’s 120 million people live in urban areas. The country has one of the highest urban growth rates in the world: cities are growing at an annual rate of 6%, three times faster than in the country as a whole. Extreme poverty, the dearth of land ownership, and a lack of employment opportunity in rural areas are forcing people to migrate to urban areas. This rapid growth of urban areas has resulted in the widespread development of slums with their attendant problems of overcrowding and poor sanitary conditions, both of which pose public health challenges for health authorities (1).

Immunization coverage among children in Bangladesh increased from 2% in 1985 to 56% in 1997 (2–9). To control infectious diseases more effectively, the government has set a goal of raising immunization coverage to 85%. Immunization rates are regularly measured through the National Immunization Coverage Survey, which follows the 30-cluster sampling method recommended by WHO. While the survey offers valuable information about the status of overall immunization coverage, it does not indicate the precise location of areas where vaccination coverage is low (10). Methods of lot quality assurance sampling have been adapted from industry for use in primary health care; using only a small sample size, these methods can differentiate clearly between areas that have or have not met the performance targets (11).

Basic Support for Institutionalizing Child Survival (BASICS) and the National Expanded Programme on Immunization project in Bangladesh conducted six surveys using lot quality assurance methods in five cities in Bangladesh (Chittagong, Dhaka, Khulna, Rajshahi, and Syedpur) in 1995 and 1997. This paper presents the results of the studies and briefly explains how they were used at the local level to identify areas with low immunization coverage in order to target them for intervention.

Methods

The six surveys were conducted in two sets. In set one, the objective was to assess immunization coverage throughout the cities of Chittagong, Khulna, and Rajshahi; the lots were all city wards (distinct administrative units). In set two, the focus was on assessing coverage in city slums (in Dhaka city’s zones 7 and 8, and in Syedpur city); the lots were all slums with populations of over 10 000 people.

Threshold levels

Two different upper and lower thresholds for accepting or rejecting lots were selected. In set one, the national immunization coverage goal — 85% of children fully immunized by 1 year of age — was selected as the upper threshold of what would be considered as an acceptable level of performance. The lower threshold was based on results of previous coverage surveys (2–9, 12) and was set at 60%. In set two, since previous studies suggested that immunization coverage in slum areas is significantly lower than the national target (13), the upper threshold was chosen as 60% and the lower threshold as 40%.

Confidence levels and sample size

Two different confidence levels and methods of determining the sample size per lot were used in the two sets. This was intended to allow the national immunization programme managers to compare results and select the most suitable method for future use. In set one, the confidence interval was 80%. In set two, the confidence interval was 95%.

In set one, the number included in each sample per lot was calculated using the conventional lot quality assurance methods described by Lemeshow et al. (14, 15). Using available reference tables for lot quality assurance methods (14), the lot sample size needed was found to be 13, and the minimum number of immunized children that would be accepted was nine. In set two, calculations of sample size were carried out according to the description in the WHO manual Monitoring immunization services using the lot quality technique (16). The required aggregated sample for all lots was calculated and divided by the number of lots to arrive at the sample number needed for each lot. Thus, in set two the lot sample size was 16. Table 1 shows the number of children sampled in each lot and the total sample size in the five cities.

The proportion of fully immunized children sampled in each ward was weighted according to the fraction of the city (or zone in Dhaka) it represented. The sum of the weighted coverage percentages in all wards gave the overall immunization coverage level for the cities or zones studied.

Inclusion criteria

Children aged 12–23 months were included in the sample, in accordance with the ages at which children are included in the Bangladesh National Immunization Coverage Surveys. Age was confirmed by vaccination card or, when this was not available, by asking mothers (using a standardized Bangla-English calendar and major holidays as reference points). In Bangladesh, a fully vaccinated child is one who has received the following: BCG vaccine; three doses of diphtheria-pertussis-tetanus (DPT) vaccine; three doses of oral polio vaccine, given with DPT; and one dose of measles vaccine. All vaccines must be administered by the time the child is 1 year of age. For the purpose of this survey a ‘‘defective unit’’ was defined as a child who had not been fully vaccinated before the age of 1 year. The survey included questions to determine reasons why a child had not been fully vaccinated.

A list of eligible couples, compiled by the national family planning programme and nongovernmental organizations for family planning services, was used as the basis for selecting households; this list was deemed to be the most complete register of households. The addresses of households in each catchment area were randomly selected from the list of eligible couples. If there was no child eligible for inclusion in this study in the selected household, the neighbouring house was tried until an eligible child was found.

Results

The coverage of BCG vaccination exceeded the upper threshold for all lots. However, the proportion of lots rejected because the coverage of measles vaccination was low varied from 0% (0/9) of lots in Syedpur to 12% (5/41) in Chittagong and 20% (6/ 30) in Dhaka’s zones 7 and 8. The proportion of lots rejected because an inadequate number of children in the sample had been fully vaccinated varied from 11% (1/9) in Syedpur to 30% (9/30) in Dhaka. Lots rejected because of low coverage of full vaccination included all lots rejected because of low coverage of measles vaccination (Table 1).

Table 2 presents the weighted immunization coverage. BCG vaccination coverage exceeded the national target of 85% in all study areas. However, measles vaccination coverage varied from 60% in Dhaka to 83% in Khulna. None of the study areas met the national target for full vaccination. The closest was Khulna with 78%; the lowest coverage was found in Dhaka’s zone 8, where only 58% of children were fully vaccinated. Table 2 also shows that there was a high drop-out rate during the BCG to measles vaccinations, ranging from 14% in Syedpur to 36% in Dhaka’s zone 8.

Discussion

Usefulness of the sampling method

A decade ago, statisticians and epidemiologists foresaw that the lot quality assurance sampling technique — traditionally used in industry for quality-assurance purposes — would be suitable for assessing a variety of primary health care services. The method is particularly valuable for measuring immunization coverage because it overcomes an important limitation of the commonly used 30-cluster survey technique, that is, the inability to identify areas with low immunization coverage.

The lot quality assurance sampling method used to assess the coverage of vaccinations in children in five cities in Bangladesh provided information needed by programme managers to help them target interventions to specific areas with low coverage. The results were then used to focus limited resources on areas with unacceptable coverage rates.

In Chittagong, Khulna, and Rajshahi, maps were made to illustrate the results and to plan interventions to improve the immunization coverage in areas of low coverage. Governmental and nongovernmental staff then focused on these areas, and parents were counselled about the importance and timing of vaccination. Additional outreach sites for vaccinations were created and health workers from wards where coverage was adequate were asked to help, particularly in planning outreach vaccination services, in neighbouring wards where coverage was inadequate. Social mobilization and communication strategies, including community meetings, the use of folk songs, and street theatre were used to raise the demand for immunization. These interventions should be followed up to determine if they were effective.

Variation in vaccination coverage

In spite of selecting lower thresholds (40% lower limit) for the second set of studies, which took place in the slums of Dhaka and Syedpur, many lots in Dhaka were rejected because the coverage of measles vaccination did not even reach this low level. This indicates that there is a discrepancy in vaccination coverage between cities, with the slums in Dhaka needing more attention.

Experience with using two different confidence intervals suggests that the 95% confidence interval is preferable to the 80% because it adds more precision to the results without adding much to the sample size. Using two methods to determine the sample size per lot in the two survey sets allowed the project staff from the National Expanded Programme on Immunization to gain experience in both methods and select the one that would be most suitable for use in the future. Both methods were easy for staff to use, and they had no particular preference for either method.

The high level of BCG vaccination coverage indicated that immunization services were universally accessible. However, there was a high drop-out rate between the BCG and measles vaccinations, mainly attributed to parents’ lack of knowledge about the vaccination schedule, fear of the side-effects of vaccination, and parent’s, and sometimes health workers’, reluctance to vaccinate infants during illness. Reasons for drop-out detected by our surveys confirm results of other surveys (2–9).

The lot quality assurance sampling method provided quick and reliable information on areas at extreme ends of the spectrum of vaccination coverage: those where coverage rates were acceptable and which required no additional attention and those where rates were low and which were in urgent need of support.

Acknowledgements

This article was made possible through support from the Basic Support for Institutionalizing Child Survival Project (BASICS II) of the US Agency for International Development, Bureau for Global Programs, Population, Health and Nutrition, under the terms of Contract No. HRN-C-00-99-00007-00. We appreciate the cooperation of the National Expanded Programme on Immunization in Bangladesh and the efforts of the staff of BASICS Bangladesh for their contributions in the field. We also thank Dr Susan Robertson at WHO, and Ms Rebecca Fields and Dr Robert Steinglass at BASICS for their valuable comments. We are grateful for the support provided by BASICS Information Center.

Conflicts of interest: none declared.

Résumé

Utilisation de la méthode LQAS (échantillonnage par lots pour l’assurance de la qualité) pour améliorer la couverture vaccinale au Bangladesh

OBJECTIF: Identifier les zones de faible couverture vaccinale dans cinq villes du Bangladesh (Chittagong, Dhaka, Khulna, Rajshahi et Syedpur).
MÉTHODES: Six études utilisant la méthode d’échantillonnage par lots pour l’assurance de la qualité (LQAS) ont été réalisées entre 1995 et 1997 par Basic Support for Institutionalizing Child Survival et le Programme national élargi de vaccination du Bangladesh.
RÉSULTATS: La couverture vaccinale par le BCG était acceptable dans tous les lots étudiés. Toutefois, la proportion de lots rejetés en raison de la faiblesse de la couverture antirougeoleuse allait de 0% à Syedpur à 12% à Chittagong et 20% dans les zones 7 et 8 de Dhaka. La proportion de lots rejetés parce qu’un nombre insuffisant d’enfants de l’échantillon avaient reçu la totalité des vaccins allait de 11% à Syedpur à 30% à Dhaka. De plus, l’analyse des données agrégées et pondérées a montré une bonne couverture par le BCG (le premier vaccin administré) et une faible couverture par le vaccin antirougeoleux (le dernier vaccin administré), ce qui indique un taux élevé d’abandon (de 14% à Syedpur à 36% dans la zone 8 de Dhaka).
CONCLUSION: Au Bangladesh, pays dont les ressources sont limitées, les résultats des enquêtes réalisées selon la méthode d’échantillonnage par lots pour l’assurance de la qualité ont permis aux responsables du Programme national élargi de vaccination d’identifier les secteurs de faible couverture vaccinale. Ces secteurs ont été désignés pour faire l’objet d’interventions ciblées. Comme la méthode n’exige qu’un échantillon de petite taille et qu’elle est facile à appliquer, elle peut être utilisée pour la surveillance de routine de la couverture vaccinale.

Resumen

Muestreo de control de la calidad de los lotes para mejorar la cobertura de inmunización en Bangladesh

OBJETIVO: Determinar las áreas de baja cobertura vacunal en cinco ciudades de Bangladesh (Chittagong, Dhaka, Khulna, Rajshahi, y Syedpur).
MÉTODOS: Entre 1995 y 1997, Apoyo Básico para Institucionalizar la Supervivencia Infantil y el Programa Ampliado de Inmunización Nacional de Bangladesh llevaron a cabo seis estudios en los que se emplearon técnicas de muestreo de control de la calidad de los lotes.
RESULTADOS: La cobertura vacunal con BCG fue aceptable en todos los lotes estudiados; sin embargo, la proporción de lotes rechazados como consecuencia de una cobertura antisarampionosa baja se situó entre el 0% de los lotes de Syedpur, pasando por el 12% de Chittagong, y el 20% de las zonas 7 y 8 de Dhaka. La proporción de lotes rechazados porque el número de niños de la muestra totalmente vacunados era insuficiente se situó entre el 11% de Syedpur y el 30% de Dhaka. Además, el análisis de la cobertura de inmunización agregada y ponderada puso de manifiesto una alta cobertura vacunal con BCG (la primera vacuna administrada) y una baja cobertura con vacuna antisarampionosa (la última vacuna administrada), lo que indica una alta tasa de abandono, entre el 14% en Syedpur y el 36% en la zona 8 de Dhaka.
CONCLUSIÓN: En Bangladesh, país con recursos limitados, los resultados de estudios en que se usaron métodos de muestreo de control de la calidad de los lotes permitió a los administradores del Programa Ampliado de Inmunización Nacional identificar áreas de baja cobertura vacunal. Se pudo así focalizar en esas zonas las intervenciones encaminadas a mejorar la cobertura. Dado que sólo requiere una pequeña muestra y resulta de fácil manejo para el personal, la aplicación de este método de muestreo a la vigilancia sistemática de la cobertura de vacunación es una opción factible.

References

1. Tawfik Y et al. Extending immunization services to the urban poor in Bangladesh: a strategy for action. Basic Support for Institutionalizing Child Survival (BASICS), 1996 (unpublished report).        

2. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1990.        

3. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1991.        

4. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1992.        

5. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1993.        

6. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1994.        

7. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1995.        

8. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1996.        

9. National coverage evaluation surveys. Bangladesh. Dhaka, Directorate General of Health Services, 1997.        

10. Singh J et al. Evaluation of immunization coverage by lot quality assurance sampling compared with 30-cluster sampling in a primary health centre in India . Bulletin of the World Health Organization, 1996, 74: 269–274.        

11. Valadez J. Assessing child survival programs in developing countries — testing lot quality assurance sampling. Harvard, MA, Harvard University Press, 1991.        

12. Perry H et al. Assessment of EPI services in zone 3 of Dhaka city: urban MCH-FP Extension Project. Dhaka, Basic Support for Institutionalizing Child Survival (BASICS), 1996.        

13. Jinnah SIA. Slum-only survey in Dhaka city. Dhaka, Bangladesh Ministry of Health, 1993.        

14. Lemeshow S et al. Adequacy of sample size in health studies. Chichester, John Wiley, 1990.        

15. Lemeshow S et al. Lot quality assurance sampling: single- and double-sampling plans. World Health Statistics Quarterly, 1991, 44: 115–132.        

16. Monitoring immunization services using the lot quality technique. Geneva, World Health Organization, 1996 (unpublished document WHO/VRO/TRAM/96.01).        

¹ Senior Child Survival Advisor, Academy for Educational Development, Support for Analysis and Research in Africa (SARA) Project, 1875 Connecticut Avenue, North West, Washington DC 20009, USA (email: ytawfik@aed.org). Correspondence should be addressed to this author.

² Director, Primary Health Care, Directorate General of Health Services, Mohakhali, Dhaka, Bangladesh.

³ Acting Country Representative, Basic Support for Institutionalizing Child Survival (BASICS), Gulshan, Dhaka, Bangladesh.

Ref. No. 99-0044