RESEARCH
A field trial of a survey method for estimating the coverage of selective feeding programmes
Essai sur le terrain d'une méthode d'enquête permettant d'estimer la couverture de programmes d'alimentation sélectifs
Ensayo sobre el terreno de un método de encuesta para estimar la cobertura de los programas de alimentación selectiva
Mark MyattI,1; Teshome FelekeII; Kate SadlerIII; Steve CollinsIII
IDivision of Epidemiology, Institute of Ophthalmology, University College London, London EC1V 9EL, England
IIValid International Ltd, London, England
IIICentre for International Child Health, Institute of Child Health, London, England
ABSTRACT
OBJECTIVE: To test a survey method for estimating the coverage of selective feeding programmes in humanitarian emergencies.
METHODS: The trial survey used a stratified design with strata that were defined using the centric systematic area sample method. Thirty 100 km² quadrats were sampled. The communities located closest to the centre of each quadrat were sampled using a case-finding approach.
FINDINGS: The method proved simple and rapid to implement and allowed overall and per-quadrat coverage to be estimated. Overall coverage was 20.0% (95% confidence intervals, 13.826.3%). Per-quadrat coverage ranged from zero (in nine quadrats) to 50% (in one quadrat). Coverage was highest in the quadrats closest to therapeutic feeding centres and in quadrats containing major roads leading to the towns in which therapeutic feeding centres were located.
CONCLUSION: The method should be used, in preference to WHO Expanded Programme on Immunization (EPI)-derived survey methods, for estimating the coverage of selective feeding programmes. Its use should also be considered when evaluating the coverage of other selective entry programmes or when coverage is likely to be spatially inhomogeneous.
Keywords: Nutrition surveys; Emergencies; Feeding methods; Child nutrition disorders-diagnosis; Child; Data collection/methods; Sampling studies; Malawi (source: MeSH, NLM).
RÉSUMÉ
OBJECTIF: Tester une méthode d'enquête permettant d'estimer la couverture de programmes alimentaires sélectifs dans des situations d'urgence humanitaire.
MÉTHODES: La méthode d'enquête testée est du type stratifiée, les strates étant définies par échantillonnage géographique systématique centré des zones. On a échantillonné trente carrés de 100 km². Les communautés les plus proches du centre de chaque carré ont fait l'objet d'un échantillonnage par recherche des cas.
RÉSULTATS: La méthode s'est avérée simple et rapide à mettre en uvre et a permis d'estimer la couverture globale et par carré. La couverture globale était de 20,0 % (intervalle de confiance à 95 % : 13,8 - 26,3 %). La couverture par carré allait de zéro (neuf carrés) à 50 % (un carré). Elle atteignait les valeurs les plus élevées dans les carrés les plus proches des centres d'alimentation thérapeutique et dans les carrés contenant des routes importantes conduisant aux villes où se situaient ces centres.
CONCLUSION: Il convient d'utiliser cette méthode, de préférence aux méthodes d'enquête correspondant à des variantes des méthodes du Programme élargi de vaccination (PEV), pour estimer la couverture des programmes d'alimentation sélectifs. Il convient également d'envisager son emploi lors de l'évaluation de la couverture d'autres programmes à admission sélective ou lorsque la couverture est susceptible d'être spatialement hétérogène.
Mots clés: Enquête nutritionnelle; Urgences; Méthodes alimentation; Troubles nutrition enfant/diagnostic; Enfant; Collecte données/méthodes; Etude échantillon; Malawi (source: MeSH, INSERM).
RESUMEN
OBJETIVO: Ensayar un método de encuesta para estimar la cobertura de los programas de alimentación selectiva en las emergencias humanitarias.
MÉTODOS: Se utilizó en este ensayo un diseño estratificado, con estratos definidos mediante el método de muestreo sistemático céntrico por áreas. Se muestrearon treinta cuadrados de 100 km². Las comunidades situadas más cerca del centro de cada cuadrado se muestrearon mediante una técnica de búsqueda de casos.
RESULTADOS: El método demostró ser de fácil y rápida aplicación y permitió estimar la cobertura general y por cuadrados. La cobertura general fue del 20,0% (intervalos de confianza del 95%: 13,8%26,3%). La cobertura por cuadrado varió entre cero (en nueve cuadrados) y 50% (en un cuadrado). La cobertura fue máxima en los cuadrados más próximos a los centros de alimentación terapéutica y en los cuadrados que contenían carreteras principales que conducían a los pueblos donde se encontraban los centros de alimentación terapéutica.
CONCLUSIÓN: El método ensayado debe utilizarse, con preferencia a los métodos de encuesta del Programa Ampliado de Inmunización (PAI) de la OMS, para estimar la cobertura de los programas de alimentación selectiva. Debería pensarse también en emplearlo a la hora de evaluar la cobertura de otros programas con selección de destinatarios o cuando se considere probable que la cobertura sea espacialmente heterogénea.
Palabras clave: Encuestas nutricionales; Urgencias médicas; Metodos de alimentación; Trastornos de la nutrición del niño/diagnóstico; Niño; Recolección de datos/métodos; Muestreo; Malawi (fuente: DeCS, BIREME).
Introduction
Programme coverage is assuming greater importance as an indicator for monitoring and evaluating humanitarian interventions. In 2003 specific coverage indicators for selective feeding programmes were included in the Sphere Project's humanitarian guidelines for the first time (1). The focus on coverage has called into question the appropriateness of traditional therapeutic feeding centre-based interventions (2, 3), but the absence of suitable methods for estimating the coverage of selective feeding programmes is hindering progress towards the acceptance of alternative interventions.
At present, an adaptation of the WHO Expanded Programme on Immunization (EPI) coverage survey method (47) is recommended for assessing the coverage of selective feeding programmes (8). This is a two-stage cluster-sampling approach which begins by dividing a population into clusters for which population estimates are available. A subset of clusters is selected in the first sampling stage. The probability of a particular cluster being selected is proportional to the size of the population in that cluster. Clusters with large populations are therefore more likely to be selected than clusters with small populations. This sampling procedure, known as probability proportional to size (PPS), helps to ensure that individuals in the area in which the programme is implemented have an equal chance of being sampled when samples of the same size are taken from each cluster in the second stage of the survey (9). In recognition of the difficulties of drawing a random sample in many developing countries (10), the EPI method uses a non-random sampling method in the second stage. The most commonly used second-stage sampling method is a proximity technique. The first household to be sampled is chosen by selecting a random direction from the centre of the cluster, counting the houses along that route, and picking one at random. Subsequent households are sampled by their physical proximity to the previously sampled household. Sampling continues until a sample of a fixed size has been collected. This sampling procedure is simple and requires neither mapping nor enumeration of households. This technique is consequently both quicker and cheaper than using simple random sampling in the second stage of the survey (11).
The EPI method does, however, have some drawbacks. The PPS process should result in a self-weighted sample, but it cannot be relied upon to do so if estimates of cluster population sizes are inaccurate (12). In addition, use of the PPS process locates the bulk of data collection in the most populous communities. This may mean that areas of low population density are not sampled (i.e. those areas consisting of communities likely to be distant from health facilities, feeding centres and distribution points). This may result in surveys reporting coverage as being adequate even when it is poor or non-existent in areas outside urban centres (13). With the exception of the first child, none of the children included in the within-cluster sample are selected using an equal probability selection method. This, together with the fact that the within-cluster sample size is usually too small to estimate coverage in any cluster with reasonable precision, means that the EPI method can return only a single estimate of coverage, even when coverage is spatially inhomogeneous. This is an important limitation, as the identification of areas where there is poor coverage is an essential step towards improving programme coverage and, hence, programme impact.
Despite these problems, an adaptation of the EPI method is frequently used to estimate the coverage of selective feeding programmes. The currently accepted method uses a two-stage cluster-sampled survey to estimate the prevalence of acute undernutrition in the programme area (8). Coverage is assessed either directly or indirectly. Coverage is estimated directly using formula 1 (Fig. 3) (8), but this approach introduces a further problem. The sample usually used in these surveys comprises 900 children collected in 30 clusters (8). This sample size allows the prevalence of acute undernutrition (i.e. wasting) to be estimated with reasonable precision, but the sample size available to estimate coverage depends on the prevalence of acute undernutrition found by the survey. When the aim of the survey is to estimate the coverage of a feeding programme for tackling severe acute undernutrition, the sample size will usually be too small to estimate coverage with reasonable precision. The effective sample size is further reduced by the design effect introduced by cluster- and proximity-sampling (12) which is likely to be considerable if coverage is spatially inhomogeneous (12) (i.e. if it is high in some communities, but low in others).
Formula 1 (see Fig. 3) includes children who may not be eligible for entry into the programme on the day of the survey (i.e. children who are in the recovery phase and whose weight-for-height is higher than that required for entry into the programme or who no longer exhibit nutritional oedema). These children, now in recovery, were recently severely undernourished. Formula 1 is, therefore, an estimator of recent coverage in a given period. This can be thought of as analogous to period prevalence estimates derived, for example, from 14-day recall. An alternative (point) estimator is the ratio of cases receiving treatment found in the sample to the total number of cases requiring treatment found in the sample (formula 2; Fig. 3). The use of this estimator is subject to the same sample size constraints as apply for formula 1. An indirect method is also used (8). This method uses a two-stage cluster-sampled survey to estimate the prevalence of acute undernutrition in the programme area, and coverage is estimated using formula 3 (Fig. 3). The denominator of this formula is subject to considerable uncertainty. The population estimate is usually derived from census data. In complex emergencies, census data may be inaccurate (e.g. as a result of political manipulation, the absence of a functioning civil society, population displacement and poor security). Population estimates are often corrected by the application of estimates of population growth which can seldom account for displacement, migration or high mortality in the target population. Survey sample sizes are usually too small to estimate the prevalence of severe acute undernutrition with precision. Such prevalence estimates will have wide confidence intervals relative to the magnitude of the estimate, which leads to similarly imprecise estimates of programme coverage.
Thus, when applied to the problem of assessing the coverage of selective feeding programmes, the EPI method has important limitations. This article describes a trial of an alternative method aimed at addressing the shortcomings described above.
Methods
Trial location
The trial survey aimed to estimate the coverage of a centre-based therapeutic feeding programme for the treatment of severe acute undernutrition in the Mchinji district of Malawi. This district had been subject to prolonged food shortages that resulted from erratic rainfall between 2000 and 2002, reduced agricultural inputs, the sale of a large proportion of the strategic grain reserve, and the reluctance of donors to release funds following allegations of fund misuse by the Government of Malawi. The prevalence of acute undernutrition reached 12.5% in March 2002. Semi-quantitative food security assessments performed in October 2001, April 2002 and August 2002 predicted continuing food shortages. A range of nutritional interventions were started in March 2002 including a therapeutic feeding programme based at three therapeutic feeding centres two of which were located at government health facilities and one at a "mission" hospital run by the Christian Health Association of Malawi. At the time of the survey reported here (March 2003) the prevalence of acute undernutrition in the district was returning to acceptable levels and had been estimated as being 2.9% in December 2002 (14).
Survey design
The trial survey used a stratified design with strata defined using the centric systematic area sample method (1417). This method involves dividing the survey area into non-overlapping squares of equal area (quadrats) and sampling the community or communities located closest to the centre of each quadrat. A 1:50 000 scale map of the district was available from the 1998 Malawi national census. A 10 x 10 km grid was overlaid on to this map. The position of the grid was chosen to maximize the area covered by the survey. All quadrats that had more than half of their area inside the district were sampled. Thirty 100 km² quadrats were sampled. The selected quadrats covered approximately 3000 km² (89.4%) of the 3356 km² total land area of Mchinji district. Areas not covered by the survey were at the periphery of the district where less than 50% of the quadrat area fell within the district boundaries.
The communities located closest to the centre of each quadrat were then sampled using a case-finding approach. The number of communities sampled from each quadrat was determined by the number of communities within that quadrat which could be sampled by a survey team in one day. This number varied between quadrats (see Table 1 web version only, available at: http://www.who.int/bulletin) and depended on the size of each community (both in terms of population size and physical extent) and on the distances between communities. Once sampling in a community had begun, it was continued until no further cases could be found. No communities were partially sampled. The location of the centre of each quadrat was identified by reference to the map. A list of communities to be sampled from each quadrat was made before the survey team visited the quadrat. The order of this list (which was also the order in which the communities were sampled) was determined by the proximity of each community to the centre of the quadrat, with the community closest to the centre of the quadrat being sampled first.
Case-finding
A case-finding approach was adopted for the within-community samples. Four methods were tested.
Screening of all children in a single community at a central location in their home community.
Screening of all children living in several communities at a central location outside their home community.
Screening of children living in a single community who had been identified by their mothers as being sick, thin or oedematous, at a central location in their home community.
Investigative case-finding which entailed screening, in their homes, of children identified as thin, sick or oedematous by the community health worker. Additional children from other households were also identified by mothers in each of the screened households. When survey teams were directed to an empty house, attempts were made to locate the occupants. In most cases, the mothers and children were close to home and were summoned by neighbours. If mothers and children could not be located immediately, the empty houses were revisited at the end of the day so that children in all identified households were screened.
Children reported as being in a therapeutic feeding centre on the day of the survey were visited and examined at the therapeutic feeding centre. Children at home were examined at home. For each method the investigators recorded the village of residence, name, sex, age, weight, height and the presence or absence of bilateral pitting oedema for each child. Examination of these data showed that all four case-finding methods identified the same children. However, the fourth method was considerably more efficient than the others, allowing a survey team to screen up to six communities in one day, and this was adopted as the case-finding method for the trial survey.
Case definitions
Cases were defined as children aged between 6 months and 5 years with < 70% of the weight-for-height median of the National Centre for Health Statistics (NCHS) reference population (19) or bilateral pitting oedema. These were also the entry criteria used for the therapeutic feeding programme. Receipt of treatment was ascertained by the child's presence in a therapeutic feeding centre or by documentary evidence (i.e. possession of a programme card or identity bracelet).
Programme coverage
Coverage in each quadrat was estimated as the ratio of cases receiving treatment found in the sample to the total number of cases found in the sample (formula 2; Fig. 3). Overall coverage was estimated by treating each quadrat as a stratum in a stratified sample (9) with sample weights derived from the population size of the communities sampled in each quadrat.
Data handling
Data were entered, checked, and cleaned using EpiInfo v6.04d (20) and analysed using the R Language for data analysis and graphics (21). The spatial distribution of coverage was investigated by estimating coverage in each quadrat and plotting the data using histograms and mesh maps.
Results
The survey method proved simple and rapid to implement. Data collection took three survey teams 10 days to complete. The data from the trial survey are shown in Table 1 (web version only, available at: http://www.who.int/bulletin). Overall coverage was 20.0% (95% confidence intervals, 13.826.3%). The distribution of per-quadrat coverage is shown in Fig. 1. Coverage ranged from zero (in nine quadrats) to 50% (in one quadrat). The spatial distribution of per-quadrat coverage is shown in Fig. 2. The lengths of the sides of the filled squares reflect the level of coverage found in each quadrat. The small open squares indicate quadrats with zero coverage. The legend gives examples of what the size of the square would be if estimated coverage were 100%, 50% or 25% as well as showing the symbol used to indicate zero coverage. The filled squares are continuously variable in size between zero and 100%. The crosses mark the approximate locations of the three therapeutic feeding centres. Dotted lines indicate the approximate locations of major roads. Coverage was highest in the quadrats closest to the therapeutic feeding centres and in the quadrats containing major roads leading to the towns in which the therapeutic feeding centres were located.
Discussion
Centric systematic area sampling is widely used in ecological studies to ascertain the spatial distribution of plant and animal species over wide areas, and in human geography to investigate point phenomena such as the distribution of specific types of retail business (1518). Its principal advantages are reported to be simplicity of use in the field (18), the ability to sample evenly across a wide area (18), simplicity of data handling (22), and the addition of a spatial dimension to survey data (16, 18). In practice, the method proved simple to use in the field although this might not be the case if useful maps are not available. Evenness of spatial sampling is virtually guaranteed by the use of a sampling grid and is only likely to be compromised when factors such as poor security prevent some quadrats from being sampled. The terrain of the trial survey location made it feasible to define quadrats by overlaying a simple grid on to a map of the programme area. More difficult terrains (e.g. programme areas divided by impassable rivers, gorges or military fronts) may require more imaginative strategies for quadrat location. Entry and management of data was considerably quicker than for a two-stage cluster-sampled survey because data analysis procedures require only summary data for each quadrat. Data handling procedures are simple enough to be performed without a computer or using a standard spreadsheet package. The use of case-finding, as opposed to probability sampling, in the second stage provides an exhaustive sample. The case-finding approach is likely to identify all, or nearly all, of the cases in sampled communities. This allows per-quadrat coverage to be calculated precisely and meaningful comparisons of per-quadrat coverage to be made. The ability to calculate per-quadrat coverage allows a spatial assessment of coverage, which provides information useful for programme management. In the trial location the survey results led to action being taken to address the zero coverage in the south-west corner of the programme area. In some situations it may be necessary to develop and test alternative case-finding procedures to ensure exhaustive within-community samples. Centric systematic area sampling may also 100be used for estimating the coverage of less restrictive programmes (e.g. supplementary feeding programmes) where investigative case-finding procedures may provide a less exhaustive sample than proved possible in the trial survey. This lack of exhaustiveness may be addressed by using other case-finding methods such as door-to-door screening. This would not affect survey costs because estimating the coverage of less restrictive programmes (i.e. programmes treating relatively common conditions) would require that only a single community be sampled from each quadrat. The exhaustiveness of case-finding procedures can be estimated using capturerecapture methods (2325). Estimating the exhaustiveness of the case-finding procedure and using this estimate to correct for under-reporting would enable the method to be used to obtain estimates of both prevalence and coverage from a single survey. The trial survey estimated the prevalence of severe acute undernutrition as 1.59% (95% confidence intervals, 1.34 1.88%), calculated as the ratio of the number of cases found in the sample to the total population aged less than 5 years in the sampled communities and assuming an exhaustive case-finding procedure. This estimate was broadly in line with that reported by a recent nutrition survey of the same population (14).
There are potential problems with the proposed survey method. The centric systematic area sampling method, like any systematic sampling method, can produce biased estimates if there is periodic variation in coverage and the sampling locations tend to coincide with this periodicity (26). This is difficult to control for without prior knowledge of the periodic variation, although simple checks, for example, ensuring that sampling locations are not all in valleys or all on hilltops, and adjusting the grid position accordingly, should help to minimize this problem. The trial survey used a proximity method to select the centrally located communities to be sampled in each quadrat. A more rigorous sampling procedure such as selecting communities in each quadrat at random or using a finer grid and selecting a single central community from each quadrat could be adopted, but this would increase the cost of the surveys and is likely to yield little increase in accuracy (22). The proposed survey method assumes homogeneity of coverage within quadrats. The area of each quadrat is, however, considerably smaller than the programme area (approximately one-thirtieth of the programme area in the case of the trial survey) making this assumption more plausible than that of homogeneity over an entire programme area.
Census population estimates of the sampled communities were used to derive sample weights. It has already been noted that census data may sometimes be unreliable. The method is, however, likely to be robust to such unreliability and, when accurate population estimates are not available, data may be analysed in the same way as if they came from a simple random sample (22). Alternatively, sample weights derived from rapid visual estimates or proxies of population size such as roof counts may be used without introducing any significant error (9). The ability of the case-finding method to return an exhaustive sample cannot be assumed. For example, a poor case-finding method might systematically exclude the children of minority groups or children living in households on the periphery of sampled communities leading to a bias in coverage estimates. The case-finding method should, therefore, be tested against the results of other methods such as door-to-door screening using capturerecapture methods (2325) prior to use in surveys.
An advantage of the proposed method is that it is likely to sample considerably more communities than would usually be sampled in an EPI-derived survey. The trial survey reported here sampled 151 communities as opposed to a maximum of 30 communities usually sampled in EPI-derived surveys. It should be noted that the PPS procedure often leads to more than one cluster being sampled from larger communities causing many EPI-derived surveys to sample fewer than the usual maximum of 30 communities. Table 2 compares the results of the trial survey with the results of a survey of Mchinji district undertaken in December 2002 using an EPI-derived method (14). The trial survey method screened more children from more communities, and found more cases in more communities, than the EPI-derived method, resulting in a more precise estimate of programme coverage. This method was also able to identify areas of low or zero coverage whereas the EPI-derived method could provide only a single district-wide estimate.
The trial survey took longer to complete than the EPI-derived survey. This is because one survey team took one day to sample one quadrat whereas in an EPI-derived survey a survey team can usually sample two clusters per day. The proposed survey method could, however, be as efficient as EPI-derived methods in situations of higher prevalence of acute undernutrition and in larger survey areas. It is important to note that active case-finding is central to both successful programme implementation and successful implementation of the proposed survey method. This means that the estimation of coverage could be integrated with programme outreach. This would allow continued estimation of coverage and prevalence to be undertaken as part of routine programme activity, removing the need for expensive and repeated cross-sectional surveys.
The method used to calculate coverage in the trial survey returned a point coverage estimate (i.e. from formula 2, Fig. 3), but a recent period coverage estimate may be calculated (from formula 1, Fig. 3) if required.
The survey method presented here addresses the shortcomings of EPI-derived methods as applied to the problem of estimating coverage in selective feeding programmes. The results indicate that the method described here should be used, in preference to EPI-derived survey methods, for estimating the coverage of selective feeding programmes. Its use should also be considered when evaluating the coverage of other selective entry programmes or when coverage is likely to be spatially inhomogeneous. 
Acknowledgements
This work was undertaken as part of the Community Therapeutic Care Research Program, a collaboration between Valid International and Concern Worldwide. It was supported by the United States Agency for International Development (USAID) through the Food and Nutrition Technical Assistance (FANTA) Project/Academy for Educational Development (AED); Nutrition for Health and Development Department, World Health Organization, Geneva; Ireland Aid; Concern Worldwide; Torchbox Ltd; and Brixton Health.
Conflicts of interest: none declared.
References
1. The Sphere humanitarian charter and minimum standards in disaster response, 2nd edition, Summer 2003 draft. Geneva: The Sphere Project; 2003. Available from: URL: www.sphereproject.org/handbook/
2. Collins S. Changing the way we address severe malnutrition during famines. Lancet 2001;358:498-501.
3. Griekspoor A, Collins S. Raising standards in emergency relief: how useful are Sphere minimum standards for humanitarian assistance? BMJ 2001;323:740-2.
4. Henderson RH, Sundaresan T. Cluster sampling to assess immunization coverage: a review of experiences with a simplified sampling method. Bulletin of the World Health Organization 1982;60:253-60.
5. Lemeshow S, Robinson D. Surveys to measure programme coverage and impact: A review of the methodology used by the Expanded Programme on Immunization. World Health Statistics Quarterly 1985;38:65-75.
6. Kok PW. Cluster sampling for immunisation coverage. Social Science and Medicine 1986;22:781-3.
7. Bennet S, Woods T, Liyanage WM, Smith DL. A simplified general method for cluster-sample surveys of health in developing countries. World Health Statistics Quarterly 1991;44:98-106.
8. Coulombier D, Ducos M, Henkens M, Pecoul B, Philips M, Rigal J, et al. Nutrition guidelines. Paris: Médecins Sans Frontières; 1995.
9. Moser CA, Kalton G. Survey methods in social investigation. 2nd edition. London: Heinemann Educational Books; 1979.
10. Ekanem EE. Field epidemiology: methodological constraints and limitations in the developing world. Public Health 1985;99:33-6.
11. Katz J, Yoon SS, Brendel K, West KP Jr. Sampling designs for xerophthalmia prevalence surveys. International Journal of Epidemiology 1997;26:1041-48.
12. Kish L. Survey sampling. New York: John Wiley; 1965.
13. Valadez JJ. Assessing child survival programs in developing countries testing lot quality assurance sampling. Boston: Harvard University Press; 1991.
14. Walsh A, Taifour H. Nutrition survey report: Salima District and Mchinji District. London: Save the Children Fund (UK)/Malawi Ministry of Health and Population; 2003.
15. Cole RG, Healy TR, Wood ML, Foster DM. Statistical analysis of spatial pattern. A comparison of grid and hierarchical sampling approaches. University of Waikato, New Zealand: Coastal Marine Group; 2000.
16. Rogers A. Statistical analysis of spatial dispersion: The quadrat method. London: Pion; 1974.
17. Cliff AD, Ord JK. Spatial processes: models and applications. London; Pion: 1981.
18. Krebs CJ. Ecological methodology. 2nd edition. Menlo Park: Addison Wesley Longman; 1999.
19. Dibbley MJ, Goldsby JB, Staehling NW, Trowbridge FL. Development of normalised curves for the international growth reference: Historical and technical considerations. American Journal of Clinical Nutriton 1987;46:736-48.
20. Dean AG, Dean JA, Coulombier D, Brendel KA, Smith DC, Burton AH, et al. EpiInfo version 6: A word processing, database, and statistics programme for public health on IBM-compatible microcomputers. Atlanta: Centers for Disease Control and Prevention; 1995.
21. Ihaka R, Gentleman R. R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics 1996;5:299-314.
22. Milne A. The centric systematic area sample treated as a random sample. Biometrics 1959;15:270-97.
23. Sekar CC, Deming WE. On a method of estimating birth and death rates and the extent of registration. American Statistical Association Journal 1949;44:100-15.
24. Seber GAF. The effect of trap response on tag recapture estimates. Biometrics 1970;26:13-22.
25. Hubert B, Desenclos JC. Evaluation of the exhaustivity and representativeness of a surveillance system by the capture-recapture method. Application to surveillance of meningococcal infections in France in 1989 and 1990. Revue d'Epidemiologie et de Sante Publique 1993;41:241-9.
26. Cochran WG. Sampling techniques. 3rd edition. New York: John Wiley and Sons; 1977.
(Submitted: 13 August 2003 Final revised version received: 10 February 2004 Accepted: 19 April 2004)
1 Correspondence should be sent to this author (email: mark@myatt.demon.co.uk).