RESEARCH
Reliability of data on caesarean sections in developing countries
Fiabilité des données relatives aux césariennes dans les pays en développement
Fiabilidad de los datos sobre las cesáreas en los países en desarrollo
Cynthia K. StantonI,1; Dominique DubourgII; Vincent De BrouwereII; Mar PujadesIII; Carine RonsmansIV
IDepartment of Population and Family Health Sciences, E4035, The Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, Maryland, 21205 USA (email:cstanton@jhsph.edu)
IIUnmet Obstetric Need Network, Department of Public Health, Institute of Tropical Medicine, Antwerp, Belgium
IIISchool of Community Health Sciences, Division of Epidemiology and Public Health, Nottingham, England
IVInfectious Disease Epidemiology Unit, London School of Hygiene and Tropical Medicine, London, England
ABSTRACT
OBJECTIVE: To examine the reliability of reported rates of caesarean sections from developing countries and make recommendations on how data collection for surveys and health facility-based studies could be improved.
METHODS: Population-based rates for caesarean section obtained from two sources: Demographic and Health Surveys (DHS) and health facility-based records of caesarean sections from the Unmet Obstetric Need Network, together with estimates of the number of live births, were compared for six developing countries. Sensitivity analyses were conducted using several different definitions of the caesarean section rate, and the rates obtained from the two data sources were compared.
FINDINGS: The DHS rates for caesarean section were consistently higher than the facility-based rates. However, in three quarters of the cases, the facility-based rates for caesarean sections fell within the 95% confidence intervals for the DHS estimate.
CONCLUSION: The importance of the differences between these two series of rates depends on the analyst's perspective. For national and global monitoring, DHS data on caesarean sections would suffice, although the imprecision of the rates would make the monitoring of trends difficult. However, the imprecision of DHS data on caesarean sections precludes their use for the purposes of programme evaluation at the regional level.
Keywords: Cesarean section/classification/statistics; Data collection/methods; Reproducibility of results; Sensitivity and specificity; Developing countries (source: MeSH, INSERM).
RÉSUMÉ
OBJECTIF: Examiner la fiabilité des taux de césariennes rapportés par les pays en développement et formuler des recommandations sur la façon dont on pourrait améliorer la collecte des données destinées aux enquêtes et aux études menées dans les installations de santé.
MÉTHODES: Les auteurs ont comparé pour six pays en développement les taux de césariennes dans la population obtenus à partir de deux sources: les enquêtes démographiques et sanitaires (DHS) et le recensement des césariennes réalisées dans les installations de santé transmis par le Réseau des Besoins Obstétricaux non couverts, en association avec des estimations du nombre de naissances vivantes. Ils ont réalisé des analyses de sensibilité en utilisant différentes définitions du taux de césariennes et ont comparé les taux obtenus à partir des deux sources de données.
RÉSULTATS: Les taux de césariennes obtenus d'après les DHS étaient régulièrement plus élevés que ceux établis d'après les données des installations. Cependant, dans les trois quarts des cas, les taux de césariennes établis d'après les données des installations se situaient dans les intervalles de confiance à 95% de l'estimation d'après les DHS.
CONCLUSION: L'importance des différences relevées entre les deux séries de taux dépend de la perspective de l'analyste. Pour la surveillance à l'échelle nationale ou mondiale, les données des DHS sur les césariennes devraient suffire, bien que l'imprécision des taux ne facilite pas le suivi des tendances. Néanmoins, l'imprécision des données des DHS exclut qu'on utilise celles-ci pour évaluer les programmes au niveau régional.
Mots clés: Césarienne/classification/statistique; Collecte données/méthodes; Reproductibilité des résultats; Sensibilité et spécificité (Epidémiologie); Pays en développement (source: MeSH, NLM).
RESUMEN
OBJETIVO: Examinar la fiabilidad de las tasas de cesárea notificadas en los países en desarrollo y formular recomendaciones sobre la manera de mejorar la recopilación de datos para las encuestas y los estudios basados en establecimientos de salud.
MÉTODOS: Se compararon, para seis países en desarrollo, las tasas poblacionales de cesárea obtenidas a partir de dos fuentes (las Encuestas de Demografía y Salud (EDS) y los registros basados en centros de salud de la red Unmet Obstetric Need Network) y las estimaciones del número de nacidos vivos. Se llevaron a cabo análisis de sensibilidad usando distintas definiciones de tasa de cesárea, y se procedió a comparar las tasas obtenidas a partir de las dos fuentes de datos.
RESULTADOS: Las tasas de cesárea según las EDS fueron sistemáticamente mayores que las tasas basadas en los establecimientos. Sin embargo, en tres cuartas partes de los casos, las segundas se encontraban dentro de los intervalos de confianza del 95% correpondientes a los valores de las EDS.
CONCLUSIÓN: La importancia de las diferencias entre esas dos series de tasas dependerá de la perspectiva del analista. A efectos de la vigilancia nacional y mundial, bastarían los datos de las EDS sobre las cesáreas, aunque la imprecisión de los valores dificultaría el seguimiento de las tendencias. Debido a esa imprecisión, se descarta el uso de los datos de EDS sobre las cesáreas para evaluar los programas a nivel regional.
Palabras clave: Cesárea/estadística; Recolección de datos/clasificación/métodos; Reproducibilidad de resultados; Sensibilidad y especificidad; Países en desarrollo (fuente: DeCS, BIREME).
Introduction
Population-based rates for caesarean section are considered an important indicator of access to essential obstetric care in developing countries. Various United Nations agencies promote this indicator for use in the evaluation of safe motherhood programmes (1). The interpretation of the indicator is difficult in the absence of information on the indication for the procedure. Studies have shown that as prevalence of caesarean sections increases, the decision on whether to perform such an operation is increasingly based on fetal rather than maternal indications (2). Moreover, high rates for caesarean sections may hide unnecessary interventions (36), even in settings where there is an unmet need for caesarean sections. Nevertheless, in settings where rates for caesarean section are low and mainly done for maternal indications, crude population-based rates for caesarean section remain very useful as they are likely to reflect life-saving care.
Data on population-based rates for caesarean section come from two sources: household surveys in which women of reproductive age are interviewed regarding recent births, and hospital-based studies which rely on record review for the numerator and estimates of the number of live births from censuses or surveys for the denominator. There has been no validation of data obtained by either method.
The objective of this study was to assess the reliability of rates for caesarean section obtained from these two data sources. The specific sources we examined were: women's self-report of a caesarean section in Demographic and Health Surveys (DHS) and hospital-based data on caesarean sections collected in Unmet Obstetric Need (UON) studies (7, 8). Recommendations for improvement of data quality are discussed for both methods.
Methods
Data sources
The data examined here came from regional estimates of the caesarean section rate between 1989 and 1999 in six countries: Benin, Burkina Faso, Haiti, Mali, Morocco and Niger. The data collection procedures for the two methods are described below. The reference points of the estimates are shown in Table 1.
DHS surveys are based on nationally-representative samples of women of reproductive age. The sample sizes for the countries in this study ranged from 600013 000, and for the individual regions from 350 to 3500. The questionnaire includes a complete live-birth history for every woman. Questions regarding maternal health care are asked of women who have had a live birth during the three or five years prior to the survey. No maternal health-care questions are asked of women who have had a stillbirth. Data for the numerator of the DHS caesarean section rate come from all positive responses to the question: Was (NAME) delivered by caesarean section? The denominator of the DHS caesarean section rate is the number of live births in the last three or five years.
The DHS interviewer manual states the following: "A caesarean section is a delivery of a baby through an incision in the woman's abdomen and womb, rather than through the birth canal. Such a delivery is necessary for some women due to pregnancy complications. Find out whether the baby was delivered by an operation and not through the birth canal" (9). Discussions with DHS staff suggested that in some countries this definition is specifically discussed during interviewer training to ensure that interviewers do not confuse caesareans with episiotomies. However, most agreed that such discussions are probably rare, because trainers believe the definition to be easily understood.
UON studies document all major surgical obstetric interventions performed within a specific administrative region. This involves an in-depth review of the records from all public and private health facilities with surgical capacity, including, for example, a referral hospital in the capital city, to include women seeking care outside their local region. Record review is conducted for caesarean section, laparotomy, hysterectomy, craniotomy, embryotomy, version and extraction, and symphisiotomy. The following data are extracted: address, indication for the intervention and outcome for the woman and her fetus.
The numerator for the facility-based caesarean section rate in the present study was the number of caesarean sections plus laparotomies and hysterectomies if the infant was delivered during this procedure. These additional procedures were included to increase the comparability of DHS and facility-based caesarean section rates, on the assumption that women could confuse these procedures. The number of live births within the region is required for the denominator and is generally calculated using data from censuses or surveys. Methods commonly used to estimate the number of births include use of the general fertility rate applied to the number of women, or the crude birth rate applied to the total population count. There are several concerns regarding estimation of the denominator.
The most recent census may have been up to 10 years prior to the year of the health facility-based study. The population size must therefore be projected, assuming constant rates of growth, over an extended period of time.
There are common patterns of error in census-based fertility data which necessitate adjustment (10). However, these adjustments are not carried out in all countries.
Census or survey-based fertility indicators may be available only at the national level and these may obscure regional differences.
Analysis
The reliability of reported rates of caesarean section was assessed in two steps. First, sensitivity analyses were conducted to document the effects on the caesarean section rate of using different definitions of the numerator for the DHS estimates and different definitions of the denominator for the health facility-based estimates. Second, the series of caesarean section rates obtained from DHS and health facility-based data for the same time and place were compared.
The numerator is the focus of the sensitivity analyses for the DHS estimates because it is based on women's self-report, which has not been validated. The denominator for the DHS caesarean section rate is of less concern because these data are collected in a standardized manner across all countries and because DHS fertility data are generally considered acceptable.
We compared caesarean section rates from DHS estimates using two definitions. Definition A excluded cases for which data on mode of delivery were missing from the numerator, but not from the denominator; represented multiple births born by caesarean section multiple times in the numerator; included all reported caesareans in the numerator; and excluded data on stillbirths in both the numerator and denominator because these data are not collected in the DHS. This is the definition used in DHS final country reports. Definition B excluded cases for which data on mode of delivery were missing from both the numerator and the denominator; excluded the double-counting of caesareans for multiple births from the numerator; and excluded caesareans reported as having taken place in a home, a health post, a community health centre or dispensary (i.e. in locations without surgical capacity). It should be noted that not all definition B caesareans were reported as having been attended by a doctor. In some cases, the most qualified person present was reported to be a nurse or midwife, and in rare occurrences, traditional birth attendants or family members. Given that neither of the variables "place of birth" or "attendant at birth" have been validated, the decision over which variable to prioritize is arbitrary. For this analysis, it was assumed that place of birth is more easily recalled than the birth attendant(s).
The denominator is the focus of the sensitivity analyses for the health facility-based rates because the method used to estimate it varies across sites and is generally dependent on available data. The numerator is of less concern because it is based on surgical records believed to be more complete than individual medical records, and because of UON efforts to collect data in both public and private health facilities. Only five private health facilities were not visited (two in Mali and three in Morocco; representing 0.2% and 0.4% of the caesarean sections in Mali and Morocco, respectively).
A series of regional rates for caesarean section were compared. These used health facility-based data on caesareans that ended in live births for the numerator and three different means for estimating the number of live births in the denominator. For definition C, the number of live births was estimated by projecting the number of women of reproductive age from the most recent census to the reference year of the UON study, and then applying DHS, region-specific general fertility rates. For definition D, the number of live births was estimated by projecting the regional population as in definition C, and then applying DHS, region-specific crude birth rates. Definition E was the same as definition D, except that a national level, census-based crude birth rate was applied to the projected regional population counts. The year of the most recent census in the six countries was: Benin, 1992; Burkina Faso, 1996; Haiti, not available; Mali, 1998; Morocco, 1994; Niger, 2001.
Results
Table 2 presents caesarean section rates obtained from the DHS data. For 11 of 31 regional rates, there was no difference between those obtained using definitions A and B. For the remaining 20 rates, those obtained using definition B were lower than those from definition A. The confidence intervals around the caesarean section rates obtained using definition B were very wide, however, and encompassed the rate obtained using definition A for all but one region (Oueme, Benin). These lower rates resulted from the exclusion of caesarean sections performed in improbable locations. The double-counting of caesarean sections for multiple births and the way in which missing data were handled had negligible effects on the caesarean section rate. The number of caesarean sections performed in improbable locations varied across countries. In Burkina Faso, all reported caesareans occurred in plausible sites. For the other countries the percentages of caesareans conducted in improbable locations were: Haiti, 3.8%; Niger, 13.9%; Benin, 27.0%; Morocco, 21.7%; and Mali, 15.9%.
Table 3 presents rates for caesarean section obtained using health facility-based data. Confidence intervals are not presented for these figures because the health facility-based estimates were not drawn from a sample. It is likely that these estimates are affected by non-sampling error, which is not easily quantifiable. Applying DHS, region-specific general fertility rates (definition C) or crude birth rates (definition D) produced very similar results. In nine of 31 regions, the caesarean section rates were the same; in 17 of the 22 regions in which the rates differed, the difference between the two was only 0.1 percentage point. In all but one of the cases where the rates obtained using definitions C and D differed, the use of definition D produced higher caesarean section rates than use of definition C.
Definitions D and E (using DHS regional crude birth rates and national census-based crude birth rates, respectively) also produced very similar results. Twelve of the 28 regions for which data were available showed exactly the same caesarean section rate. Nine of the estimates differed only by 0.1 percentage point and the remaining regions differed by up to 0.6 percentage point. Where definitions D and E gave different rates, the rates obtained using definition E were always lower than those based on definition D.
Fig. 1 compares the caesarean section rates based on definition B (DHS data) with those obtained using definition C (health facility-based data). The upper line in Fig. 1 represents the slope obtained from regressing the DHS rates on the health facility-based rates; the lower line is a 45º angle representing equality between the two sets of rates. In 23 of the 31 regional estimates shown here, the DHS caesarean section rate was higher than the corresponding health facility-based rate. The intercept in the equation was 0.363 and the regression coefficient on the facility-based rates was 1.05 (95% confidence intervals, 0.701.15). Furthermore, the 95% confidence intervals for the DHS estimates encompassed the facility-based estimate in 23 of the 31 regions. The relative difference between the two series of estimates was large in some cases, however. In some regions in Mali (Tombouctou), Niger (Dosso, Zinder Diffa), Burkina Faso (east, north, central/south) and Morocco (oriental, centre-south) the DHS rate was three to nine times higher than the health facility-based rate. The number of maternal deaths among women who had caesarean sections in health facilities was too small to affect the comparison to the DHS rates which are based only on the reports of women who survived.
Discussion
This study examined the sensitivity of several definitions on population-based caesarean section rates from two sources, and then assessed the reliability of the rates obtained. The sensitivity analysis for the DHS rates suggests that double-counting caesareans for multiple births and different ways of handling missing data had negligible effects on the rates obtained. Excluding caesareans done in improbable locations, however, decreased caesarean section rates by up to a third in some regions. The sensitivity analysis for the facility-based rates suggests that once the population size has been projected to the appropriate reference year, the method used for estimating the number of live births has surprisingly little effect on the rate of caesarean sections.
The comparison of rates for caesarean sections obtained using DHS estimates with those from health facility-based data showed that the rates obtained using DHS data are generally about one-third higher than those obtained using facility-based data. However, in three-quarters of the cases, the facility-based rates fall within the 95% confidence intervals for the DHS rate. Whether differences between the two series of rates are of a magnitude that would alter interpretation of the data depends on the analyst's perspective. In absolute terms, the differences are extremely small. In relative terms, the differences are in some cases very large. For example, in one area (North, Burkina Faso), caesarean section rates obtained from DHS data were nine times greater (2.8%) than those determined using facility-based data (0.3%).
In areas of very low use of essential obstetric care where caesareans are mostly done to save the woman's life, a small increase in caesarean section rates from, say, 0.2% to 0.6%, may represent averted maternal deaths, and current practice is to interpret this increase as meaningful progress. The considerable variation in the rates between survey and health facility-based data in some regions, however, suggests that the survey-based estimates are too imprecise for the purposes of evaluation in a small area. The small variation in caesarean section rates obtained by means of calculating the denominator using health facility-based data, however, is promising.
It should be noted that the health facility-based rates presented here were extracted from very detailed data collected for UON studies. These findings therefore do not necessarily represent all facility-based efforts to generate caesarean section rates. For example, recent reports documenting population-based caesarean section rates from health facilities suggest that the omission of data from private health facilities is common (11, 12).
When considering survey data, it is important to note that the DHS are not designed to precisely measure indicators as low as the caesarean section rates presented here. Sample designs for DHS surveys aim for at least 1000 women per domain to produce estimates of prevalence of contraceptive use or child mortality (13). To detect a difference between caesarean section rates of 1.0 and 0.8%, with a 95% confidence interval and 80% power, a sample of approximately 18 200 births would be required in each domain. Clearly, the DHS is not the appropriate tool if highly precise caesarean section rates are needed. If indicators reflecting approximate levels suffice, it is promising that the DHS estimates do not differ significantly from the facility-based rates, which are based on careful review of hospital records.
Why the rates for caesarean sections obtained using DHS estimates were higher than those obtained from the health facility-based data is unclear. It is unlikely that a woman would forget having a caesarean section, or that she would fabricate a story about having had one. The rates from DHS were higher despite the inclusion in the health facility-based rates of emergency hysterectomies. Given the increasing use of episiotomy in the developing world (14), and the extremely low caesarean section rates presented here, it is doubtful that women frequently confuse episiotomy with caesarean section. Some of the differences may be due to occasional under-reporting of caesarean operations in surgical wards, though this is believed to be uncommon.
Based on these findings, we propose the following recommendations regarding data collection and use. Survey training should specify that a caesarean section entails the non-vaginal, operative delivery of the baby. Collection of survey data should be designed such that only women who report giving birth in a health facility with surgical capacity are asked to answer the question on mode of delivery. DHS data users should pay careful attention to the codes in the original questionnaire for "place of birth" and how these have been translated into the DHS data sets which are made available to the public. Knowledge of a country's health system is often required to identify health facilities with surgical capacity.
Health facility-based caesarean section rates should rely on a projection of the number of women of reproductive age and region-specific general fertility rates, when possible. This recommendation is based on an assumption that there is less error in a projection of numbers of women of reproductive age than there is in a projection of the entire population. It is also recommended that survey-based indicators of fertility be used to estimate the number of live births, as survey data are often more current and result from more detailed birth histories than are collected in a census. Population totals should always be projected to the reference year of interest, particularly when the most recent census was more than a couple of years before the reference year. Data from all public and private facilities with surgical capacity should be collected, and completeness of surgical records should be established.
Conclusions
DHS surveys tend to produce higher caesarean section rates than data collected from health facilities, although in most cases, the sampling error surrounding the DHS estimates encompasses the facility-based rate. For the purposes of global and national monitoring, the precision of DHS caesarean section rates in countries where the number of caesareans is very low is sufficient to reflect seriously inadequate provision of services. Given the imprecision of these indicators, care is required in the interpretation of trends. For the purposes of the evaluation of regional programmes, DHS caesarean section rates are insufficiently precise. 
Acknowledgements
This work was undertaken as part of an international research programme IMMPACT (Initiative for Maternal Mortality Programme Assessment. See: http://www.abdn.ac.uk/immpact), funded by the Bill & Melinda Gates Foundation, the Department for International Development, the European Commission and the United States Agency for International Development. The funders have no responsibility for the information provided or views expressed in this paper. The views expressed herein are solely those of the authors.
Competing interests: none declared.
References
1. UNICEF, WHO, UNFPA. Guidelines for monitoring the availability and use of obstetric services. New York: United Nations Children's Fund, World Health Organization and United Nations Population Fund; 1997.
2. Barrett JF, Jarvis GJ, Macdonald HN, Buchan PC, Tyrrell SN, Lilford RJ. Inconsistencies in clinical decisions in obstetrics. Lancet 1990;336:549-51.
3. Francome C, Savage W. Caesarean section in Britain and the United States, 12% or 24%: is either the right rate? Social Science and Medicine 1993;37:1199-218.
4. van Roosmalen J, Van der Does CD. Caesarean birth rates worldwide: a search for determinants. Tropical and Geographical Medicine 1995;47:19-22.
5. Kabir A, Steinmann WC, Myers L, Khan MM, Herrera EA, Yu S, et al. Unnecessary cesarean delivery in Louisiana: An analysis of birth certificate data. American Journal of Obstetrics and Gynecology 2004;190:10-9.
6. Lomas J, Enkin M. Variations in operative delivery rates. In: Chalmers I, Enkin M, Keirse MJNC, editors. Effective care in pregnancy and childbirth. New York: Oxford University Press; 1989.
7. Belghiti A, De Brouwere V, Kegels G, Van Lerberghe W. Monitoring unmet obstetric need at district level in Morocco. Journal of Tropical Medicine and International Health 1998;3:584-91.
8. Unmet Obstetric Need Network. Tackling unmet need for major obstetric interventions. Part 1: concept, general principles and international network. Antwerp: Unmet Obstetric Need Network. Available from: http://www.uonn.org/uonn/pdf/Guide1.pdf
9. ORC Macro. Interviewer's manual for use with model A questionnaire for high contraceptive prevalence countries (English), Demographic and Health Surveys basic documentation No. 3. Calverton (MD): ORC Macro; 2001.
10. United Nations Department of International Economic and Social Affairs. Estimation of fertility based on information about children ever born. Manual X. Indirect techniques for demographic estimation. New York: United Nations; 1983.
11. AMDD Working Group on Indicators. Using UN process indicators to assess needs in emergency obstetric services: Morocco, Nicaragua and Sri Lanka. International Journal of Gynaecology and Obstetrics 2003;80:222-30.
12. AMDD Working Group on Indicators. Using UN process indicators to assess needs in emergency obstetric services: Niger, Rwanda and Tanzania. International Journal of Gynaecology and Obstetrics 2003;83:112-20.
13. Macro International. Sampling manual DHS-III basic documentation No. 6. 2-3. Calverton Maryland: Macro International; 1996.
14. Buekens P. Over-medicalisation of maternal care in developing countries. In: De Brouwere V, Van Lerberghe W, editors. Safe motherhood strategies: a review of the evidence. Antwerp: ITG Press; 2001. p.195-206.
(Submitted: 9 July 2004 Final revised version received: 24 October 2004 Accepted: 29 October 2004)
1 Correspondence should be sent to this author.