Rodolfo A. Bulatao¹ & John A. Ross²

OBJECTIVE: To assess maternal and neonatal health services in 49 developing countries.
METHODS: The services were rated on a scale of 0 to 100 by 10 – 25 experts in each country. The ratings covered emergency and routine services, including family planning, at health centres and district hospitals, access to these services for both rural and urban women, the likelihood that women would receive particular forms of antenatal and delivery care, and supporting elements of programmes such as policy, resources, monitoring, health promotion and training.
FINDINGS: The average rating was only 56, but countries varied widely, especially in access to services in rural areas. Comparatively good ratings were reported for immunization services, aspects of antenatal care and counselling on breast feeding. Ratings were particularly weak for emergency obstetric care in rural areas, safe abortion and HIV counselling.
CONCLUSION: Maternal health programme effort in developing countries is seriously deficient, particularly in rural areas. Rural women are disadvantaged in many respects, but especially regarding the treatment of emergency obstetric conditions. Both rural and urban women receive inadequate HIV counselling and testing and have quite limited access to safe abortion. Improving services requires moving beyond policy reform to strengthening implementation of services and to better staff training and health promotion. Increased financing is only part of the solution.

Keywords Maternal health services; Perinatal care; Health services accessibility; Delivery of health care; Family planning; Program evaluation; Comparative study; Developing countries (source: MeSH, NLM).

Mots clés Service santé maternelle; Soins périnatals; Accessibilité service santé; Délivrance soins; Contrôle naissances; Evaluation programme; Etude comparative; Pays en développement (source: MeSH, INSERM).

Palabras clave Servicios de salud materna; Atención perinatal; Accesibilidad a los servicios de salud; Prestación de atención de salud; Planificación familiar; Evaluación de programas; Estudio comparativo; Países en desarrollo (fuente: DeCS, BIREME).

Introduction

Experts in developing countries have rated family planning programme efforts (1 – 3), thus facilitating useful comparisons between countries (4) and allowing the effects of programme services on outputs to be analysed (5). Expert ratings for HIV/ AIDS programmes have also recently been introduced (6).

Such indicators of programme and service adequacy that are comparable across countries can be useful in identifying deficiencies, interpreting morbidity patterns and planning improvements. On the basis of judgements made by experts, we have obtained indicators for maternal and neonatal health services in 49 developing countries. The indicators, designed after a review of existing indicators for maternal health services (7 – 12), cover both routine and emergency care The present paper describes overall patterns across services and makes preliminary comparisons between countries.

Methods

The Futures Group International conducted the study in 1999 and early 2000, identifying individual consultants or consultant institutions for each of 49 developing countries and working with them to identify and recruit expert raters. Of the 10 to 25 raters selected in each country, at least two were from each of the following sources: the ministry of health (working in maternal and child health, hospitals, training, management information or elsewhere); private health care providers, including nongovernmental and community organizations; resident staff of international donors and related agencies; and medical schools and universities, associations of obstetricians and gynaecologists, nurses and midwives, and similar groups of knowledgeable observers.

Service providers comprised 42% of the 1037 raters; 61% of all the raters were physicians. The average country rater had eight years of experience at the national level and at least as much additional experience at the provincial, district or community level. There was no evidence of substantial systematic biases in ratings associated with the training or experience of raters (13).

The experts were asked to rate services on an 81-item questionnaire, the Maternal and Neonatal Programme Effort Index. The items in the questionnaire covered antenatal care, treatment for complications of delivery, neonatal care, immunization, the control of sexually transmitted infections, and many other areas. The items were grouped not by medical condition but with reference to different stages involved in organizing and delivering care, from setting policy to attending to patients (Fig. 1). Also included, although not represented in the diagram, was the provision of family planning, the one substantive area separately covered. Taken together the elements represented programme effort as distinct from health outcomes.

The experts rated services from 0 to 5. For example, in relation to the statement "All pregnant women have their labour monitored", a rating of 5 indicated that this was completely true whereas a rating of 0 indicated this was completely false. The ratings were multiplied by 20 to give a range of 0 to 100. A slightly different scale was employed for the assessment of access to services, indicating the percentage of pregnant women with adequate access to each service. Thus a range of 0 to 100 was again used. The experts were expected to give their own opinions but were free to consult with colleagues or refer to available health system or household survey data.

Of the 49 countries, 23 were in Africa, 13 were in the Americas and 13 were in Asia (Table 6). They included the largest developing countries as well as countries of special policy interest, comprising 84% of the population of the developing regions. Partly because of its size, India was treated in more detail than the other countries: programmes were rated separately for each of 14 states containing some 85% of the national population. Population-weighted averages of the ratings for the states were used as national ratings.

Results

Capacity of health centres and district hospitals to provide maternal health services

The mean ratings across countries of the capacity of health facilities and the rankings by item are shown in Table 1. That a health centre had adequate antibiotic supplies was about equally likely to be true or false (mean rating = 52). A health centre was more likely to have the capacity to administer antibiotics intravenously (mean rating = 61), but there was only a 52% probability of the antibiotics being available. Health centres tended not to use the partograph or to have transportation available in the event of obstructed labour and were especially unlikely to be able to offer manual vacuum aspiration or electric suction.

District hospitals scored somewhat better than health centres. Hospitals were best at doing the things that health centres were supposed to do (rated 67), but their capacity to provide blood transfusions was, on average, close to even odds.

Variation between countries in the capacity of facilities was largest with regard to the use of the partograph, as indicated by the standard deviations across countries per item (not shown). This simple tool had apparently been easily assimilated into the procedures of health centres in some countries whereas it was still largely unknown in others.

Although the above ratings appear low, the question arises whether they, and others indicated below, are higher than they should be. A comparison of selected items with household survey data (see below) reveals no substantial discrepancies in ratings. There were few systematic differences between raters and there was no particular reason to distrust any category of raters (13). The ratings given are averages, and many countries indeed have much lower ratings. In addition, the ratings apply only to existing facilities. Ratings may be high in a situation where facilities are too few.

Access to services in rural and urban areas

The raters indicated the proportion of pregnant rural and urban women with adequate access to each service (Table 2). On average across eight items, their ratings imply that 68% of urban women and 39% of rural women had such access.

For access to a 24-hour district hospital, the figures for urban and rural women were 81% and 58% respectively. For antenatal care, the figures were similar. For delivery care by a trained professional attendant, the figure was also similar for urban women, but for rural women it was lower, under 50%. Treatment for postpartum haemorrhage, obstructed labour and complications of abortion was accessible to slightly more than two-thirds of urban women but to only one-third of rural women. Fewer than half of urban women and only one-fifth of rural women had access to safe abortion, the least accessible service.

In some particulars, the ratings of health services accessibility and of the capacity of facilities are in agreement. Overall, however, they do not correspond to one another. The wordings of items differ, and comparison is further complicated by the absence of information on the geographical distribution of pregnant women in relation to that of facilities.

Maternal and neonatal health care received

Ratings were obtained for maternal care received at antenatal visits, at delivery and for neonates (Table 3). These ratings were somewhat higher than those for the capacity of facilities and for access, indicating better than even odds that care would be received. This was partly because the items placed proportionally less emphasis on obstetric emergencies and more on routine types of care.

The best odds for receiving care related to immunization. This probably reflected the vigour of the worldwide immunization effort. Nevertheless, the ratings of 76 - 78 suggested that many children were not covered.

Very poor ratings were obtained for voluntary HIV counselling and testing (rated 30). HIV was not regarded as a pressing problem in a number of the countries covered, but even where it should be so regarded, the ratings were low. At 51 the rating for examination and treatment for syphilis was also low. Care for sexually transmitted infections appeared to be a serious weakness, lagging well behind other areas.

Various other types of care fell between immunization and sexually transmitted infections. In general the care of neonates appeared somewhat better than antenatal or delivery care. The weakest aspect of neonatal care was prophylactic eye treatment.

With regard to antenatal care, tetanus immunization was the most frequently available type of care, while counselling on HIV and sexually transmitted infections was the least frequently available. Hypertension received slightly more attention at antenatal visits than iron folate supplementation, which in turn received more attention than counselling on danger signs in pregnancy. Thus, even during antenatal visits, there was an apparent bias towards medical interventions as opposed to nutritional supplementation and simple counselling.

At delivery, the odds of having a trained professional attendant were slightly better than even (rated 56). The odds were better for the encouragement of breast feeding, counselling on care of the umbilical cord, and checking for hypertension, anaemia and other conditions, presumably performed by a less trained person. The odds for receiving care in an emergency were also better than even (rated 55), but labour was less likely to be monitored in order to provide warning of an emergency (rated 52). Any monitoring may not be up to standard, given that the capacity to use the partograph had a lower rating (Table 1). The item with the lowest rating was a scheduled check-up within 48 hours after delivery.

Family planning provision

Ratings of family planning provision combined elements of the capacity of facilities, access and care received (Table 4). These ratings, ranging from 36 to 71, were not particularly high relative to other items despite substantial previous donor assistance in this area.

The likelihood of postpartum family planning being routinely offered was rated 56 for district hospitals and 51 for health centres. Table 2 showed that 61% of urban women had access to postpartum family planning services, suggesting that, proportionally, urban facilities offering this service attended to more deliveries than facilities not offering it. However, only 36% of rural women had such access, indicating that, whatever services health centres and hospitals provided, substantial proportions of rural women did not have access to the facilities themselves, or at least to those facilities that were adequately staffed and equipped.

Policy and support services

Ancillary services were divided into five areas (Table 5). Of these, broad policy was generally the strongest. Having a basic policy and a service director with a high rank in the bureaucracy were rated 72 and 67 respectively. Similarly, such other policy items as authorizing appropriate personnel to provide services, consulting interested groups on policy development, and issuing frequent public statements of support were better rated than most other ancillary services. The lowest ratings for policy items were given to official approval for treating complications of abortion and to active policy implementation through high- level reviews and action plans.

The weakness of implementation was also reflected in poor scores for resources. The odds were essentially even that the budget for public services would be adequate. In contrast the odds were better that the private sector would be active.

Active monitoring is required in order to ensure that policy is effective. Item ratings varied in this area. They were best for surveys of maternal events (rated 64), followed by statistical reporting systems (and their use for monitoring and decision-making) and then by centralized listings of facilities. Hospital reviews of all their maternal deaths was the item with the lowest rating. This was unfortunate, since such reviews could trigger immediate improvements in practices. In general, review and follow-up were particular weaknesses of maternal health programmes. As earlier noted, the odds were barely even, or worse than even, that other reviews of services would take place, whether high-level reviews or the equivalent at the client level, scheduled client check-ups. Providers were possibly too busy with clients or too absorbed in competing activities to review their work with a view to improvement.

Educating the public about pregnancy complications, safe places to deliver and harmful practices, an important adjunct to the provision of services, received relatively little attention. All the items on health promotion were in a tight cluster of ratings below 50.

For staff training, on the other hand, the spread in ratings was quite large. Hands-on training as part of medical curricula was rated relatively highly, whereas training for new medical staff received one of the lowest ratings. Refresher training within the preceding five years was given intermediate ratings. As might be expected, doctors were less likely to receive either new-provider or refresher training than nurses and midwives, but the difference in each case was only 2 points.

Variation between countries

Variation between countries was substantial. Table 6 shows one indicator of this, a rating of national access to maternal care obtained by averaging all the urban and rural access ratings, weighted by population. Over 80% of women were estimated to have access to services in Jamaica and the Islamic Republic of Iran; under 30% had such access in Ethiopia, Nepal, Pakistan, and the Republic of Yemen. This large difference primarily reflected differences in rural access. Urban access also varied but the gap was smaller.

For other non-access items (not shown) the gaps tended to be somewhat smaller even between these two extreme groups of countries. In particular the gaps were only half as wide for ratings of average policy and budget adequacy.

Africa had a preponderance of countries with very weak access ratings, but here, and even more so on other continents, the variation in ratings was wide. There were some extreme contrasts between neighbouring countries, e.g., between the Islamic Republic of Iran and Pakistan and between the Dominican Republic and Haiti.

Conclusions

How credible are expert ratings of reproductive health programmes? Such measures obtained over three decades for family planning programmes, based on similar types of questions and using expert groups recruited by the same procedures, have proved useful for research and policy purposes. They have been essential in analyses of the contribution of family planning programmes to contraceptive use and fertility transition and have been used by persons arguing both for and against such a contribution (14, 15). The ratings have also been used as a means of drawing attention to weaknesses in these programmes in particular countries and of mobilizing and focusing national efforts.

Two decades after their introduction, ratings of family planning programmes were validated against objective data for two countries (16). Similar detailed validation of ratings of maternal health programmes is not possible at present, but limited comparisons can be made with household survey data to show that the experts are generally accurate. The Demographic and Health Surveys (DHS) asked national samples of women who their attendants were at any birth in the previous five years and whether the women had received tetanus injections beforehand. Responses are available for surveys conducted between 1994 and 1998 for 27 of the 49 countries considered in this paper (17). The proportions of births in the presence of a trained attendant, as indicated by the DHS, agree well with the current ratings for attended births, which generally refer to 1999. The correlation across countries is 0.70. The correlation is even stronger, at 0.83, with ratings as of three years previously, i.e. effectively for 1996. For the proportion receiving at least two tetanus injections the correlations are also strong at 0.62 for current ratings and 0.74 for ratings as of three years previously.

The level of ratings is also of interest. For the countries covered by the DHS, the mean percentage of births in the presence of a trained attendant was 55, virtually identical to the mean current rating of 56 for these countries and higher than the rating of 43 for three years previously. The DHS indicated the mean percentages receiving at least one and at least two tetanus injections to be 69% and 46% respectively, these values being below the current mean rating of 77 for "needed" tetanus injections in the same countries but encompassing the mean rating of 66 for three years previously. The complications of these comparisons cannot be explored here. The important point is that expert ratings match, to some degree, data derived from large, representative household surveys, providing a quicker method of obtaining an overall programme evaluation.

What do the ratings imply for policy in this area? Clearly, there is a need for greater programme effort. With only a 56% likelihood that a typical service item is adequate, maternal health programmes in developing countries have serious deficiencies. The wide range in mean country ratings shows that some countries face much greater challenges than others.

The need to improve services is greater in rural areas. Only 39% of rural women were estimated to have adequate access to the average service item, as opposed to 68% of urban women. Rural women were especially disadvantaged in respect of the treatment of emergency obstetric conditions. Since rural access ratings are among the most variable between countries, the lower ratings may be remediable.

What services most require improvement? Emergency obstetric services are a possible choice, being much less adequate than many routine services for pregnant women, such as antenatal care, nutrition supplementation during pregnancy and the care of neonates. Raising less concern are such services as immunization and the encouragement of breast feeding at delivery, whose ratings are among the best. Nevertheless, some services that could be considered routine received even lower ratings than emergency care. Among these were HIV counselling and testing (rated 30), safe abortion in both urban and rural areas (ratings 45 and 21 respectively), and the scheduling of a postnatal check-up within 48 hours (rating = 42).

However, it is not possible to conclude that the weakest services should have top priority. This analysis has not considered why particular ratings are low, the interactions between them, the epidemiological implications, and the costs of remedies. Such issues would have to be taken into account if ratings were to be used to justify focusing on the weakest areas, or on the weakest country programmes, which would also require attention to government commitment and capacity.

Arguably, however, national policy reform should not be the main focus. Ratings of official maternal health policies are better than many of the ratings for actual services. Implementation is clearly the crux of the matter, and increased financing is only part of the answer. The likelihood of an adequate budget is rated close to 50%. While not good, this is not substantially worse than other ratings. Between countries at opposite extremes, moreover, the contrasts in budget adequacy are substantially weaker than the contrasts in access to services. The improvement of training must be another part of the answer. The training of new providers is uncommon, and refresher training occurs only slightly more frequently. The adequacy of training has not been ascertained. Health promotion is another problematic area. All media-related items are rated, on average, below 50. Whether this is due to inattention, a lack of resources or a lack of skills in health promotion, the weakness in this area suggests substantial needs.

One cannot rely on the private sector to compensate for inadequate public services. Where service ratings are weak, ratings for private sector activity may also be weak. This is especially true for South Asia generally.

The picture may seem bleak, but there are indications that maternal health care services have improved since the 1994 Cairo conference. Raters estimated that adequacy improved by 10 points on the typical item over three years. Assuming this to be reliable, such performance, if sustained, could eventually lead to substantial progress.

Acknowledgements

The data were provided by 1037 raters, who cannot be acknowledged individually. They were recruited by consultants in each country, who also made important contributions. Advice on the design of the study and on the questionnaire was received from Carla Abou-Zahr, Wendy Graham, Marge Koblinsky, Deborah Maine, Cindy Stanton, Patricia Stephenson, Amy Ong Tsui and others. Katherine Abel and Katharine Cooper-Arnold assisted with data processing. The MEASURE Evaluation project and the Rockefeller Foundation provided financial support.

Conflicts of interest: none declared.

Résumé

Notation des soins de santé maternelle et néonatale dans les pays en développement

OBJECTIF: Evaluer les services de santé maternelle et néonatale dans 49 pays en développement.
MÉTHODES: Dans chaque pays, les services ont été notés de 0 à 100 par 10 à 25 experts. Pour établir la note, divers éléments étaient pris en compte : services d'urgence et services courants (y compris les services de planification familiale) dans les centres de santé et les hôpitaux de district, accès à ces services pour les femmes des zones rurales et urbaines, probabilité pour les femmes de recevoir certains soins anténatals et obstétricaux, et éléments d'appui (politiques, ressources, surveillance, promotion de la santé et formation).
RÉSULTATS: La note moyenne n'était que de 56, mais avec de grandes variations d'un pays à l'autre notamment au niveau de l'accessibilité des services dans les zones rurales. Des notes relativement bonnes ont été obtenues pour les services de vaccination, les soins anténatals et le conseil en matière d'allaitement au sein. Elles étaient particulièrement faibles pour les soins obstétricaux d'urgence en milieu rural, les services d'interruption de grossesse et le conseil relatif au VIH.
CONCLUSION: Les efforts consacrés à la santé maternelle dans les pays en développement sont nettement insuffisants, surtout dans les zones rurales. Les femmes des zones rurales sont désavantagées à de nombreux égards, mais plus particulièrement en ce qui concerne le traitement des urgences obstétricales. Dans les zones rurales comme dans les zones urbaines, le dépistage et le conseil en matière de VIH sont insuffisamment proposés aux femmes, qui n'ont en outre qu'un accès limité aux services d'interruption de grossesse. Pour redresser la situation, il faut aller au-delà de la réforme des politiques pour renforcer la mise en œuvre des services et améliorer la formation des personnels et la promotion de la santé. L'augmentation du financement ne représente qu'une partie de la solution.

Resumen

Evaluación de los servicios de salud materna y neonatal en los países en desarrollo

OBJETIVO: Evaluar los servicios de salud materna y neonatal en 49 países en desarrollo.
MÉTODOS: Unos 10 - 25 expertos puntuaron el funcionamiento de los servicios con arreglo a una escala de 0 a 100 en cada país. La evaluación abarcó los servicios de urgencia y los servicios ordinarios, incluida la planificación familiar, de los centros de salud y los hospitales de distrito, el acceso a esos servicios por las mujeres, tanto rurales como urbanas, la probabilidad de que las mujeres recibieran determinadas formas de atención prenatal y obstétrica, y elementos de apoyo de los programas tales como las políticas, los recursos, la vigilancia, la promoción de la salud y la capacitación.
RESULTADOS: La puntuación media fue sólo de 56, pero con amplias diferencias entre los países, especialmente en lo referente al acceso a los servicios en las zonas rurales. Se asignaron puntuaciones comparativamente buenas a los servicios de inmunización y a los aspectos de la atención prenatal y los consejos sobre la lactancia materna. Obtuvieron en cambio una puntuación particularmente baja la atención obstétrica de urgencia en las zonas rurales, el aborto y los consejos relacionados con el VIH.
CONCLUSIÓN: Las actividades de los programas de salud materna emprendidos en los países en desarrollo adolecen de graves deficiencias, especialmente en las zonas rurales. Las mujeres de estas zonas están desfavorecidas en muchos aspectos, sobre todo en lo tocante al tratamiento de los problemas obstétricos urgentes. Tanto las mujeres rurales como las urbanas carecen de servicios suficientes de asesoramiento y pruebas sobre el VIH y tienen un acceso muy limitado a la posibilidad de abortar sin riesgos. A fin de mejorar los servicios, es necesario rebasar el marco de las reformas de política para reforzar la implantación de servicios y mejorar la formación del personal y la promoción de la salud. El aumento de la financiación es sólo una parte de la solución.

References

1. Mauldin WP, Lapham RJ. Measuring family planning program effort in LDCs: 1972 and 1982. In: Birdsall N, editor. The effects of family planning programs on fertility in the developing world. Washington (DC): World Bank; 1985 (Staff Working Paper No. 677).         

2. Ross JA. Mauldin WP. Family planning programs: Efforts and results, 1972-94. Studies in Family Planning 1996,27:137-47.         

3. Ross JA, Cooper-Arnold K. Effort scores for family planning programs: An alternative approach. Glastonbury, Connecticut: Futures Group International; 2001 (Measure Evaluation project working paper no. WP-01-28).         

4. Mauldin WP, Berelson B. Conditions of fertility decline in developing countries. Studies in Family Planning 1978;9:89-147.         

5. Schultz TP. Human capital, family planning, and their effects on population growth. American Economic Review 1994;84:255-60.         

6. Stover J, Rehnstrom J, Schwartlander B. Measuring the level of effort in the national and international response to HIV/AIDS: The AIDS program effort index (API). XIIIth International AIDS Conference, Durban, South Africa, 9 - 13 July 1999.         

7. World Health Organization. Indicators to monitor maternal health goals: Report of a technical working group, Geneva, 8 - 12 November 1993. Geneva: World Health Organization; 1994.         

8. Campbell O, Koblinsky M, Taylor P. Off to a rapid start: Appraising maternal mortality and services. International Journal of Gynecology and Obstetrics 1995;48 Suppl:S33-52.         

9. Koblinsky M, McLaurin K, Russell-Brown P, Gorbach P. Indicators for reproductive health program evaluation: Final report of the subcommittee on safe pregnancy. Chapel Hill, North Carolina, USA: Evaluation Project, Carolina Population Center; 1995.         

10. Bulatao RA, Shrestha LB. Key indicators for reproductive health projects. Washington (DC): World Bank; 1996.         

11. Maine D, Akalin MZ, Ward VM, Kamara A. The design and evaluation of maternal health programs. New York: Columbia University Center for Population and Family Health; 1997.         

12. United Nations Children's Fund, World Health Organization, United Nations Population Fund. Guidelines for monitoring the availability and use of obstetric services. New York: United Nations Children's Fund; 1997.         

13. Bulatao RA, Ross JA. Rating maternal and neonatal health programs in developing countries. Chapel Hill: University of North Carolina; 2000 (Measure Evaluation project working paper no. WP-00-26).         

14. Pritchett LH. Desired fertility and the impact of population policies. Population and Development Review 1994;20:1-55,248-51.         

15. Bongaarts J. The role of family planning programmes in contemporary fertility transitions. In: Jones GW, editor. The continuing demographic transition. Oxford: Clarendon Press; 1997. p.422-3         

16. Mauldin WP, Ross JA, Kekovole J, Barkat - e - Khuda, Barkat A. Direct and judgmental measures of family planning program inputs. Studies in Family Planning 1995;26:287-95.         

17. El-Zanaty F, Hussein EM, Shawky, GA, Way AA, Kishor S. Egypt Demographic and Health Survey 1995. Cairo: National Population Council; 1996. Calverton, Maryland: Macro International Inc.; 1996.         

¹ Independent Consultant, 8201 16th Street, Silver Spring, Maryland, MD 20910, USA (email: Bulatao@erols.com). Correspondence should be addressed to this author.

² Senior Fellow, Futures Group International, Glastonbury, Connecticut, USA.

Ref. No. 00-0972