Comparison of two training strategies for essential newborn care in Brazil

Suely Arruda Vidal,¹ Luca Ronfani,² Suzana da Mota Silveira,³ Maria J. Mello,³ Erlene R. dos Santos,⁴ Roberto Buzzetti,⁵ & Adriano Cattaneo⁶

OBJECTIVE: To compare the effectiveness of two training strategies for improving essential newborn care in the state of Pernambuco, Brazil.
METHODS: Eight hospitals were selected, divided into two groups of four, and paired by geographical, structural, and functional characteristics. Doctors and nurses working at hospitals in Group 1 were given a conventional 5-day training course. Those in Group 2 were given the same manual used by Group 1 but the training course was organized as self-directed learning, with the participants having 5 weeks to complete the course. Participants' knowledge was tested at baseline, immediately after the course, and 3-6 months later. Participants' practices were observed before training and 3–6 months after training during 20 births and by interviewing 20 mothers before discharge at each hospital.
FINDINGS: Not all participants completed all of the tests. The scores on the tests of knowledge improved more among those in Group 2 than those in Group 1 when the answers were classified as right or wrong, but there was no difference between groups when a scoring method was used that classified answers as correct, partially correct, incorrect, or missing. Practices related to thermal control after birth improved among those in Group 2 after training but practices related to thermal control on the ward worsened. The promotion of breastfeeding improved in both groups.
CONCLUSION: There was no difference between the two training strategies, although self-directed learning was cheaper than conventional training. Neither strategy brought about the expected improvements in the quality of care. Other interventions in addition to training may be needed to improve care.

Keywords Perinatal care; Education, Medical, Continuing/methods; Education, Nursing, Continuing/methods; Medical staff, Hospital/education; Nursing staff, Hospital/education; Comparative study; Brazil (source: MeSH).

Mots clés Soins périnatals; Enseignement médical post-universitaire/méthodes; Formation continue infirmière/méthodes; Personnel médical hôpital/enseignement; Personnel infirmier hôpital/enseignement; Etude comparative; Brésil (source: INSERM).

Palabras clave Atención perinatal; Educación médica continua/métodos; Educación continua en enfermería/métodos; Cuerpo médico de hospitales/educación; Personal de enfermería en hospital/educación; Estudio comparativo; Brasil (fuente: BIREME).

More than 7.5 million perinatal deaths occurred in 1995, 98% of which were in developing countries (1). Perinatal mortality is an indicator not only of social and economic development, but also of access to, coverage of, and the use and quality of perinatal care (2, 3). Perinatal mortality is high in the state of Pernambuco, in north-east Brazil. The rate of stillbirths is unknown and so there is no reliable estimate of perinatal mortality. Neonatal mortality in 1997 was around 30/1000 live births, but this was probably an underestimate, and there were wide variations among districts (4). Early neonatal mortality is probably about half that figure, or 15/1000 live births. Perinatal mortality may be more than twice that of early neonatal mortality, as it has been estimated that in 1988 about 50% of perinatal deaths had not been registered (5); registration might be improving but it is far from complete. From census data, infant mortality in 1997 was estimated to be 52/1000 live births and believed to range from 12/1000 to 69/1000 in different districts (6).

In 1997, the government of Pernambuco developed interventions to reduce maternal and perinatal mortality, targeting them at maternity units. These interventions were an attempt to improve the quality of care: it was presumed that the majority of perinatal deaths could be prevented by providing good obstetric and neonatal care. The plan was to carry out an initial survey and then provide essential equipment and supplies to all maternity units and to train doctors and nurses in essential obstetric care and essential newborn care. The training focused on simple, effective interventions that would be relevant to the majority of deliveries in all maternity units. The programme also included an evaluation component, which was completed for a sample of the units. The objective of the evaluation was to determine which training strategy was more effective and whether it could be extended to include non-professional staff. Unfortunately, complete and reliable data were gathered only for the component on essential newborn care; data on essential obstetric care are not presented in this paper.

The instrument used for training health professionals in essential newborn care was a manual published by the WHO Regional Office for Europe (7), which was translated into Portuguese. The section on counselling women about breastfeeding was taken from a WHO manual (8) and integrated into the instrument.

Given the lack of evidence on the effectiveness of short in-service training courses and the need to identify the most feasible training strategy, it was decided to test two options in two groups of hospitals. The first training option was a conventional 5-day course taught by two trainers and two or three facilitators for around 20 participants in each hospital (Group 1). The second option was a self-directed learning course which was offered in the remaining hospitals (Group 2).

The self-directed course was offered to all doctors and nurses in a particular hospital by a supervisor and two or three facilitators. It took about 5 weeks to complete. Pernambuco's Department of Health provided the trainers and the supervisors; the facilitators were selected from the senior clinicians in each hospital. Each self-directed course started with a half-day meeting to explain the objectives and methods and to distribute the first part of the manual, corresponding to one day of the taught course offered to Group 1. Weekly meetings were then organized to allow participants to: discuss the study materials with supervisors and facilitators, resolve any queries, verify the level of learning by questions and answers, conduct practical sessions as described in the manual, and hand out the subsequent parts of the manual. The facilitators were accessible to the participants between weekly meetings to address any queries. The participants worked at their own pace during the week. At the end of each course, irrespective of the learning method, there was a half- day session in which trainers, supervisors, facilitators, and participants, in collaboration with local managers and decision-makers, used a problem-solving approach to draft an action plan for implementing their new skills.

Evaluation of knowledge levels

Knowledge was assessed using a 61-item test, which was part of the manual, consisting of multiple- choice, true or false, and short-answer questions. The test was administered immediately before the beginning of each course, immediately after it ended, and again 3–6 months later. The results were first analysed by classifying answers as right or wrong; incomplete or missing answers were marked as wrong. It was then decided to take a more detailed look at the data by scoring the answers as follows: 2 points for a correct and complete answer, 1 point for a correct but incomplete answer, 0 points for a missing answer, and a –1 point for an incorrect answer. The mean score for each participant in each test was obtained by summing all the scores and dividing by the number of questions. Multiple regression and two-way analysis of variance (using SPSS 7.0 for Windows) were used to compare the results between tests and groups.

Evaluation of practices

To evaluate changes in practice, two forms were used: one while observing a delivery and one during an interview with the mother at discharge. The forms were completed by the authors. Twenty deliveries were observed and 20 discharged mothers were interviewed in each hospital before training and 3–6 months after training. Again, the data were recorded and analysed using SPSS 7.0 for Windows. Each variable was first analysed individually. A value of 0 was assigned to a practice that was not observed or that was done incorrectly and a value of 1 was assigned to a practice done correctly. The results were analysed using multiple regression. Later, some variables were grouped to analyse overall practices in a single area, such as prevention of infection, thermal control at birth, promotion of breastfeeding, and thermal control on the ward.

Eight hospitals were selected that were thought to have a sufficient number of doctors and nurses to be trained and a sufficient number of deliveries that could be observed over 4–5 days. The results of the government's 1996 survey — carried out before the implementation of the perinatal plan, to assess the needs of the hospitals in terms of equipment and supplies — helped us to select these eight hospitals out of the 227 private and public health facilities where deliveries were carried out in Pernambuco, the majority having fewer than 100 deliveries per year. The assignment to Group 1 or 2 was not random in order to ensure that pairing by geographical, structural, and functional characteristics was accurate.

The study setting

Characteristics of the eight hospitals in 1996 are shown in Table 1. The total number of deliveries represented about 20% of all deliveries registered by the public health system in Pernambuco; the number of beds represented about 15% of all public maternity beds. The hospital's perinatal mortality and early neonatal mortality figures per 1000 live births were derived from routine reports and may be incomplete. The low figures in some hospitals may have occurred because most sick newborns are referred to tertiary care hospitals.

The number of doctors and nurses trained and the test results are shown in Table 2. The mean score ranges between –1 (all incorrect answers) and +2 (all correct answers). The number of staff trained is different from the number reported to be working at a specific hospital in 1996 because it is based on the actual numbers as opposed to annual reports and because training was offered to doctors and nurses who had applied for jobs at the hospitals but had not been yet been hired. Overall, training coverage was similar in both groups: almost 100% in all hospitals except for the two hospitals in Recife (Bandeira Filho and Barros Lima), where private practice is common and many doctors declined to participate.

Score rates

Not all doctors and nurses who were trained completed the series of tests. The test immediately after the course was not carried out in Petrolina and Bandeira Filho because of administrative problems. The percentage of participants who completed the tests dropped from baseline. The trainees in Group 2 did better than those in Group 1 on the test given at 3–6 months after training (70% vs 52% correct answers; P P P P P

The effect on practices observed

A total of 159 deliveries and 172 discharged mothers were studied at baseline and 150 deliveries and 155 discharged mothers were studied 3–6 months after training. The number of deliveries observed was

Table 3 and Table 4 show the percentage of practices observed that were performed correctly. Some practices were apparently not influenced by training. These included practices that seemed to be rarely performed (e.g. taking the newborn's temperature at birth and having a companion present at the birth), some that seemed to be performed moderately often (e.g. covering the weighing scale with a towel, checking the newborn's heart rate, ensuring that the newborn was breastfed within one hour), and some that were performed frequently (e.g. using sterile gloves, gauze, and instruments; adequately dressing the cut umbilical cord; offering vitamin K and ocular prophylaxis; and bathing the newborn).

There were, however, improvements in other practices. Some of these improvements occurred more often in the group that was conventionally trained (e.g. weighing the newborn and not using a feeding bottle. Some of these occurred more often in the group that had had self-directed learning (e.g. hand washing before delivery, covering the mother and baby, keeping the newborn warm on the way to the ward, allowing family visits, bathing the newborn). Some improvements occurred in both groups (e.g. thoroughly drying the newborn, discarding the towel used to dry the newborn and replacing it with a clean one, adequately assessing the newborn, providing skin-to-skin contact, taking the newborn's temperature, and washing hands on the ward). In some cases, there was an unexplained worsening of practices after training (e.g. bedding-in (mother and newborn together in the same room after delivery) decreased in Group 1 and the newborn's breathing was checked less often and there was less exclusive breastfeeding in Group 2 (Table 4)).

The results are probably easier to understand and interpret when grouped by type of practice. Fig. 1 shows how these practices changed before and after training. The doctors and nurses in Group 2 provided better thermal control at birth after training (P P P

The state Department of Health spent an average of US$ 8160 for each conventional training course and US$ 6260 for each self-directed one. This included paying trainers and supervisors and providing meals for all those involved in conventional training. The local cost of covering coffee breaks, stationery, and incidentals was not recorded but can be assumed to be minimal and similar for both courses. Thus, the estimated savings of using the self- directed approach might be of the order of 20–25%.

This study has several limitations: the hospitals were not randomly assigned to the groups (though great care was taken to pair hospitals with similar features); the data on knowledge before and after the courses are incomplete; data on practices come from a combination of observations and interviews with mothers (and these may yield different results); and the presence of observers at the delivery may have influenced the practices of the doctors and nurses. Yet this is the only study to compare two strategies for short in-service training. The conclusions may be relevant to a larger audience given the huge number of similar courses offered in many countries.

Knowledge seemed to improve with both strategies. This is in line with other reports, including a programme for rural perinatal training using 5 days of outreach education in the USA (9), a perinatal education programme using supervised self-directed learning in South Africa (10–12), and a continuing education programme in perinatal skills in Oklahoma City, USA (13). The apparent improvement after training, however, may be due to self-selection by the best participants. It is impossible to know whether self-selection was similar between the two groups; as a consequence, it is difficult to conclude that knowledge is retained better (at least in the analysis by right or wrong answer) with the self-directed strategy unless it is assumed that those who did not respond did not change the comparability of the groups. Even the studies from the USA and South Africa found that response rates dropped, and their results may also be difficult to interpret. More research is needed before concluding that short in-service courses lead to a permanent improvement in knowledge.

The important factor, however, is the effect on practice. We are not aware of other studies showing the effect of different training strategies on practice. The evaluation of the education programme in South Africa found that some practical skills improved (e.g. the ability to interpret antenatal record cards and partograms), but the comparison was with a group that had no training at all (14, 15). In our study, there were some practices that did not improve at all, some that improved with conventional training, and others that improved with self-directed learning. Overall, however, improvement in practices was not as great as had been desired.

It is difficult to explain these results when looking at the practices individually. It is easier to understand the results when practices are grouped together: the practices for which scores were lower at baseline (such as thermal control at birth and promoting breastfeeding) seemed to improve more than those practices for which scores were higher at baseline. Thus, perhaps standard training, using all the sections of a given manual, should be avoided and in-service training should be tailored to the local situation after an initial assessment. It is also clear that training alone is not enough. A combination of training and supervision combined with other methods of quality assurance — such as perinatal audit, which has already been shown to be an effective intervention (16, 17) — and the provision of essential equipment and supplies is more likely to lead to permanent changes.

Other factors to be taken into account when trying to improve the practices of health professionals are related to the organization of work. In Brazil, for instance, most doctors are contracted by hospitals to cover 24 hours on duty. They tend to accumulate four or five such contracts with different hospitals, thus returning to the same hospital at weekly intervals, so continuity of care cannot be ensured. Furthermore, doctors do not feel that they "belong" to a hospital and may therefore not be motivated to participate in training on quality improvement. Most of them also have private practices. Additionally, to be effective, training, supervision, quality assurance, and audit should be offered to the health workers actually carrying out the work. In the hospitals in this study, for example, training was offered to doctors and nurses, but the decision to admit a companion to the delivery room can be taken only by the director of the hospital, and many basic procedures are carried out by auxiliary staff. These constraints are even more pronounced in smaller and more peripheral hospitals and maternity units.

In this study both types of training strategies — conventional and self-directed learning — seemed insufficient to bring about significant improvements in practice. The self-directed strategy does not seem to be less effective than the conventional one. It is certainly more feasible, at least in hospitals of the size of those selected for this study; smaller hospitals with fewer health workers to be trained and that see an insufficient number of cases to allow for hands-on practice may present different problems. Self-directed courses require less organization and management than conventional courses and may cost less.

The authors wish to thank Dr Gilliatt Falbo Neto, State Secretary for Health at the time of the study, for professional and financial support. The study was also supported by a grant from the Maternal Health and Safe Motherhood Programme of WHO, Geneva, Switzerland.

Conflicts of interest: none declared.

Résumé

Comparaison de deux stratégies de formation en matière de soins essentiels aux nouveau-nés au Brésil

OBJECTIF: Comparer l'efficacité de deux stratégies de formation pour améliorer les soins essentiels aux nouveau-nés dans l'Etat de Pernambuco (Brésil).
MÉTHODES: Huit hôpitaux ont été choisis, répartis en deux groupes de quatre et appariés d'après leur localisation géographique, leur structure et leur fonctionnement. Les médecins et le personnel infirmier des hôpitaux du groupe 1 ont reçu un cours de formation classique de cinq jours. Ceux du groupe 2 ont reçu le même manuel que ceux du groupe 1 mais le cours consistait en un auto-apprentissage, les participants ayant cinq semaines pour terminer le programme. Un contrôle des connaissances a été effectué peu avant le début du cours, immédiatement après et 3 à 6 mois après. Dans chaque hôpital, les pratiques des participants ont été observées avant la formation et 3 à 6 mois après sur 20 naissances et en interrogeant les mères avant leur sortie.
RÉSULTATS: Les participants n'ont pas tous subi la totalité des tests. Les résultats du contrôle des connaissances se sont davantage améliorés parmi les participants du groupe 2 que parmi ceux du groupe 1 lorsque les réponses étaient classées en bonnes ou mauvaises, mais il n'y avait pas de différence entre les groupes lorsqu'une autre méthode de notation était utilisée, avec classement des réponses en correctes, partiellement correctes, incorrectes ou manquantes. Les pratiques concernant le contrôle de la température du nouveau-né à la naissance se sont améliorées chez les participants du groupe 2 après la formation mais les pratiques relatives au contrôle ultérieur de la température se sont dégradées. La promotion de l'allaitement maternel s'est améliorée dans les deux groupes.
CONCLUSION: Il n'y avait pas de différence entre les deux stratégies, même si l'auto-apprentissage était moins coûteux que la formation classique. Aucune des stratégies n'a apporté les améliorations attendues en ce qui concerne la qualité des soins. Pour améliorer celle- ci, d'autres interventions pourraient être nécessaires en plus de la formation.

Resumen

Comparación de dos estrategias de capacitación para la atención esencial a recién nacidos en el Brasil

OBJETIVO: Comparar la eficacia de dos estrategias de capacitación destinadas a mejorar la atención esencial a recién nacidos en el Estado de Pernambuco (Brasil).
MÉTODOS: Se seleccionaron ocho hospitales, que se repartieron en dos grupos de cuatro, emparejándolos de acuerdo con criterios geográficos, estructurales y funcionales. A los médicos y enfermeras que trabajaban en los hospitales del grupo 1 se les impartió un cursillo convencional de 5 días. Los del grupo 2 recibieron el mismo manual empleado por el grupo 1, pero el cursillo de formación se organizó como un proceso de aprendizaje autodirigido, en el que los participantes disponían de 5 semanas para terminar el cursillo. Se determinaron los conocimientos de los participantes en condiciones basales, inmediatamente después del cursillo, y al cabo de 3-6 meses. Además se evaluó el desempeño práctico de los participantes antes de la capacitación y a los 3-6 meses de la misma, observando como se desenvolvían en 20 partos y entrevistando a 20 madres antes del alta en cada hospital.
RESULTADOS: Algunos participantes no hicieron todas las pruebas. La mejora de la puntuación obtenida en las pruebas teóricas fue mayor en el grupo 2 que en el grupo 1 cuando se procedió a clasificar las respuestas como correctas o erróneas, pero no se observó ninguna diferencia entre los grupos cuando se utilizó otro método de puntuación consistente en clasificar las respuestas como correctas, parcialmente correctas, incorrectas o inexistentes. La práctica de control de la temperatura tras el nacimiento mejoró en el grupo 2 después de la capacitación, pero el control de esa variable en la sala empeoró. En los dos grupos se observó una mejora de la promoción de la lactancia materna.
CONCLUSIÓN: No se observó diferencia alguna entre las dos estrategias de capacitación, pero el aprendizaje autodirigido es más barato que la capacitación convencional. Ninguna de las estrategias propició las mejoras esperadas en la calidad de la atención. Posiblemente se necesiten otras intervenciones además de la capacitación para mejorar la asistencia.

1. New estimates of perinatal mortality. Weekly Epidemiological Record, 1996, 71: 297–303.        

2. Dunn PM. Major ethical problems confronting perinatal care around the world. International Journal of Gynaecology and Obstetrics, 1995, 51: 205–210.        

3. Richardus JH et al. The perinatal mortality rate as an indicator of quality of care in international comparisons. Medical Care, 1998, 36: 54–66.        

4. Lessa F. Mortalidade perinatal na Cidade de Recife. Monografia para o Curso de Especialização em Epidemiologia [Perinatal mortality in the city of Recife] [Dissertation]. Recife, Pernambuco, Nucleo de Estudos em Saúde Coletiva de Pernambuco, 1993.         

5. Barros FC, Victora CG. Epidemiologia da saúde infantil: um manual para diagnósticos comunitários. [The epidemiology of child health: a manual for community diagnosis.] São Paulo, HUCITEC/UNICEF, 1991.         

6. Simões C. Estimativas de mortalidade infantil para as microrregiões de Pernambuco. [Estimates of infant mortality in small regions of Pernambuco.] Recife, Pernambuco, Secretaria de Saúde do Estado de Pernambuco, Diretoria de Desenvolvimento Social, 1997.         

7. Essential newborn care and breastfeeding. Copenhagen, WHO Regional Office for Europe, 1997.        

8. Breastfeeding counselling: a training course. Geneva, World Health Organization, 1993.        

9. Clarke SB. Enhancing regional perinatal care: a clinical traineeship for perinatal nurses in a predominantly rural area. Neonatal Network, 1993, 12: 39–47.        

10. Woods DL, Theron GB. The impact of the Perinatal Education Programme on cognitive knowledge in midwives. South African Medical Journal, 1995, 85: 150–153.        

11. Le Roux E et al. Does successful completion of the Perinatal Education Programme result in improved obstetric practice? South African Medical Journal, 1998, 88 (Suppl. 2):180–182.        

12. Theron GB. Improved cognitive knowledge of midwives practising in the eastern Cape Province of the Republic of South Africa through the study of a self-education manual. Midwifery, 1999, 15: 66–71.        

13. Harris JK, Yates B, Crosby WM. A perinatal continuing education program: its effects on the knowledge and practices of health professionals. Journal of Obstetric, Gynecologic, and Neonatal Nursing, 1995, 24: 829–835.        

14. Theron GB. Effect of the maternal care manual of the perinatal education programme on the ability of midwives to interpret antenatal cards and partograms. Journal of Perinatology, 1999, 19: 432–435.        

15. Theron GB. Improved practical skills of midwives practicing in the Eastern Cape Province of the Republic of South Africa through the study of a self-education manual. Journal of Perinatology, 2000, 20: 184–188.        

16. Wilkinson D. Reducing perinatal mortality in developing countries. Health Policy and Planning, 1997, 12: 161–165.        

17. Mancey-Jones M, Brugha RF. Using perinatal audit to promote change: a review. Health Policy and Planning, 1997, 12: 183–192.        

¹ Public Health Officer, Diretoria de Desenvolvimento Social, Secretaria Estadual de Saúde, Recife, Brazil.

² Paediatrician, Unit for Health Services Research and International Cooperation, Istituto per l'Infanzia, Trieste, Italy.

³ Neonatologist, Instituto Materno Infantil de Pernambuco, Recife, Brazil.

⁴ Statistician, Diretoria de Desenvolvimento Social, Secretaria Estadual de Saúde, Recife, Brazil.

⁵ Medical Statistician, Epidemiology Unit, Local Health Authority, Bergamo, Italy.

⁶ Epidemiologist, Unit for Health Services Research and International Cooperation, Istituto per l'Infanzia, Via dell'Istria 65/1, 34137 Trieste, Italy (email: cattaneo@burlo.trieste.it). Correspondence should be addressed to this author.

Ref. No. 00-0856