R.E. Cibulskis1 & G. Hiawalyer2
ABSTRACT: This paper describes (i) how a national health information System was designed, tested and implemented in Papua New Guinea, (ii) how the system was integrated with other management information systems, and (iii) how information has been used to support decision-making. It concludes that central coordination of systems design is essential to make sure that information systems are aligned with government priorities and can deliver the information required by managers. While there is often scope for improving the performance of existing information systems, too much emphasis can be placed on revising data collection procedures and creating the perfect information system. Data analysis, even from imperfect systems, can stimulate greater interest in information, which can improve the quality and completeness of reporting and encourage a more methodical approach to planning and monitoring services. Our experience suggests that senior decision-makers and political leaders can play an important role in creating a culture of information use. By demanding health information, using it to formulate policy, and disseminating it through the channels open to them, they can exert greater influence in negotiations with donors and other government departments, encourage a more rational approach to decision-making that will improve the operation of health services, and stimulate greater use of information at lower levels of the health system. The ability of information systems to deliver these benefits is critical to their sustainability.
Keywords Information systems/organization and administration; Management information systems/utilization; National health programs; Information management/organization and administration; Papua New Guinea (source: MeSH, NLM).
Mots clés Système information/organisation et administration; Système information gestion/utilisation; Programme national santé; Gestion information/organisation et administration; Papouasie-Nouvelle-Guinée (source: MeSH, INSERM).
Palabras clave Sistemas de información/organización y administración; Sistemas de información administrativa/utilización; Programas nacionales de salud; Gerencia de la información/organización y administración; Papua Nueva Guinea (fuente: DeCS, BIREME).
Introduction
Information on health needs, the delivery of services, and the availability and use of resources is important to all health service organizations. Such information can help an organization to increase its efficiency, effectiveness and responsiveness in several ways. First, it can help managers to align health system resources with client needs (a planning or prospective role) and determine whether their plans are progressing satisfactorily or whether there is a need for corrective action (a monitoring or retrospective role) (1). Second, information can be used to increase accountability within an organization and allow the public, their elected representatives, or donors to determine whether they are obtaining value for money (2, 3). Third, information can be used to market health programmes, secure appropriate levels of funding or engage public support (4). Fourth, information amassed over time can help an organization to learn what works and does not work and thereby provide valuable know-how, which can lead to greater efficiency in the production process (3).
Experience suggests that health service organizations have difficulty acquiring information for use in any of these roles (5, 6). Successful information systems have been developed on a small scale, such as for an individual district or programme, but the sustainability of these initiatives is often questionable and their relation to national information systems development is not clear (4). This paper considers how a national health information system (NHIS) was established, how it was integrated with other information systems, and how the information collected has been used to plan and monitor services. It then considers some of the factors necessary for the development of successful and sustainable information systems.
Background
Information systems received much attention in Papua New Guinea after health service administration was decentralized to the provincial level in 1983. A management strengthening project sought to improve the use of data from existing systems by defining a core set of indicators and introducing computers for data processing at provincial level (710). While broadly successful in what it sought to achieve, the project did not attempt to revise data collection systems or consider how systems should be managed in a decentralized environment. These issues became increasingly important since, after decentralization, national level managers ceased to have control over information systems; they could advise provinces on what data should be collected, but lacked the authority, capability or resources to ensure that their recommendations were heeded. Provinces could develop their own systems if they wished and differences gradually emerged between them, induced partly by donor projects that introduced programme- specific information systems in selected areas. Any uniformity in systems appeared to be due to several provinces ordering their stationery from the same printing company, which provided a limited range of forms in its catalogue. The haphazard state of the information systems made it increasingly difficult to aggregate data on a national scale, compare results between provinces, or maintain systems. This prompted the Department of Health to embark on a major programme to standardize its health management information systems.
Introducing a national health information system
System design
A task force to review information systems was established in 1994, consisting of programme managers from the Papua New Guinea Department of Health and an international consultant. Nine out of the 20 provinces were visited and discussions held with health workers, managers, training schools, other government departments, nongovernmental organizations, and donors. One district was visited on a monthly basis for a year to understand information needs at health facility level. The review identified several problems in data collection, reporting and analysis, but indicated that there would be little benefit in collecting radically different data, changing reporting procedures or establishing different methods of analysis, as this would require major investments in training and support that could not be delivered using available resources. The major change proposed was to limit monthly reporting to a single form, which consisted of a folded sheet of A3 paper, and which combined the essential features of the seven or more forms in use. This would reduce the reporting burden on health facility staff and make it easier for provincial and national level staff to compile and analyse data. The form would contain data elements for calculating key indicators already in use and a few additional indicators for monitoring the latest national health plan. Instruments were also designed to support data capture, such as tally sheets, daily summary books, and monthly analysis books. These represented the minimum amount of stationery required for data gathering. Health facilities would still need registers, patient record cards, and other materials but this stationery could not be afforded at the time the work was undertaken.
Testing
The proposed system was tested in only one province of the country owing to financial and time constraints. Stationery and training materials were prepared and computer software for processing data at provincial level was rewritten. The software was deliberately similar to the applications already in use but was upgraded to accommodate new data items and new possibilities for analysis, and made year 2000 compliant. A 3- day training workshop was held for staff from every health facility.
The system was allowed to run for 6 months. Testing suggested that the number of disease categories should be increased to prevent health workers misclassifying certain conditions. Otherwise the new system was found to be more useful and easier to operate than previous systems.
Implementation
The need to train staff from all health facilities prevented the simultaneous implementation of the revised information system in all health facilities. Instead the system was introduced province by province over 12 months. This enabled visits to be made to the first provinces to check that the system was functioning as intended, that supplies of stationery were adequate, and that training was appropriate. Thereafter implementation was intensified and follow-up visits reduced.
Most provinces started using the new system within 2 months of training, although workshops were repeated in three provinces where implementation was incomplete. Provinces were provided with a 2-year supply of stationery, but it soon became obvious that larger quantities were required if occasional shortages were to be avoided.
Integration of systems
Department of Health systems
Health information systems are part of a wider management information system that provides data on population, resource availability and use (Table 1). While the NHIS was being developed, other components of the management information system were strengthened. A national inventory of health facilities was undertaken to obtain data on the number of facilities, staff, buildings and equipment. Such data are useful in their own right but are more valuable if they can be linked to health information. A major impetus to link information systems came from a reform of provincial and local-level government in 1995, in which district boundaries were revised to coincide with political electorates. This required health facility codes to be updated on all computer systems (the middle two digits of a health facility code represent a district) and allowed databases to be merged. Inputs could then be related to outputs and the efficiency of resource use explored. For example, performance in family health programmes was found to be related to staff:population ratios.
Other sectors
The health management information system was also linked to other sectors using the national census as a common base. Each health facility was traced to its nearest census unit. Because the approximate location of census units was known, maps could be drawn showing the distribution of health facilities, health conditions or any other feature that uses the common coding systems of the Department of Health. The maps are a convenient way of summarizing data and are useful for influencing decision-makers who are not overly interested in numbers (Fig. 1).
Because of its intrinsic link to population, the system can also be used to determine how many people live within a certain distance of a health facility with specific attributes, e.g. the nearest facility with a doctor or radio. Such information is important because geographical distance influences use of services and is critical in deciding where to open, close, or upgrade facilities and locate staff (1113). Other government departments are using a similar scheme to identify the location of schools and other infrastructure. When such data are linked with health information they provide a tremendous resource for intersectoral planning and the targeting of poverty- reduction programmes (14).
Using information
Attempts were made to increase the analysis of information at all levels of the health system. Health workers were provided with tools and training in data analysis including a "Health centre record" a booklet designed to assess trends over time and provide a permanent record of a health facility's activities. At provincial level, strategies for analysing indicators were reinforced by revising computer software and training provincial managers.
At national level, analysis and dissemination were initially limited to an annual booklet on family health indicators. By limiting the analysis to a small area of activity in which data were readily available, and considered important, the publication could be prepared in advance of an annual meeting of provincial health managers where the performance of individual provinces was publicly disclosed. Such feedback helped to increase reporting rates from 73% in 1994 to 85% in 1995 and to 93% in 2000.
Improvements in information systems eventually allowed the government to undertake more extensive analysis and use information for assessing priorities in the national health plan. Targets were set on the basis of previous performance statistics and they helped managers to assess which health programmes were proceeding as planned and how each province was performing.
Information was also used to market the plan to other government sectors and parliamentary leaders. Each Member of Parliament was presented with a graphical summary of health indicators, which showed the status of their constituencies in comparison to others (Fig. 2). The aim was to inform national leaders about the health situation and bring health issues to the forefront of the political agenda. More specifically it was hoped it would secure health budgets and persuade Members of Parliament to pay attention to health development in their own constituencies.
Discussion
System design
The definition of management functions at each level of the health system and the identification of information needs and indicators are crucial steps in revising information systems (15, 16). It is customary to seek the involvement of health service managers in this task to ensure that systems meet their needs and to facilitate eventual implementation (16, 17). In practice, however, managers are not always sure of the information they require and the process can result in long lists of indicators, many of which cannot be measured (17, 18).
In Papua New Guinea the definition of information needs was assisted by four factors. First, managers were familiar with concepts of indicators and monitoring as a result of previous management strengthening efforts (710). Second, existing information was analysed and disseminated before system revision commenced decisions on what indicators should be generated by an information system are easier if managers have data in front of them and can see how various indicators compare with targets, between geographical locations and over time. Third, revision of systems coincided with the development of a national health plan that provided guidance on what should be measured. Fourth, the decision to use a single form for monthly data collection which was taken following extensive consultation with health workers placed limitations on the amount of data that could be collected. These factors resulted in an information system that was not very different from previous systems, but was more clearly defined, making it easier for health workers to adapt to it, reducing the need for major retraining and ensuring consistency in time-series.
Implementation
Good information systems may fail to take hold if they are poorly implemented. In Papua New Guinea several strategies were used to reduce the risk of failure (1820). Widespread consultation ensured that staff were aware of the changes and had contributed to them, and that the system's design was realistic. Testing on a limited scale helped to confirm the system's appropriateness. Improved analysis and dissemination of information before implementation created a more favourable climate in which to introduce new systems. It stimulated interest in information, generated support from senior levels in the Department of Health, and gave credibility to programme managers leading the change.
Attention was given to how the system would be introduced to each province, with the organization of workshops, printing and distribution of stationery, revision of computer software, and discontinuation of existing systems. Private printing companies and training schools were kept aware of the changes. At national level, procedures were established for follow-up of missing reports, data quality control, updating coding systems, data summary and provision of feedback. Provisions were included in the National Health Administration Act, 1997, to compel all health facilities to report using the NHIS. Certificates were awarded to health facilities and provinces that provided the best reports. Financial support was secured for stationery, training, freight/postage, communications, periodic upgrading of computers, and software.
Management of systems
Our experience suggests that tasks required for the smooth functioning of information systems are best managed and financed by a single unit at national level rather than being devolved to provinces or split among separate programmes at national level. Such an arrangement provides more stable funding, greater consistency in approach, and avoids duplication. There is no standard model, however, and in other situations a different strategy may be preferred, particularly if the monitoring capacity of individual programmes is well developed and programme staff are motivated to seek and use the information. Nevertheless it will still be necessary to ensure that opportunities exist to undertake consolidated analysis of information if management of systems is distributed over several units.
Coordination of systems development
Mechanisms are needed to coordinate system developments across programmes (21). In Papua New Guinea, a committee was established to review all information system proposals and develop a coordinated strategy for improving systems. The committee included representation from several programme managers and donors but there were still some agencies that preferred to bypass government procedures and install parallel systems, sometimes employing consultants who use non- standard software and fail to work with local systems developers (19). This approach is partly explained by the perception that governments are not capable of managing information systems and it is easier for a donor to bypass dysfunctional structures than to strengthen them. However, such short cuts do not help to develop the management capacity of health systems. Moreover, experience suggests that independently developed information systems rarely outperform government systems and rapidly collapse once donor support is withdrawn. They put undue pressure on government staff, disrupt normal operations, and are ultimately detrimental to information system performance and sustained development.
What should donors do?
In most cases it is preferable to channel donor support into existing monitoring units, with funds for stationery, equipment and technical assistance where necessary. If several donors are operating in a country, it may be advantageous to employ common indicators of performance and common data collection methods. Clearly, there are difficulties in developing a coordinated monitoring strategy where donors do not share common objectives. However, they can be assisted if government objectives are clear. In Papua New Guinea, the national health plan defines the goals and objectives of the health sector and how progress will be monitored. The plan has helped to ensure donor support for national programmes and how they are monitored.
Human resources
A critical factor governing the sustainability of information systems is the availability of qualified and experienced personnel. Many systems are developed with external assistance because necessary skills are not available locally. It is unrealistic to expect these skills to be developed rapidly. Certainly, it would help to strengthen the capacity of local institutions to offer training in statistics, epidemiology, and computing to diploma or degree level. But it is difficult to recruit or retain staff without significantly improving pay or working conditions. Many countries are therefore faced with the stark choice of neglecting information systems or relying on external assistance whose goals may be at variance with those of the government.
Creating a culture of information use
It is hoped that lower levels of management, including health care providers, will use information for planning and monitoring health services, and it is possibly at this level that information use can have the greatest impact on the efficiency and effectiveness of health services (22). This can be encouraged through training and the provision of tools for data analysis. While necessary, such initiatives are seldom sufficient to change the way staff use information. Our experience suggests that an important way of developing the periphery in this respect is by setting a good example at the top. If senior management seeks information and uses it openly then the importance of information is reinforced throughout the health system. A promising initiative has been to summarize health statistics according to parliamentary constituencies and ensure their widespread circulation. This is not only of interest to health managers but also to a wider audience who wish to see accountability among public servants and politicians (23). It has given a readily observable purpose for the health management information system and helps to legitimize requests for data from health workers and mid-level management.
Information has also been used to defend the position of the health sector in negotiations with other government departments and donors. Indeed, the ability to set priorities and monitor progress was a key factor in attracting a health sector programme loan in which national health plan targets are used as benchmarks for assessing progress, and ultimately to the adoption of a sector-wide approach for the coordination of donor inputs. These benefits will ultimately be critical to ensuring the sustainability of information systems. According to UNICEF (24), sustainability is the ability of a system to produce benefits valued sufficiently by users and stakeholders to ensure enough resources to continue activities. Hence, information systems can be sustainable if they deliver benefits to senior decision-makers and donors who control resources, and if there is sufficient motivation for health workers to support them. 
Acknowledgements
The work described in this paper was undertaken while Dr Cibulskis was Adviser in Health Information Systems for the Asian Development Bank Third Rural Health Services Project and Dr Hiawalyer was Deputy Director of Monitoring and Research in the Department of Health, Papua New Guinea. We recognize the efforts of staff from the Monitoring and Research Section and provincial health information officers who have been responsible for building the country's information systems. Special credit should be given to Ahouta Badu, Ila Rouka (discharge information); Gahusi Gahusi, Kheila Tawai (national health information system); Mikes Arere, Tware Teka (health facilities); Steve Mellor, Jojie Urbiztondo, John Mondo, George Tuges (computer systems); and Kwalu Ora (publications). We also acknowledge the efforts of all health workers who routinely submit their monthly activity reports through the NHIS. Dr Tonya Villafana and three anonymous referees reviewed and made helpful comments on the manuscript.
A preliminary draft of this paper was presented at the Twelfth Meeting of Commonwealth Health Ministers, Barbados, 1998.
Conflicts of interest: none declared.
Résumé
Systèmes d'information pour la surveillance du secteur de la santé en Papouasie-Nouvelle-Guinée
Le présent article décrit comment un système d'information sanitaire national a été conçu, testé et mis en uvre en Papouasie-Nouvelle-Guinée, comment ce système a été intégré avec les autres systèmes de gestion de l'information et comment l'information a été utilisée dans la prise de décision. S'il est vrai que la performance des systèmes existants d'information peut souvent être améliorée, il arrive qu'une trop grande place soit accordée à la modification des méthodes de collecte des données et à la conception d'un système d'information parfait. L'analyse des données, même quand elles proviennent d'un système imparfait, permet de développer l'intérêt pour l'information, ce qui peut améliorer la qualité et la complétude des signalements et favoriser une approche plus méthodique de la planification et de la surveillance des services. Les stratégies possibles sont nombreuses pour améliorer l'utilisation de l'information dans le secteur de la santé mais l'adoption de nombreuses initiatives est variable. D'après notre expérience, les responsables des décisions et les responsables politiques peuvent jouer un rôle important en suscitant une culture de l'utilisation de l'information, en sollicitant l'information sanitaire, en l'utilisant pour formuler les politiques et en la disséminant par les moyens à leur disposition. On peut s'attendre à en retirer plusieurs avantages, et notamment la capacité d'exercer une plus grande influence dans les négociations avec d'autres services gouvernementaux et les donateurs, une plus grande rationalité des décisions qui améliorera le fonctionnement des services de santé et une incitation à utiliser l'information aux niveaux inférieurs du système sanitaire. La faculté des systèmes d'information à générer des avantages pour les divers intéressés est en fin de compte capitale dans leur maintien.
Resumen
Sistemas de información para la vigilancia del sector de la salud en Papua Nueva Guinea
En este artículo se describe cómo se diseñó, ensayó e implantó en Papua Nueva Guinea un sistema nacional de información sanitaria, cómo se integró dicho sistema con otros sistemas de información para la gestión, y cómo se ha utilizado la información para apoyar la adopción de decisiones. Si bien pueden identificarse a menudo diversas posibilidades para mejorar el funcionamiento de los sistemas de información existentes, se corre el riesgo de hacer demasiado hincapié en la revisión de los procedimientos de acopio de datos y en el perfeccionamiento incesante de esos sistemas. El análisis de los datos, aun de sistemas imperfectos, puede suscitar más interés por la información, lo que puede traducirse en una mejora de la calidad e integridad de los datos notificados y propiciar un enfoque más metódico de la planificación y vigilancia de los servicios. Pueden utilizarse diversas estrategias para mejorar el uso de la información en una organización de salud, la adopción de decisiones. Si bien pueden identificarse a menudo diversas posibilidades para mejorar el funcionamiento de los sistemas de información existentes, se corre el riesgo de hacer demasiado hincapié en la revisión de los procedimientos de acopio de datos y en el perfeccionamiento incesante de esos sistemas. El análisis de los datos, aun de sistemas imperfectos, puede suscitar más interés por la información, lo que puede traducirse en una mejora de la calidad e integridad de los datos notificados y propiciar un enfoque más metódico de la planificación y vigilancia de los servicios. Pueden utilizarse diversas estrategias para mejorar el uso de la información en una organización de salud, pero el interés de las iniciativas es desigual. Nuestra experiencia parece indicar que las altas instancias decisorias y los dirigentes políticos pueden contribuir considerablemente a promover una cultura del uso de la información solicitando información sanitaria, utilizándola para formular políticas y difundiéndola por los circuitos a su alcance. Cabe prever como resultado diversos efectos positivos, como la capacidad de ejercer mayor influencia en las negociaciones con otros ministerios y donantes, la adopción de decisiones más racionales que mejorarán el funcionamiento de los servicios de salud, y el fomento del uso de la información en los niveles inferiores del sistema sanitario. La capacidad de los sistemas de información para beneficiar a los interesados directos es en último término decisiva para su sostenibilidad.
References
1. Smith P. Information systems and the white paper proposals. In: Culyer AJ, Maynard A, Posnett J, eds. Competition in health care: reforming the NHS. Basingstoke: Macmillan; 1990.
2. Smith P. The use of performance indicators in the public sector. Journal of the Royal Statistical Society Series A 1990;153:53-72.
3. Mackay K. Public sector performance the critical role of evaluation. In: Mackay K, editor Public sector performance the critical role of evaluation: selected proceedings of a World Bank Seminar. Washington (DC): World Bank;1998. p.ix-xvi.
4. Cibulskis RE, Izard J. Monitoring systems. In: Janovsky K, editor. Health policy and systems development: an agenda for research. Geneva: World Health Organization; 1996. p.191-206.
5. Keller A. Management information systems in maternal and child health/family planning programs: a multi-country analysis. Studies in Family Planning 1991;22:19-30.
6. Finau SA. National health information systems in the Pacific Islands: in search of a future. Health Policy and Planning 1994;9:161-70.
7. Newbrander WC, Thomason JA, Kolehmainen-Aitken RL, Campos-Outcalt D, Afo G, Davidson W. Management support for provinces: a programme for developing provincial health management capabilities in Papua New Guinea. International Journal of Health Planning and Management 1988;3:45-55.
8. Newbrander WC, Thomason JA. Computerising a national health system in Papua New Guinea. Health Policy and Planning 1988;3:255-9.
9. Campos-Outcalt D, Vickers P. For fuller utilization of health data. World Health Forum 1988;9:405-8.
10. Campos-Outcalt D. Microcomputers and health information in Papua New Guinea: a two year follow-up evaluation. Health Policy and Planning 1991;6:348-53.
11. Muller I, Smith T, Mellor S, Rare L, Genton B. The effect of distance from home on attendance at a small rural health centre in Papua New Guinea. International Journal of Epidemiology 1998;27:878-84.
12. Rose M, Abderrahim N, Stanton C, Helsen D. Maternity care: a comparative report on the availability and use of maternity health services. Chapel Hill (NC), Measure Evaluation, 2001. MEASURE Evaluation Technical Report Series, No. 9.
13. Oppong JR, Hodgson MJ. Spatial accessibility to health care facilities in Suhum District, Ghana. Professional Geographer 1994;64:199-209.
14. Bigman D, Dercon S, Guillaume D, Lambotte M. Community targeting for poverty reduction in Burkina Faso. The World Bank Economic Review 2000;14:167-93.
15. Lippeveld T, Sauerborn R. A framework for designing health information systems. In: Lippeveld T, Sauerborn R, Bodart C, editors. Design and implementation of health information systems. Geneva: World Health Organization; 2000. p. 15-32.
16. Bodart C, Shrestha L. Identifying information needs and indicators. In: Lippeveld T, Sauerborn R, Bodart C, editors. Design and implementation of health information systems. Geneva: World Health Organization; 2000. p. 49-72.
17. Lippeveld T. Approaches to strengthening health information systems. In: Lippeveld T, Sauerborn R, Bodart C, editors. Design and implementation of health information systems. Geneva: World Health Organization; 2000. p. 243-52.
18. Bunge C. Health management information system for provincial health administrations Cameroon. In: The practice of health information systems. London: Commonwealth Secretariat; 2001. p. 19-31.
19. Jayasuriya R. Managing information systems for health services in a developing country: a case study using a contextualist framework. International Journal of Information Management 1999;19:335-49.
20. Heeks R, Mundy M, Salazar A. Why health care information systems succeed or fail. Manchester: Institute for Development Policy and Management; 1999. Working Paper Series, No. 9.
21. Van Konkelenberg R, Ring I. Guidelines for the development of health management information systems. Manila: WHO Regional Office for the Western Pacific; 1993.
22. Sandiford P, Annett H, Cibulskis RE. What can information systems do for primary health care? An international perspective. Social Science and Medicine 1992;34:1077-87.
23. Marcel M. Lessons from Chile. In: Mackay K, editor. Public sector performance the critical role of evaluation: selected proceedings of a World Bank Seminar. Washington (DC): World Bank; 1998. p. 67-71.
24. Health policies and strategies. Sustainability, integration and national capacity-building. New York (NY): UNICEF: 1992. Unpublished document E/ICEF/1992/L.2.
1 Takemi Program, Department of Population and International Health, Harvard School of Public Health, 677 Huntington Avenue, Boston, MA 02115, USA (email: cibulskis@aol.com). Correspondence should be addressed to this author.
2 Department of Health, Waigani, Papua New Guinea.
Ref. No. 01-1177