POLICY AND PRACTICE
Health information technology in primary health care in developing countries: a literature review
Technologies de l'information et soins de santé primaires dans les pays en développement : une revue de la littérature
Tecnologías de la información sanitaria en la atención primaria en los países en desarrollo: una revisión de la literatura
Elaine Tomasi1 ; Luiz Augusto Facchini; Maria de Fatima Santos Maia
Federal University of Pelotas, Avenue Duque de Caxias 250, Pelotas, 96030-002, Brazil
ABSTRACT
This paper explores the debate and initiatives concerning the use of information technology (IT) in primary health care in developing countries. The literature from 1992–2002 was identified from searches of the MEDLINE, Latin American and Caribbean Health Science Literature Database (LILACS), Cochrane Library and Web of Science databases. The search identified 884 references, 350 of which were classified according to the scheme described by the Pan American Health Organization (PAHO). For the analysis of advantages, problems and perspectives of IT applications and systems, 52 articles were selected according to their potential contribution to the primary health-care processes in non-developed countries. These included: 10 on electronic patient registries (EPR), 22 on process and programmatic action evaluation and management systems (PPAEM) and 20 on clinical decision-support systems (CDS). The main advantages, limitations and perspectives are discussed.
Keywords: Primary health care; Information technology; Information systems/utilization; Decision support systems, Clinical/utilization; Medical records systems, Computerized/utilization; Education, Distance; Telemedicine/utilization; Review literature; Developing countries (source: MeSH, NLM).
RÉSUMÉ
Le présent article examine les débats et les initiatives concernant l'utilisation des technologies de l'information dans le cadre des soins de santé primaires dans les pays en développement. Les articles publiés en 1992-2002 ont été identifiés au moyen d'une recherche dans plusieurs bases de données - MEDLINE, LILACS (Latin American and Caribbean Health Science Literature Database), Cochrane Library et Web of Science. La recherche a fourni 884 références, dont 350 ont été classées selon le système décrit par l'Organisation panaméricaine de la Santé. Pour l'analyse des avantages, des limites et des perspectives des applications et systèmes informatiques, 52 articles ont été sélectionnés pour leur contribution potentielle aux processus de soins de santé primaires dans les pays non développés : 10 concernaient les dossiers médicaux informatisés, 22 les systèmes d'évaluation et de gestion des processus et de l'action programmatique et 20 les systèmes d'aide à la décision clinique. L'article présente une discussion des principaux avantages, limites et perspectives de ces applications et systèmes.
Mots clés: Soins santé primaire; Technologie de l'information; Système information/utilisation; Système informatique aide décision clinique/utilisation; Dossier médical informatisé/utilisation; Enseignement à distance; Télémédecine/utilisation; Revue de la littérature; Pays en développement (source: MeSH, INSERM).
RESUMEN
En este artículo se analizan el debate y las iniciativas relacionadas con el uso de las tecnologías de la información (TI) en la atención primaria en los países en desarrollo. Se buscaron publicaciones del periodo 1992–2002 a través de MEDLINE, de la Base de Datos de la Literatura Latinoamericana y del Caribe en Ciencias de la Salud (LILACS), de la Biblioteca Cochrane y de la base de datos Web of Science. Se localizaron 884 referencias, 350 de las cuales se clasificaron conforme al sistema descrito por la Organización Panamericana de la Salud (OPS). Al objeto de analizar las ventajas, los problemas y las perspectivas de las aplicaciones y los sistemas de IT, se seleccionaron 52 artículos de acuerdo con su contribución potencial a los procesos de atención primaria en los países no desarrollados. Esa cifra se desglosa así: 10 artículos sobre los registros electrónicos de pacientes, 22 sobre los sistemas de evaluación y gestión de procesos y acciones programáticas, y 20 sobre los sistemas de apoyo a las decisiones clínicas. Se examinan las principales ventajas, limitaciones y perspectivas.
Palabras clave: Atención primaria de salud; Tecnología de la información; Sistemas de información/utilización; Sistemas de apoyo a decisiones clínicas/utilización; Sistemas de registros médicos computarizados/utilización; Educación a distancia; Telemedicina/utilización; Literatura de revisión; Países en desarrollo (fuente: DeCS, BIREME).
Introduction
In health care, especially in developed countries, the use of different types of information technology (IT) has progressed considerably since the beginning of electronic patient registration, leading to improvements in the interfacing and fusion capabilities of a large variety of computer and telecommunication technologies. Such evolution can be attributed partly to the peculiarities of the health-care sector namely, its scope, its status as a large market for computer businesses, and its need for facilities for information storage and management, improvements in quality of care, and expenditure control, in both the public and private sectors.
The health-care systems of developed countries have generally been in existence for at least two decades longer than those in developing countries and have been accumulating experience in the use of such technologies, especially in primary health care, which has provided relevant lessons regarding the use of IT in the health-care system as a whole. The computerization of medical records in hospitals and health clinics; the use of the Internet for communication and information exchange; the development of magnetic cards for user identification; electronic scheduling systems for appointments, examinations and hospital admissions; and computerized protocols for diagnosis and treatment support are just a few examples. Health IT has facilitated access to health literature, both to online journals, books and databases, and offline to informational CD-ROMs, that support practising professionals.
A recent review of the literature on the computerization process in basic health care between 1980 and 1997 summarizes in its title the current situation, i.e. "a descriptive feast but evaluative famine" (1). The authors pointed out the lack of research on the impact of IT on the health status of the population, and the methodological limitations in the design of the studies published so far.
In Brazil, as in other developing countries, structural deficiencies due to the current economic situation have led to considerable deficits in social policies including those related to public health care. Changes in demographic and epidemiological profiles, in urbanization and in the level of industrialization have created a need for new models of health care. Such models attribute an increasing level of importance to primary health care, the strengthening of which is considered central to the improvement of health-care coverage (2, 3).
The goal of the present review is to explore the debate and initiatives concerning the use of IT in primary health care in developing countries.
Methodology
Identification of publications
Publications were identified by an initial generic search using words from any database field (i.e. words from titles, keywords and abstracts) after which all keywords yielding relevant publications were listed. Some of the search terms used were:
- database management systems AND primary health care;
- health information AND primary health care;
- IT AND primary health care;
- information systems AND primary health care;
- software AND primary health care; and
- software AND ambulatory care information systems.
Searches were performed for publication dates ranging from 1992 to 2002, and data sources included MEDLINE, Latin American and Caribbean Health Science Literature Database (LILACS), Cochrane Library and Web of Science. Some additional relevant studies were identified using a demonstration version of EMBASE and from the web site www.hi-europe.info.
Inclusion and exclusion criteria
Duplicate references were excluded, as were references without abstracts, those not specifically related to health, and those that were not concerned with health IT. Publications with abstracts were classified according to the PAHO criteria (Table 1) (4). From the PAHO classification, the three categories most closely related to primary health-care practices were selected, i.e. systems for facilities, decision-support systems and electronic patient registration. The decision-support systems were subdivided into two groups: clinical decision-support (CDS) systems and process and programmatic action evaluation and management (PPAEM) systems. The systems initially considered as belonging to the facility category were later reclassified as belonging either to electronic patient registries (EPR) or to PPAEM.
A further selection of abstracts was performed, including those related to the development and/or evaluation of IT within the scope of primary health care, which emphasized their potential contribution to health-care evaluations. Papers included at this stage were assigned to the following categories: benefits, barriers to implementation and improvement requisites.
Results
The search yielded a total of 884 publications (Fig. 1). Of these, 534 (60.4%) were excluded (references that did not include both an abstract and full text (331); those that were duplicated between databases (14); those with no mention of any kind of health IT, beyond opinion articles, editorials and essays on generic issues related to the subject (139); and publications dealing solely with technical specifications for the writing of software, not necessarily for the health sector (50)).
According to the PAHO classification (Table 1), most of the publications identified (27.7%) described the development and/or implementation of information systems at health facilities, including hospitals, clinics, physician's offices and diagnosis and treatment support centres. These information systems were used for data storage, processing, recovery or diffusion purposes. This group also included articles about systems designed for the management of clinical and administrative information within a specific facility or between different facilities.
The next group (23.1% of the publications identified) was related to decision-support systems that facilitate clinical and administrative decision-making by means of interactive dialogues. These include clinical diagnosis, individual monitoring applications, facility and institution management applications and "virtual health libraries". The third group (18.9%), were publications dealing with electronic data exchange, including general infrastructure designed to allow interaction and information exchange between the users and services and between systems themselves, by means of the Internet and electronic mail. The next group dealt with support systems for educational activity, directed towards distance-learning and improvement of teaching ability in the education and training of health-care professionals (14.0%). Electronic patient registries were the subject of 10.6% of the publications. These are systems that integrate and promote access (from a single site to multiple locations) to collections of clinical and administrative data concerning the patient, based on a distributed database and including different means of support, such as intelligent optical card technology.
The final group comprised publications related to telemedicine such as support systems for diagnosis teleconferences, transmission of high-resolution images and vital signs for long-distance diagnosis and robotic telesurgery. These subjects accounted for 3.1% of the publications, and medical imaging systems designed to store, process, recover and transmit medical images for 2.6%.
Following a detailed analysis of the 350 abstracts included in Table 1, 135 were excluded. These were publications concerned with data exchange, educational activities, and telemedicine or medical imaging, or because of language problems (three). Of the 215 remaining references, 163 were not concerned with the evaluation of technology (Fig. 1).
We attempted to obtain the full texts of the remaining 52 publications: 45 of them were successfully located, corresponding to 13% of the total. We then proceeded to the identification of advantages, problems and potential solutions related to the use of computerized systems in primary health care. From the reference lists included in the articles, and further searches using other sources, seven more relevant publications were located, giving a total of 52.
Electronic patient registries
When compared to manual registration, the main advantages of electronic patient registries (EPR) are greater accuracy (5) and a higher proportion of correct information (6, 7); time saved in locating information (8); more economical use of financial resources; and greater ease and speed of recovery of patient data (12).
Several articles reported on the limitations of such technologies, highlighting the resistance and difficulties of using EPR among health-care professionals, especially physicians (13–15). Emphasis was also placed on aspects related to confidentiality of information and respect for privacy, the need for continuing training and support for human resources (16, 17), and the lack of automatic standardization and codification of the data entered (14).
Most authors agreed on the need for a gradual replacement of paper-based registries with electronic ones, as well as on the need for user-friendly interfaces, and for at least minimal training programmes (18, 19).
Process and programmatic action evaluation and management systems
The first group of process and programmatic action evaluation and management (PPAEM) systems identified were those concerning patient referral and "counter-referral activities" (i.e. the return of the patient to his or her physician after specialist consultation) both between different levels of care (e.g. to specialists or hospitals) and, for example, for the electronic return of the results of laboratory examinations. Their main advantages are reliability (7), speed (8) and the optimization of available resources (7).
The second group of applications were those designed for the monitoring of patients linked to specific health programmes, such as immunization at mother and child clinics, antenatal care and diabetes programmes. This monitoring was mainly carried out by means of "notices" generated when patients missed scheduled appointments, and the issuing of pre-appointment reminders. The advantages reported included reductions in registration errors, identification of absentees, integration of prevention and control activities, and detection of risk factors and complications (8, 20–22).
A third set of publications was concerned with the analysis and extraction of selected information from electronic patient registries, allowing the identification of risk factors and groups of at-risk patients and the obtainment of care-quality indicators and their comparison between different health units (23, 24). The authors agreed that such systems could assist with evaluations of morbidity and patterns of drug prescription (25), allow managers to monitor compliance with conduct and norms regulated between different levels of care (26), and optimize the prevention and early detection of risk factors (27).
One limitation is the lack of studies evaluating the impact of the use of these systems on quality of care (11). Another drawback is the lack of standardization among the different systems which reduces the usefulness of automatically generated indicators (28). When data entry is retrospective, there is a tendency to transfer the deficiencies of a manual registry to the computerized registry (29). It is often necessary to develop additional system tools, such as, for example, codification of the reasons for appointments (30).
Clinical decision-support systems
This category of products includes mainly those that function as computerized protocols for patient management, both for diagnosis and treatment, including electronic prescription and requests for laboratory tests. These may be rule-based systems, cognitive and simulation (Bayesian) systems, or tree-decision systems that could include active patient participation.
Problems such as hypertension and cardiopathies in general (31, 32), asthma (33–35), and depression (36, 37) are among the most cited examples of clinical decision-support (CDS) systems. Such health problems, together with prevention programmes, constitute the main reasons for utilization of primary health care, making the adoption of standardized protocols that can be optimized with the support of IT easier (34). Positive experiences have been reported to result from implementation of these systems (38), including increases in physician adherence to standardized therapeutic plans (32, 33, 39), cost reduction (33), and easier standardization and regulation of requests for secondary and tertiary health care and for examinations (40), thus reducing variability between services.
From an administrative point of view, it is possible to obtain greater adhesion to public policies (41). Standardized programmes for the early detection of diseases would tend to have greater diagnostic value thus contributing towards the promotion of equity, and the reduction of complications and costs related to more complex treatments (42).
As with the other technologies reviewed, the limitations were related to the low adhesion rates among health-care professionals, the great variety of systems available which hindered evaluation of their validity and reproducibility, and difficulties in standardization and integration with other applications (43).
Discussion
Given the rapid progress in health IT in developed countries and in spite of the differences in infrastructure and health facilities, it is useful to extract some relevant lessons for developing countries, especially those that are trying to design health informatics policies for primary care services. MEDLINE, LILACS, the Cochrane Library and the Web of Science were the main sources of published papers, which mainly reported the experiences of developed countries. One possible gap in the present review is related to the fact that papers from non-indexed journals (i.e. those not registered in international databases, such as MEDLINE) could not be identified. We tried to minimize this potential bias by including both papers that reported the experiences in developed countries and experiences in developing countries.
Even with the aid of the PAHO classification, a detailed examination revealed that a single paper may simultaneously encompass several categories, such as electronic patient registries, clinical and management decision-support systems, and process and programmatic action evaluation tools. A further limitation was that many references did not provide abstracts and full text (37%), a possible source of selection bias. However, we consider that the most important papers were included because their abstracts were available from indexed journals. Many papers were focused solely on opinions on the advantages and disadvantages of the use of IT, and lacked any evaluation of their concrete application to health care. This may be partly attributed to the search strategy adopted, which was based on widely inclusive keywords: almost 200 of the publications found were not related to application of IT to the health sector.
There is a consensus concerning the usefulness of computerized systems in primary health care, especially for promoting greater efficiency in management processes. Although studies evaluating the impact of such technologies on indicators of health and quality of care are still rare, most authors agree that positive effects attributed to the implementation of the different systems and applications can be maintained during routine use and improved through monitoring. With regard to EPR the main lessons are related to system security, especially the maintenance of privacy and confidentiality. The interconnection between different systems and software is another relevant issue. It would be imperative to adopt standards for vocabulary, contents, images, objects and communication tasks.
The finding of a low level of adhesion among physicians to protocols for computerization in primary health care is almost ubiquitous. Although the reasons have yet to be explained, it is possible that the autonomy regarding clinical decisions a paradigm of traditional medical practice must be made to coexist with regulated and more cooperative activities, although this will be no easy task. Furthermore, a substantial number of the articles reviewed stressed the need for continued motivation and training for all team members as an important requisite for the success of any initiative in this area (19). This lesson would be very relevant to the establishment of IT in primary health-care systems. It may be pertinent here to quote the reflections by Branco (2) on the significance of training, that is, the amplification of knowledge: "… knowledge of the logic behind health information production and flux must be provided to all persons involved, and should include the understanding of the goals of the systems to which they have access, and of the utilization possibilities of the information produced …"
Another consensual aspect was the difficulty of finding adequate methods for evaluation, given the enormous variety of applications and contexts in which IT is used. Similarly to the situation for medicinal drugs some 40 years ago, IT has not yet been regulated to ensure its safety and efficacy. Thus, every facility or organization, in order to fulfil its particular needs, "orders" specific products to provide a solution to specific problems, reproducing the specialists' view of health in their administrative and evaluative demands. The results of specific evaluations lack external validation, because health services are extremely variable in terms of population seen, team composition, qualifications, motivation and extent of computerization. This hampers comparability and generalizability (37). In addition, the complexity of clinical and organizational management processes is often underestimated (44).
In the consideration of CDS systems in particular, emphasis has been placed on quality and safety concerns. The main drawbacks of such systems include the lack of consensual standardization for a number of conditions, the probably negative effect on the physician–patient relationship (for example, the perception that computers take over the physician's role), the difficulty in addressing complex conditions, the profusion of different systems with different formats, and the need for training and support (1).
The most important lesson comes from the trends in adoption of national health information systems. Countries such as Australia, Canada and England have recently been heavily involved in implementing large information systems in an unprecedented effort towards standardization and the incorporation of new technologies (48–50). Scotland is another example; it maintains a single information system that functions in 75% of its services, thus facilitating the comparison of data and the extension of benefits resulting from its improvement (26). Brazil is currently planning a national health information policy that should lead to a significant improvement in public health care, especially in primary health-care services. If the developing countries learnt the important lessons provided by the developed countries, they would be able to reduce the time and resources required to increase IT utilization. 
Conflicts of interest: none declared.
References
1. Mitchell E, Sullivan F. A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97. BMJ 2001;322:279-82.
2. Branco MAF. Informação e tecnologia: desafios para a implantação da Rede Nacional de Informações em Saúde. [Information and technology: challenges to develop a national health information network.] Physis: Rev Saúde Coletiva 1998;8:95-123. In Portuguese.
3. Bodstein R. Atenção básica na agenda da saúde. [Primary care in the health agenda.] Ciênc Saúde Coletiva 2002;7:401-12. In Portuguese.
4. Organización Panamericana de la Salud. Sistemas de información y tecnologia de información en salud: desafios y soluciones para América Latina y el Caribe. [Health information systems and technologies: challenges and perspectives to Latin America and the Caribbean.] Washington: Pan American Health Organization; 1998. In Spanish.
5. Hassey A, Gerrett D, Wilson A. A survey of validity and utility of electronic patient records in a general practice. BMJ 2001;322:1401-5.
6. Flygt C, Homelius B, Jacobsson B, Foldevi M, Trell E. Essential data set for computer management of distributed primary care services. Medical Informatics 1995;20:331-41.
7. Gaudet LA. Electronic referrals and data sharing: can it work for health care and social service providers? Journal of Case Management 1996;5:72-7.
8. Borowitz SM. Impact of a computerized patient tracking system in a pediatric clinic. International Journal of Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium Medical Informatics 1996. p. 508-11.
9. Ornstein SM, Garr DR, Jenkins RG. A comprehensive microcomputer-based medical records system with sophisticated preventive services features for the family physician. Journal of the American Board of Family Practice 1993;6:55-60.
10. Singh AK, Kohli S, Moidu K, Bostrom K, Trell E, Wigertz O. Primary health care computing analysis of Swedish maternal health records. Journal of Medical Systems 1994;18:221-8.
11. Branger PJ, van't Hooft A, van der Wouden JC, Moorman PW, van Bemmel JH. Shared care for diabetes: supporting communication between primary and secondary care. International Journal of Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium Medical Informatics 1999;53:133-42.
12. Wager KA, Lee FW, White AW, Ward DM, Ornstein SM. Impact of an electronic medical record system on community-based primary care practices. Journal of the American Board of Family Practice 2000;13:338-48.
13. Manchester GW, Raia TJ 3rd, Scott J, Emery L, Russo AR. Primary Care Health Information System: a hybrid electronic–paper medical record system. Journal of Ambulatory Care Management 1992;15:13-29.
14. Chin HL, Krall MA. Successful implementation of a comprehensive computer-based patient record system in Kaiser Permanente Northwest: strategy and experience. Effective Clinical Practice 1998;1:51-60.
15. Souther E. Implementation of the electronic medical record: the team approach. Computers in Nursing 2001;19:47-55.
16. Blignaut PJ, McDonald T, Tolmie CJ. System requirements for a computerised patient record information system at a busy primary health care clinic. Curationis 2001;24:68-76.
17. Williamson L, Stoops N, Heywood A. Developing a district health information system in South Africa: a social process or technical solution? Medinfo 2001;10:773-7.
18. Mangoud AM, Al-Ruwashed FA. A computer program to fit a family and community medicine set-up. Computer Methods and Programs in Biomedicine 2002;68:129-34.
19. Nobel J. Changes in health care: challenges for information system design. International Journal of Bio-medical Computing 1995;39:35-40.
20. Singh AK, Kohli M, Trell E, Wigertz O, Kohli S. Bhorugram (India): revisited. A 4 year follow-up of a computer-based information system for distributed MCH services. International Journal of Medical Informatics 1997;44:117-25.
21. Szecsenyi J, Uphoff H, Ley S, Brede HD. Influenza surveillance: experiences from establishing a sentinel surveillance system in Germany. Journal of Epidemiology and Community Health 1995;49 Suppl:1:9-13.
22. Wei F, Wright K, Heaton T, Kincaid W. A regional integrated information system to assure maternal-child health services: a progress report. American Journal of Preventive Medicine 1996;12 Suppl:20-5.
23. Adams WG, Conners WP, Mann AM, Palfrey S. Immunization entry at the point of service improves quality, saves time, and is well-accepted. Pediatrics 2000;106:489-92.
24. Assaf AR, Banspach SW, Lasater TM, Ramsey J, Tidwell RJ, Carleton RA. The FPbase microcomputer system for managing community health screening and intervention data bases. Public Health Reports 1992;107:695-700.
25. Wilson AE, Pollock C, Weekes T, Dowell A. Can general practice provide useful information? Evaluation of a primary health care information project in northern England. Journal of Epidemiology and Community Health 1995;49:227-30.
26. Milne RM, Taylor MW, Taylor RJ. Audit of populations in general practice: the creation of a national resource for the study of morbidity in Scottish general practice. Journal of Epidemiology and Community Health 1998;52 Suppl 1:20S-24S.
27. Ivankovic D, Kern J, Bartolic A, Vuletic S. Analysis of data as information: quality assurance approach. International Journal of Health Care Quality Assurance 1993;6:9-12.
28. Rascon-Pacheco RA, Santillana-Macedo MA, Romero-Arredondo ME, Rivera-Icedo BM, Romero-Cancio JA, Cota-Rembau AI. Epidemiologic monitoring system for the diabetic patient; use of computerized technology in the quality of medical care. Salud Publica de Mexico 2000;42:324-32.
29. McColl A, Roderick P, Smith H, Wilkinson E, Moore M, Exworthy M, et al. Clinical governance in primary care groups: the feasibility of deriving evidence-based performance indicators. Quality in Health Care 2000;9:90-7.
30. Letrilliart L, Viboud C, Boelle PY, Flahault A. Automatic coding of reasons for hospital referral from general medicine free-text reports. International Journal of Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium 2000. p. 487-91.
31. Pinciroli F, Combi C, Pozzi G, Rossi R. MS2/Cardio: towards a multi-service medical software for cardiology. Methods of Information in Medicine 1992;31:18-27.
32. Starkey C, Michaelis J, de Lusignan S. Computerised systematic secondary prevention in ischaemic heart disease: a study in one practice. Public Health 2000;114:169-75.
33. Austin T, Iliffe S, Leaning M, Modell M. A prototype computer decision support system for the management of asthma. Journal of Medical Systems 1996;20:45-55.
34. Khoury AT. Support of quality and business goals by an ambulatory automated medical record system in Kaiser Permanente of Ohio. Effective Clinical Practice 1998;1:73-82.
35. Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ 2002;325:941.
36. Kobak KA, Taylor LH, Dottl SL, Greist JH, Jefferson JW, Burroughs D, et al. A computer-administered telephone interview to identify mental disorders. JAMA 1997;278:905-10.
37. Unutzer J, Katon W, Williams JW Jr, Callahan CM, Harpole L, Hunkeler EM, et al. Improving primary care for depression in late life: the design of a multicenter randomized trial. Medical Care 2001;39:785-99.
38. Bradley JH, King DE. Electronic medical records for prenatal patients: challenges and solutions. MD Computing: Computers in Medical Practice 1998;15:316-22, 331.
39. Papshev D, Peterson AM. Electronic prescribing in ambulatory practice: promises, pitfalls, and potential solutions. American Journal of Managed Care 2001;7:725-36.
40. Alonso Lopez FA, Anzola Fernandez B, Arratibel Arrondo I, Gancedo Gonzalez Z. The evaluation of a computer program for the filling of repeat prescriptions. Atencion primaria / Sociedad Espanola de Medicina de Familia y Comunitaria 1993;12:96-8.
41. Chuang CT. An efficient fault-tolerant out-patient order entry system based on special distributed client/server architecture. Medical Informatics 1998;23:145-57.
42. FitzHenry F, Kiepek WT, Shultz EK, Byrd J, Doran J, Miller RA. Implementing outpatient order entry to support medical necessity using the patient's electronic past medical history. Proceedings: a Conference of the American Medical Informatics Association. AMIA Annual Fall Symposium 2002 p. 250-4.
43. Delaney BC, Fitzmaurice DA, Riaz A, Hobbs FD. Can computerised decision support systems deliver improved quality in primary care? Interview by Abi Berger. BMJ 1999;319:1281.
44. Cibulskis RE, Posonai E, Karel SG. Initial experience of using a knowledge based system for monitoring immunization services in Papua New Guinea. Journal of Tropical Medicine and Hygiene 1995;98:107-13.
45. Rigby M, Forsstrom J, Roberts R, Wyatt J. Verifying quality and safety in health informatics services. BMJ 2001;323:552-6.
46. Heathfield H, Pitty D, Hanka R. Evaluating information technology in health care: barriers and challenges. BMJ 1998;316:1959-61.
47. Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised health information systems: hard lessons still to be learnt. BMJ 2003;326:860-3.
48. Australian Department of Health and Ageing, National Health Information Management Advisory Council (NHIMAC). Health Online: a health information action plan for Australia; 2003. Available from URL: http://www.health.gov.au/healthonline
49. Government of Canada. Canada Health Infoway. Toronto, Ontario; 2003. Available from URL: http://www.canadahealthinfoway.ca
50. Majeed A. Ten ways to improve information technology in the NHS. BMJ 2003;326:202-6.
Submitted: 30 July 2003 – Final revised version received: 9 May 2004 – Accepted: 19 May 2004
1 Correspondence should be sent to Dr Tomasi (email: etomasi@epidemio-ufpel.org.br).
Additional references for Tables 3 and 4
Af Klercker T, Zetraeus S. Dilemmas in introducing World Wide Web-based information technology in primary care: a focus group study. Family Practice 1998;15:205-10.
Bergvin L, Johansson B, Borjesson U. Distribution of laboratory test results to primary health care centres with the EDIFACT standard. Clinica Chimica Acta 1993;222:141-5.
Dewey JB, Manning P, Brandt S. Acceptance of direct physician access to a computer-based patient record in a managed care setting. Proceedings of the Annual Symposium on Computer Application in Medical Care 1993:79-83.
Gerbert B, Bronstone A, Maurer T, Hofmann R, Berger T. Decision support software to help physicians triage skin cancer: a pilot study. Archives of Dermatology 2000;136:187-92.
Gordon C, Jackson-Smale A, Thomson R. DILEMMA: logic engineering in primary care, shared care and oncology (AIM Project A2005). Computer Methods and Programs in Biomedicine 1994;45:37-9.
Hobbs FD, Delaney BC, Carson A, Kenkre JE. A prospective controlled trial of computerized decision support for lipid management in primary care. Family Practice 1996;13:133-7.
Margolis CZ, Warshawsky SS, Goldman L, Dagan O, Wirtschafter D, Pliskin JS. Computerized algorithms and pediatricians' management of common problems in a community clinic. Academic Medicine 1992;67:282-4.
Ross KM. Complex ambulatory settings demand scheduling systems. Ambulatory Outreach 1998:23-5.
Safran C. Searching for answers on a clinical information system. Methods of Information in Medicine 1995;34:79-84.
Shiffman RN, Brandt CA, Freeman BG. Transition to a computer-based record using scannable, structured encounter forms. Archives of Pediatrics and Adolescent Medicine 1997;151:1247-53.
Tirol R, Menezes PR. Computerized psychiatric interviews: can they help to improve medical care? Revista Paulista de Medicina 1998;116;1821-2.