POLICY AND PRACTICE
Research synthesis and dissemination as a bridge to knowledge management: the Cochrane Collaboration
Collaboration Cochrane : comment relier synthèse et diffusion des travaux de recherche et gestion des connaissances
La síntesis y difusión de investigaciones como eslabón fundamental de la gestión de conocimientos: la Cochrane Collaboration
Jimmy VolminkI,1; Nandi SiegfriedII; Katharine RobertsonIII; A. Metin GülmezogluIV
IProfessor, Primary Health Care Directorate, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa (jvolmink@cormack.uct.ac.za)
IINuffield Medical Fellow, University of Oxford, Oxford, England
IIIResearch Fellow, Primary Health Care Directorate, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
IVMedical Officer, UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Reasearch Training in Human Reproduction (HRP), Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland
ABSTRACT
In the current information age, research synthesis is a particularly useful tool for keeping track of scientific research and making sense of the large volumes of frequently conflicting data derived from primary studies. The Cochrane Collaboration is a global initiative "to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of healthcare interventions". In this paper we set the work of the Cochrane Collaboration in historical perspective, explain what a Cochrane review is, and describe initiatives for promoting worldwide dissemination of synthesized information. We also consider emerging evidence of the Cochrane Collaboration's impact on health-care practice, policy, research and education. Finally, we highlight the need for increased investment in the preparation and maintenance of Cochrane reviews, particularly those that address health issues that are relevant to people living in low- and middle-income countries.
Keywords: Research; Databases, Factual/history/utilization; Review literature; Evidence-based medicine; Information dissemination; Delivery of health care (source: MeSH, INSERM).
RÉSUMÉ
À l'âge de l'information, la synthèse des travaux de recherche constitue un outil particulièrement utile pour garder la trace des recherches scientifiques et donner un sens aux gros volumes de données souvent contradictoires tirées des études primaires. La Collaboration Cochrane est une initiative mondiale pour aider les personnes à prendre des décisions éclairées concernant les soins de santé, en préparant et en tenant à jour des revues systématiques des effets des interventions thérapeutiques et préventives, et en favorisant l'accès à ces revues. Le présent article place le travail de la Collaboration Cochrane dans une perspective historique, explique ce qu'est une revue Cochrane et décrit les initiatives pour promouvoir la diffusion dans le monde entier des informations synthétisées. Il examine également l'émergence de preuves attestant l'impact des activités de la Collaboration sur la pratique, la politique, la recherche et la formation médicales. Enfin, il souligne la nécessité de renforcer les investissements dans la préparation et le maintien à jour des revues Cochrane, notamment celles traitant de questions sanitaires relatives à des personnes vivant dans des pays à revenu faible ou moyen.
Mots clés: Recherche; Base données factuelles/histoire/utilisation; Revue de la littérature; Médecine factuelle; Diffusion de l'information; Délivrance soins (source: MeSH, NLM).
RESUMEN
En la actual era de la información, la síntesis de investigaciones es una herramienta particularmente valiosa para conocer los últimos trabajos científicos y extraer sentido de las grandes cantidades de datos, con frecuencia conflictivos, aportados por los estudios primarios. La Cochrane Collaboration es una iniciativa mundial que tiene por objeto «ayudar a las personas a tomar decisiones fundamentadas acerca de la atención de salud, procediendo para ello a preparar, mantener y divulgar revisiones sistemáticas sobre los efectos de la atención sanitaria». En este artículo analizamos los trabajos de la Cochrane Collaboration desde una perspectiva histórica, explicamos en qué consiste una revisión Cochrane y describimos diversas iniciativas destinadas a promover la difusión mundial de información sintetizada. También examinamos los nuevos datos disponibles acerca del impacto de la Cochrane Collaboration en la práctica, las políticas, las investigaciones y la educación sanitarias. Por último, destacamos la necesidad de hacer una mayor inversión en la preparación y el mantenimiento de las revisiones Cochrane, en particular de las que abordan temas de salud de especial interés para las personas que viven en los países de ingresos bajos y medios.
Palabras clave: Investigación; Bases de datos factuales/historia/utilización; Literatura de revisión; Medicina basada en evidencia; Acceso a la información; Diseminación de la información; Prestación de atención de salud (fuente: DeCS, BIREME).
Introduction
Health-care practice and policy decisions should be based on up-to-date syntheses of reliable and relevant research (1). The type of research chosen for inclusion in a synthesis of the results of research depends on the nature of the question one is trying to answer (2). Randomized controlled trials provide the best evidence for making decisions on what does and does not work (3). However, evidence of this kind is often not available or, even when available, might not be readily accessible. Consequently, practitioners, policy-makers and patients remain ignorant of the true effects of many interventions, even those that are routinely used in health care.
In 1979, the epidemiologist and physician Archie Cochrane noted in a now famous essay: "It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials" (4). The Cochrane Collaboration was formed in response to this challenge in 1993. Its declared mission is "to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of healthcare interventions" (5). The collaboration was established in Oxford, England, and there are now in excess of 10 000 people from more than 80 countries contributing to its work. This article aims to set the work of the Cochrane Collaboration in historical perspective, to explain what a Cochrane review is, and to describe current initiatives for promoting worldwide dissemination of synthesized information. It also discusses some evidence on the impact of Cochrane reviews and calls for increased investment in reviews addressing global health priorities. Further information about the Cochrane Collaboration can be found elsewhere (69).
Research synthesis in historical perspective
In the current information age, synthesizing the results of research is a particularly useful tool for keeping track of scientific research and making sense of the large volumes of frequently conflicting data derived from primary studies. However, the idea of research synthesis is not new (10). As far back as 1753, James Lind, the Scottish naval surgeon credited for proving that oranges and lemons had therapeutic effects in the treatment of scurvy, recognized that systematic methods for identification, extraction and appraisal of information from individual studies were necessary for reducing bias in the interpretation of research (11). (See Public Health Classic and Commentary, pp 791-796.) These techniques, enhanced in recent years by developments in information technology and epidemiology, remain fundamental to the process of research synthesis.
A different aspect of research synthesis finds its roots in astronomy. Towards the end of the 17th century astronomers found that combining data from individual studies could enhance the certainty of their observations (10, 12). Such methods for reducing statistical imprecision (known today as meta-analysis), were first introduced into medical research 100 years ago by the statistician Karl Pearson. In the course of reporting on the evidence relating to the use of serum inoculations to prevent enteric fever Pearson wrote: "Many of the groups … are far too small to allow of any definite opinion being formed at all, having regard to the size of the probable error involved" (13).
He went on to calculate correlation coefficients for each of 11 studies and synthesized these coefficients to produce "average correlations." Statistical procedures for combining the results of individual studies were further refined by statisticians working in the field of agriculture in the 1930s (notably Ronald Fisher). These methods were, however, not widely used until the 1970s when social scientists embraced them (and coined the term meta-analysis).
Notable efforts to apply the tools of meta-analysis in health care began during the mid-1980s with the fields of cardiovascular disease, oncology and perinatal care leading the way (10). These developments led to some important medical breakthroughs. For example, in 1988 the Early Breast Cancer Trialists' Collaborative Group used meta-analysis to establish the beneficial effects of tamoxifen and cytotoxic therapy on mortality in patients with early breast cancer (14).
Beginning in 1978 the work of Iain Chalmers and colleagues at the UK's National Perinatal Epidemiology Unit led to a large compilation of controlled trials in perinatal medicine (15). This prompted an international collaborative effort to prepare hundreds of systematic reviews on trials relating to pregnancy, childbirth and the neonatal period and culminated in the publication of the two-volume compendium Effective care in pregnancy and childbirth (16). These early seminal works of research synthesis in health care arose out of international collaboration in specific fields. Global collaboration remains an essential ingredient for the success of efforts to synthesize research, and it is therefore one of the key principles upon which the Cochrane Collaboration is founded.
Systematic reviews and the Cochrane Collaboration
Research synthesis uses systematic methods to overcome two types of challenges: bias (systematic error) and statistical imprecision (random error). While it is always important to minimize bias, statistical pooling of the results of different studies to yield a more precise estimate overall is not always possible or appropriate. Because these two dimensions of research synthesis tend to be confused, the Cochrane Collaboration has adopted the term "systematic review" to highlight the distinction. The Cochrane reviewers' handbook glossary states that a systematic review is "a review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarize the results of the included studies" (17). The steps involved in conducting a systematic review are similar to those of other types of research (Box 1).
The Cochrane Collaboration has provided a unique global infrastructure for preparing and maintaining systematic reviews. Cochrane systematic reviews (also known as Cochrane reviews) focus on the benefits and risks of health-care interventions, follow a set format, undergo extensive peer review (starting at the protocol stage), are published electronically in the Cochrane Library, and are updated periodically in light of new information and when other ways of improving them are identified. In 2004, issue 1 of the Cochrane Library included nearly 2000 completed Cochrane reviews and more than 1400 review protocols of reviews in progress (5). Although these reviews address a wide range of health problems, much work remains to be done since recent estimates indicate that at least 10 000 reviews will be necessary to cover all of the important questions related to the effectiveness of health care (18). Keeping all of these Cochrane reviews up to date will be a formidable challenge.
To ensure that Cochrane reviews are focused on topics relevant to all consumers of health care, the Cochrane Collaboration actively promotes widespread participation by stakeholders at all stages of the review process. For instance, the Cochrane Consumer Network uses various mechanisms to enable consumers to comment on reviews and protocols prior to publication, to identify important topics for review, and to disseminate the findings of Cochrane reviews to a wide consumer audience. In addition, in response to a survey of reviewers living in developing countries, the collaboration has launched a developing country initiative to increase participation in Cochrane activities by researchers and consumers living in these countries.
Are Cochrane reviews reaching health-care decision-makers?
In the sea of health information available in print and electronic media, much of which is of poor quality, Cochrane reviews represent islands of more reliable evidence. Such information provides a more secure foundation for informing the decisions made by health-care providers, policy-makers, researchers and consumers than do traditional reviews or consensus statements by experts (19). Access to Cochrane reviews is therefore crucial.
Cochrane Library
The primary form of dissemination of Cochrane reviews is via the Cochrane Library, which is available on CD-ROM and over the Internet through paid subscription (20). Access to the Cochrane Library, while still a significant problem for people in living in low-income and middle-income countries, is steadily improving as institutional and national subscriptions become more common. In several industrialized countries government grants enable citizens to use the library at no cost (Box 2). In low- and middle-income countries, such as Brazil and South Africa, free access to the library is provided through national subscriptions or through global initiatives promoting access to health-care information, such as the Health InterNetwork Access to Research Initiative (HINARI) (21).
HINARI is a partnership led by WHO that allows institutions in low-income countries with annual per capita income of < US$ 1000 to have free online access to the library and more than 2000 scientific periodicals. Countries with an annual per capita income between US$ 1000 and US$ 3000 can obtain access by paying a nominal subscription charge. For institutions to benefit from this initiative reliable, fast and affordable Internet access is needed. Unfortunately this is not available in many low- and middle-income countries. In addition, it is likely that those accessing journals via HINARI will be primarily researchers. Innovative approaches are, therefore, required to reach health workers in developing countries. These may include the use of different forms of media, such as CD-ROMs and print products that summarize the information in an easily readable and understandable format for clinicians. Using satellite technology for digital broadcasting that is downloadable to a personal computer seems promising but currently it is expensive (22).
Specialized versions of the Cochrane Library
A number of projects have been launched with the aim of providing access via CD-ROM to Cochrane reviews covering a specific field. These specialist databases include the WHO Reproductive Health Library (23), the Mental Health Library (24) and the Cancer Library (25). The Reproductive Health Library focuses on reproductive health problems associated with the highest global disease burden, and it is the most well established of these databases. In addition to full-text Cochrane reviews, the Reproductive Health Library includes peer-reviewed commentaries that discuss the relevance of the evidence to resource-poor settings. This library is updated annually and is available on a free-subscription basis that is, access is free but there is a formal subscription process to people based in low- and middle-income countries; it is available through a paid subscription to those in industrialized countries. There are more than 13 000 subscribers to the Reproductive Health Library and most of them are based in developing countries.
Cochrane reviews in health-care journals
Given their high quality (26) it is not surprising that editors of major health-care journals are keen to consider publishing Cochrane reviews. The BMJ, Lancet, JAMA, Journal of Health Services Research and Policy, Obstetrics and Gynecology, South African Medical Journal and the British Journal of Surgery all have policies that welcome submission of versions of Cochrane reviews. Other journals routinely publish abstracts of Cochrane reviews with supplementary commentaries that help clinicians put the evidence into context. The best known examples of this type of publication are the American College of Physicians Journal Club, Evidence-based Medicine and the American Journal of Family Medicine (through its Cochrane for Clinicians series).
Other forms of dissemination
Practitioners and policy-makers typically have limited time for reading, and it is therefore important to have shorter versions of Cochrane reviews tailored to their needs. The Global Health Council's Evidence for Action series provides one-page synopses of Cochrane and other systematic reviews for its worldwide membership. These summaries are produced monthly and made available on the Council's web site and through its paper publications (27). Similarly, the Effective Health Care Alliance Programme at the Liverpool School of Tropical Medicine provides short summaries of Cochrane reviews under the name Evidence Update and distributes these via the World Wide Web (28).
Other publications draw heavily on Cochrane reviews. Clinical Evidence (29) is a regularly updated directory of evidence published by the BMJ Publishing Group. It attempts to provide information in a user-friendly format for clinicians. Information is arranged under headings that are clinical questions typically asked by practitioners. Clinical Evidence provides a concise account of the state of knowledge and uncertainty about the treatment and prevention of common clinical conditions using findings from systematic reviews (frequently drawn from the Cochrane Library). Clinical Evidence is available both in print and online, and is widely distributed in Europe and North America. It is also available to qualifying low- and middle-income countries through the HINARI gateway (21).
Impact of the Cochrane Collaboration
Evidence generated by the Cochrane Collaboration is undoubtedly reaching an ever- increasing readership. But is it making a difference? The proponents of evidence-based health care have been accused of failing to use their own rigorous criteria to evaluate their impact on practice and policy (30). However, the complex nature of behaviour change within professional practice and the multiple influences on policy development (31) mean that the true impact of the collaboration may be difficult to test. Silagy et al. (32) warn that it is important not to evaluate the effects of such a group prematurely or use inappropriate outcomes. With these caveats in mind we offer the following as useful indicators of the impact of the collaboration.
First, there is growing evidence that practitioners, policy-makers, educators and consumers are giving more attention to Cochrane reviews. The evidence for this is as follows:
1. subscriptions to the Cochrane Library have grown, with a large number of countries seeking access to the library for their citizens (Box 2);
2. in many health-care training institutions use of the Cochrane Library is incorporated into undergraduate curricula;
3. interactive post-graduate training programmes and continuing medical education programmes drawing on Cochrane reviews have been launched in many parts of the world. For example, the Effective Health Care Alliance Programme has implemented the Better Births Initiative (33); there are educational workshops based on the Reproductive Health Library (24); and the Evidence-based Reproductive Health workshop and a board game have been developed by the South African Cochrane Centre (34);
4. those involved in developing clinical practice guidelines are increasingly basing their recommendations on Cochrane reviews, e.g. national asthma guidelines in several countries including Australia, Canada and India;
5. the Cochrane Collaboration and WHO's Essential Drugs and Medicines department work together on the revisions and updates of the List of Essential Medicines (35);
6. consumer health web sites quote Cochrane reviews as their source of evidence on the effects of interventions, e.g. Informedhealthonline (http://www.informedhealthonline.org/item.aspx).
Two examples show how Cochrane reviews are beginning to influence far-reaching health policies. WHO has long promoted the use of oral rehydration solution (ORS) with a specific sugar and salt content to treat dehydration associated with diarrhoea. While it is effective for reducing mortality regardless of the cause of diarrhoea, the standard ORS formulation does not lower stool output or duration of diarrhoea, thus reducing its acceptability in many communities. A WHOUNICEF expert committee recently recommended that countries move away from manufacturing and using a standard ORS in favour of a reduced osmolarity formula (36). This decision was influenced by a Cochrane review demonstrating that a reduced osmolarity ORS is safe and more effective than the standard ORS (37). It has been predicted that the global adoption of this new ORS formula will prevent 14 000 deaths and save US$ 7.1 million for every 1 million episodes of diarrhoea (36).
A second example concerns the reintroduction of amodiaquine to treat malaria. Amodiaquine, previously banned following case reports of haematological side effects in people using the drug for malaria prophylaxis, was reintroduced to WHO's essential drugs list for the treatment of malaria in 2003. This followed the publication of a Cochrane review that included a number of unpublished studies and reports published in languages other than English that showed for the first time that the drug was more effective than and as safe as chloroquine (38).
The second indicator of the impact of the collaboration is that funders and research ethics committees are starting to heed the message propagated by the collaboration that new research should be undertaken in the light of systematic reviews of relevant research. The UK National Health Service's Health Technology Assessment Programme and the British, Dutch and South African medical research councils now require researchers to conduct systematic reviews before they consider funding specific trials. In Denmark, the national research ethics committee system requires that researchers applying for ethical approval of new research prove that the proposed study is necessary by providing an adequate synthesis of existing research.
The third indicator of the collaboration's impact is that its centralized database of controlled trials, which is accessible through the Cochrane Library, has become a valuable resource for those conducting systematic reviews or searching for trials. The database comprises more than 400 000 reports of studies; thus it is the most comprehensive source of information on trials in the world and includes citations that may not be available through widely used bibliographic databases, such as Medline and EMBASE, citations available in languages other than English, those available only in conference proceedings, and even previously unpublished material (39). As part of this international effort to locate and register existing trials, the South African Cochrane Centre has embarked upon a unique initiative called the African Trials Register (40). This project involves tracking down all controlled trials conducted in Africa by searching global and regional bibliographic databases and hand-searching African journals. The African Trials Register aims to ensure that the results of trials conducted in Africa do not disappear and are available for inclusion in systematic reviews (41).
The collaboration and its members have also called for prospective registration of all randomized controlled trials throughout the world so that the progress of a trial and its results, whether published or not, will be in the public domain or at the very least be available to researchers (42).
Finally, methodological research undertaken by members of the Cochrane Collaboration has contributed to improving the quality of systematic reviews. Such research spans the entire systematic review process including identifying hard-to-find trials, assessing the quality of trials, managing and analysing data, and editing reviews. This work, along with similar efforts made outside the collaboration, is documented in the Cochrane Methodology Register in the Cochrane Library.
The way ahead
The Cochrane Collaboration has launched a global effort to make the best available evidence on the effects of health-care interventions available to health-care decision-makers. In its 10 years of existence it has made immense strides in terms of producing and disseminating systematic reviews. This information is starting to have a significant impact on education, practice, research and policy. However, much work lies ahead.
Substantial resources and considerable commitment will be required to prepare reviews on as yet untouched topics and to continually update these reviews in the future. The proportion of Cochrane reviews addressing health issues relevant to people living in less-developed countries is still small (43, 44). However, this is changing. The collaboration, governed as it is by the principles of building on the enthusiasm of individuals, ensuring relevance and enabling wide participation, is actively encouraging reviewers from developing countries to participate through its Developing Country Initiative. There is also a pressing need for reviews focusing not only on individual-level determinants of health but also on the often more important macro-level determinants (45). To address these gaps, specific Cochrane entities have been established, including the Effective Practice and Organisation of Care Collaborative Review Group and the Health Promotion and Public Health Field. Plans are under way for the registration of a group that will focus on systematic reviews of interventions to promote health equity (P. Tugwell, personal communication, 2003).
At the very least, the collaboration can be said to have increased access to vital knowledge on the effects of health care. It has been effective in criticizing complacent and uncritical forms of health care based on assumptions of benefit rather than proof and encouraging an evidence-based approach to practice, training, research and policy-making (45). Promoting a collaborative rather than a competitive philosophy has allowed researchers from all over the world to work together to gather the evidence required to make informed health-care decisions. It remains to be seen to what extent the efforts of the Cochrane Collaboration will move evidence-based health care from rhetoric to reality in the long term. 
Conflicts of interest: none declared.
References
1. Chalmers I. The James Lind initiative. Journal of the Royal Society of Medicine 2003;96:575-6.
2. Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. BMJ 2004;328:39-41.
3. Sackett DL. Evidence-based medicine: how to practice and teach EBM. New York: Churchill Livingstone; 2000.
4. Cochrane AL. 1931-1971: a critical review, with particular reference to the medical profession. In: Teeling-Smith G, Wells N, editors. Medicines for the year 2000. London: Office of Health Economics; 1979. p. 1-11.
5. Cochrane Collaboration. Cochrane: the reliable source of evidence in healthcare, 2003. Available from: http://www.cochrane.org
6. Bero L, Rennie D. The Cochrane Collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. JAMA 1995;274:1935-8.
7. Laupacis A. The Cochrane Collaboration how is it progressing? Statistical Medicine 2002;21:2815-22.
8. Clarke M. The Cochrane Collaboration: providing and obtaining the best evidence about the effects of health care. Evaluation and the Health0 Professions 2002;25:8-11.
9. Ault A. Climbing a medical Everest. Science 2003;300:2024-5.
10. Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Evaluation and the Health Professions 2002;25:12-37.
11. Lind J. A treatise of the scurvy. Stewart CP, Guthrie D, editors. Edinburgh: Edinburgh University Press, 1953.
12. Egger M, Davey Smith G, Altman DG, editors. Systematic reviews in health care: meta-analysis in context. 2nd ed. London: BMJ Publishing Group; 2001.
13. Pearson K. Report on certain enteric fever inoculation statistics. BMJ 1904;iii:1243-6.
14. Early Breast Cancer Trialists' Collaborative Group. Effects of adjuvant tamoxifen and of cytotoxic therapy on morality in early breast cancer: an overview of 61 randomized trials among 28,896 women. New England Journal of Medicine 1988;319:1681-92.
15. National Perinatal Epidemiology Unit. A classified bibliography of controlled trials in perinatal medicine 1940-1984. Oxford: Oxford University Press; 1986.
16. Chalmers I, Enkin M, Keirse MJNC, editors. Effective care in pregnancy and childbirth. Oxford: Oxford University Press; 1989.
17. Clarke M, Oxman AD, editors. Cochrane reviewers' handbook glossary: version 4.1.5. Issue 1. Chichester: John Wiley; 2004 (also available from http://www.cochrane.org/resources/handbook/glossary.pdf).
18. Mallett S, Clarke M. How many Cochrane reviews are needed to cover existing evidence on the effects of healthcare interventions? ACP Journal Club 2003;139:A11.
19. Irwig L, Zwarenstein M, Zwi A, Chalmers I. A flow diagram to facilitate selection of interventions and research for health care. Bulletin of the World Health Organization 1998;76:17-24.
20. Update Software. The Cochrane Library, 2003. Available from: http://www.update-software.com/cochrane/
21. Health InterNetwork. Health InterNetwork, 2003. Available from: http://www.healthinternetwork.org/
22. WorldSpace Satellite Radio. Welcome to WorldSpace. Available from: http://www.worldspace.com/
23. World Health Organization. The WHO reproductive health library, 2004. Available from: http://www.who.int/reproductive-health/rhl/
24. Update Software. Mental health library, 2002. Available from: http://www.update-software.com/mhl/mhlogon.htm
25. Update Software. The cancer library, 2004. Available from: http://www.update-software.com/cancer/
26. Shea B, Moher D, Graham I, Pham B, Tugwell P. A comparison of the quality of Cochrane reviews and systematic reviews published in paper-based journals. Evaluation and the Health Professions 2002;25:116-29.
27. Global Health Council. The Global Health Council, 2004. Available from: http://www.globalhealth.org/
28. Liverpool School of Tropical Medicine. Effective Health Care Alliance Programme: Evidence Update, 2004. Available from: http://www.liv.ac.uk/lstm/ehcap/EU/evidenceupdate.htm
29. Clinical Evidence. Welcome to Clinical Evidence, 2004. Available from: http://www.clinicalevidence.com/
30. Miles A, Grey JE, Polychronis A, Price N, Melchiorri C. Current thinking in the evidence-based health care debate. Journal of Evaluation in Clinical Practice 2003;9:95-109.
31. Black N. Evidence based policy: proceed with care. BMJ 2001;323:275-9.
32. Silagy CA,Weller DP, Middleton PF, Doust JA. General practitioners' use of evidence databases. Medical Journal of Australia 1999;170:393.
33. Liverpool School of Tropical Medicine. Effective Health Care Alliance Programme: Better Births Initiative, 2004. Available from: http://www.liv.ac.uk/lstm/ehcap/BBI/bbimainpage.htm
34. Siegfried NB, Bredekamp B, Spruyt L. Making evidence-based decisions in reproductive health: monitoring and evaluation toolkit. Geneva: WHO, Department of Reproductive Health and Research; 2003.
35. World Health Organization. WHO Essential Medicines Library, 2004. Available from: http://mednet3.who.int/eml/eml_intro.asp
36. WHO, UNICEF. Expert consultation on oral rehydration salts (ORS) formulation. New York: UNICEF; 2001.
37. Hahn S, Kim Y, Garner P. Reduced osmolarity oral rehydration solution for treating dehydration due to diarrhoea in children: systematic review. BMJ 2001;323:81-5.
38. Olliaro P, Mussano P. Amodiaquine for treating malaria. Issue 4. Oxford: Update Software; 2003. (Cochrane Review.)
39. Dickersin K, Manheimer E, Weiland S, Robinson KA, Lefebvre C, McDonald S. Development of the Cochrane Collaboration's CENTRAL Register of controlled clinical trials. Evaluation and the Health Professions 2002;25:38-64.
40. Isaakidis P, Swingler GH, Pienaar E, Volmink J, Ioannidis JP. Relation between burden of disease and randomised evidence in sub-Saharan Africa: survey of research. BMJ 2002;324:702-7.
41. South Africa HealthInfo. SA HealthInfo, 2004. Available from: http://www.sahealthinfo.org/
42. Dickersin K, Rennie D. Registering clinical trials. JAMA 2003;290:516-23.
43. Volmink J, Swingler G, Siegfried N. Where to practise evidence-based medicine? Lancet 2001;357:724.
44. Swingler GH, Volmink J, Ioannidis JP. Number of published systematic reviews and global burden of disease: database analysis. BMJ 2003;327:1083-4.
45. Davey Smith G, Ebrahim S, Frankel S. How policy informs the evidence. BMJ 2001;322:184-5.
(Submitted: 12 February 2004 Final revised version received: 30 July 2004 Accepted: 5 August 2004)
1 Correspondence should be sent to this author.