CURRENT TOPICS TEMAS DE ACTUALIDAD
El Registro Latinoamericano de Ensayos Clínicos en Curso (LATINREC)
Ludovic ReveizI, * ; Martha B. DelgadoII; Gerard UrrutiaIII; Zulma OrtizIV; Marcelo Garcia DieguezV; Arturo Martí-CarvajalVI; Erwin CalguaVII; Armando VieyraVIII; Agustín CiapponiIX; Ricardo HidalgoX; Tomás PantojaXI; Luis María SanchezXII; Flora Martínez PecinoXIII; Mario TristanXIV
ICoordinator, Colombian Collaborating Branch of the Iberoamerican Cochrane Network, Instituto de Investigaciones Fundación Universitaria Sánitas (II-FUS), Santa Fe de Bogotá, Colombia. Director, LATINREC
IICoordinator, Colombian Collaborating Branch of the Iberoamerican Cochrane Network, Unidad de Epidemiología Clínica, Pontificia Universidad Javeriana. Editorial Project Manager, LATINREC. Santa Fe de Bogotá, Colombia
IIIIberoamerican Cochrane Center. Servei d'Epidemiologia Clinica i Salut Publica de l'Hospital de la Sant Creu i Sant Pau, Barcelona, Spain
IVCoordinator, Argentine Collaborating Branch of the Iberoamerican Cochrane Network (CIE, Academia Nacional de Medicina), Buenos Aires, Argentina
VCoordinator, Argentine branch of LATINREC. Argentine Collaborating Branch of the Iberoamerican Cochrane Network (CIE, Academia Nacional de Medicina), Buenos Aires, Argentina
VICoordinator, Venezuelan Collaborating Branch of the Iberoamerican Cochrane Network, Universidad de Carabobo, Valencia, Venezuela
VIICoordinator-Director, International Health Central American Institute Foundation (Guatemala), Central American Collaborating Branch of the Iberoamerican Cochrane Network; Universidad de San Carlos de Guatemala, Guatemala City, Guatemala
VIIICoordinator, Mexican Collaborating Branch of the Iberoamerican Cochrane Network. Instituto Nacional de Salud de México, México, D.F., México
IXCoordinator, Argentine Colombian Collaborating Branch of the Iberoamerican Cochrane Network (IECS), Buenos Aires, Argentina
XCoordinator, Ecuatorian Collaborating Branch of the Iberoamerican Cochrane Network, Quito, Ecuador
XICoordinator, Chilean Collaborating Branch of the Iberoamerican Cochrane Network, Unidad Medicina Basada en Evidencia, Pontificia Universidad Católica de Chile, Santiago, Chile
XIICoordinator, Madrid Collaborating branch of the Iberoamerican Cochrane Network, Madrid, Spain
XIIICoordinator, Andalucian Collaborating Branch of the Iberoamerican Cochrane Network. Agencia de Evaluación de Tecnologías Sanitarias de Andalucía (CC-AETSA), Seville, Spain
XIVCoordinator, Costa Rica Collaborating Branch of the Iberoamerican Cochrane Network, San José, Costa Rica
Key words: clinical trial, conflict of interest, ethics committees, publication bias, randomized controlled trials.
Debido a los sesgos que afectan a la publicación de ensayos clínicos y sus resultados, los estudios cuyos resultados son positivos son más fáciles de encontrar que los que tienen resultados sin significación estadística y a ello se debe que los primeros estén sobrerrepresentados. Para contrarrestar este tipo de sesgo se ha propuesto ingresar en un registro toda investigación, desde sus comienzos. No obstante, estos registros se encuentran en distintas fases de evolución, especialmente en países en desarrollo, de tal manera que la Red Cochrane Iberoamericana, parte de la Colaboración Cochrane, ha establecido el Registro Latinoamericano de Ensayos Clínicos en Curso (LATINREC, por Latin American Clinical Trial Registry) con la idea de facilitar el registro de los datos contenidos en el protocolo de todo ensayo clínico que se esté llevando a cabo en un momento dado y poner esa información a la disposición del público. El LATINREC, que viene a respaldar los objetivos de la Organización Mundial de la Salud (OMS), representa un intento por reducir la duplicación de trabajo y el financiamiento poco equitativo de la investigación sobre enfermedades rezagadas al olvido; por evitar que se efectúen investigaciones sobre asuntos de poca cuantía o que se midan resultados poco útiles; y por fomentar las prácticas éticas y la transparencia. Se han detectado algunos obstáculos mayores que hasta ahora han impedido crear un registro único y común de ensayos clínicos. Con el fin de franquearlos, LATINREC será un registro gratuito que permitirá hacer búsquedas y que se ceñirá a la Plataforma Internacional de Registro de Ensayos Clínicos (ICTRP) de la OMS. Además, LATINREC permitirá que los investigadores ingresen en el registro cualquier modificación del protocolo, así como los resultados preliminares. LATINREC ofrecerá grandes ventajas para los consumidores, el gobierno, los profesionales de la salud pública y la industria farmacéutica al incrementar la accesibilidad de la información y la participación en los ensayos clínicos. La disponibilidad de información objetiva acerca de todo ensayo clínico que se inicie ayudará a garantizar que todos tengan libre acceso a los conocimientos generados.
Key words: Ensayos clínicos, conflicto de intereses, comités de ética, sesgo de publicación, ensayos clínicos aleatorios.
Although the need to request researchers and research institutions to register clinical trials at their inception in a publicly and universally accessible register has been considered for years (1), it has only recently become a major issue (24). The community and scientific world have been confronting a complex and chaotic mass of information with conflicting messages about scientific evidence of efficacy and harms. A recent editorial by the International Committee of Medical Journal Editors (ICMJE) supported the need for a comprehensive trial register that meets several criteria (2). Additionally, the World Health Organization (WHO) is promoting an international initiative to develop a metaregister of controlled trials that would offer a onestop search portal fed from existing registers and provide a unique identification number for clinical trials from certified registries that meet standard criteria for the exchange of essential trial data (4).
Some important challenges are to develop a scheme to reduce duplication of work, inequitable funding of research, and neglected diseases; to avoid research on irrelevant issues or the measurement of irrelevant outcomes; and to enhance ethics and transparency. Research should be freely available to the public to avoid publication bias and selective reporting and to improve access to information on benefits and harms.
Despite the intense interest in this topic there is a wide gap between theoretical postulates on trial registration and their implementation. This gap may be due to the absence of universal criteria for registration and differing interests of researchers, the pharmaceutical industry, funders, government, and society. In addition, registers are at different stages of development, particularly those in developing countries.
The purpose of this article is to discuss the concepts of publication bias and prospective registration of clinical trials. Subsequently, we will consider Latin America as a source of ongoing trials and propose The Latin American Ongoing Clinical Trial Register (LATINREC) as a solution to mitigate the problem of publication bias in Latin America and the Caribbean.
PUBLICATION BIAS, SELECTIVE REPORTING, AND MULTIPLE PUBLICATIONS
Publication bias refers to the bias introduced by the selective publication of research results (5). For example, publication bias is known to occur "because studies with results that are statistically significant, interesting, from large well-funded studies, or of higher quality are more likely to be submitted, published, or published more rapidly than work without such characteristics" (5). This tendency is particularly relevant when results appear significant, because negative or near-neutral results are almost never published.
For a range of reasons, including keeping readership high and costs low, journals limit their trial reporting to a fraction of the available trials and findings. People searching for biomedical information may remain oblivious to or unable to access trial results published in journals not indexed in broadly available databases. To address this issue, the Cochrane Collaboration published CENTRAL, a database fed by hand-searching for trials from a broad range of publications from 1948 to the present. Approximately one third of the trials found in CENTRAL were not indexed in MEDLINE, the most widely used open access biomedical bibliographic database (1). Hand searching has helped to reduce publication bias and duplicate research, but is a costly and time-consuming option that remains susceptible to the omission of relevant or essential trials.
Trials may not be published for a variety of reasons, such as methodological errors in design, failure to enroll an adequate number of patients, lack of statistical power, publication of similar results that make the trial irrelevant, the material being the subject of an undergraduate thesis, funding cuts, lack of time for statistical analysis and manuscript preparation, editorial rejection, or because the trial results were unfavorable to the sponsor's product (6, 7). There is good empirical evidence of a significant difference between the results from unpublished and published trials. However, the direction, impact, and extent of publication bias may vary. Publication bias can arise from different components involved in research and its dissemination, and investigators, peer reviewers, editors, funding bodies and research sponsors are frequently found to be involved (79).
Egger and colleagues evaluated the characteristics of "difficult-to-locate trials" and estimated the impact of excluding these trials on the pooled estimates of metaanalyses. They found that unpublished trials contributed, on average, about 18% of the weight in individual metaanalyses. The changes in pooled estimates of metaanalyses when unpublished trials were removed ranged from a reduction of 28% to an increase of 24% in the benefits of the intervention. These authors also found that unpublished trials were less frequently about evaluating drugs, had smaller sample sizes, and were less likely to present statistically significant results (7).
Major ethical issues have been raised by several authors regarding not publishing and not reporting trials results (1, 1012). Dickersin and Rennie have rightly argued that "the distortion of medical evidence, aside from being unethical, actually harms the patients" (1).
Outcome reporting bias refers to the bias introduced in trials where a range of outcome measures may be collected but not all are always reported. This type of bias has been found in several studies of published trials, including trials approved by ethical review boards (13, 14). The specification of primary outcomes and analysis plans in protocols before trial initiation has been suggested to prevent outcome reporting bias (2).
Duplicate publication is publication of an article that overlaps substantially with an article published in print or electronic media (15). Von Elm and colleagues identified six distinct duplication patterns and found that the prevalence of covert duplicate articles (without a cross-reference to the main article) was 5.3%. Systematic reviewers frequently have to deal with this practice, which wastes resources and time and is misleading, as the conclusion of a systematic review may change when the results of a trial are included twice (16).
An additional problem in metaanalyses that speaks in favor of more visible access to science produced in Latin America is the inclusion of research published in languages other than English. This could lead to a particular form of publication bias called language bias, in which randomized controlled trials (RCTs) with positive effects or greater estimates of effect size tend to be published in English-language journals rather than in the original authors' native languages (17, 18). A comparison of the estimated effect of non-English language trials to those published in English found little influence on summary treatment effect, but the importance is difficult to predict for individual systematic reviews (19). Comprehensive searches are strongly suggested as Anglophone databases and journals considerably under-represent the totality of RCTs (2022).
Pharmaceutical industry funding of biomedical research has increased dramatically in the last two decades, leading to frequent conflicts of interest and complex financial relationships between corporate sponsors of research and the investigators who perform clinical trials (23, 24). The CONSORT statement increased the awareness of the need to adequately report the findings of RCTs (25). In addition, a number of industry members have proposed a series of guidelines to promote good publication practices in order to achieve greater transparency (26, 27). However, a substantial number of completed trials are believed to remain unpublished (6); therefore, many safeguards are needed to ensure a transparent trial reporting system.
PROSPECTIVE TRIAL REGISTERS
The prospective, universal registration of all studies at their inception has been proposed as a solution to mitigate publication bias, selective reporting and multiple publication. Proponents include the International Committee of Medical Journal Editors, the World Association of Medical Editors (WAME), and the Cochrane Collaboration (2, 28, 29). During the Twelfth Cochrane Colloquium held in Ottawa in 2004, this issue was debated and the Ottawa group created. The Ottawa Group is an independent grassroots organization of interested stakeholders that has been conducting a worldwide dialogue on trial registration. It developed the Ottawa Statement Part 1 on the principles of trial registration, which was recently published and endorsed by about 150 individuals and groups (3). The Ottawa Statement Part 2, on the principles of implementation of trial registration, is currently open for comments and endorsement at http:// ottawagroup.ohri.ca.
Numerous database registers have been developed and used, most of them based in wealthy countries. Some database registers have implemented rules that allow users to search, register and share information about RCTs with a minimum data set that provides basic information for each clinical trial and a unique identification scheme. The Current Controlled Trials register (www. controlled-trials.com) developed in the UK, and ClinicalTrials.gov (www.clinicaltrials.gov), sponsored by the United States National Library of Medicine, have accumulated practical experience in recent years and are recognized participants in discussions and decision-making processes relating to trial registration (22, 30).
The Ottawa group and WHO are promoting the agreement and international norms and standards for clinical trial registration and reporting (3, 4). WHO proposed the development of an International Clinical Trials Registry Platform (ICTRP), whose primary objectives are "to ensure that all clinical trials are registered and thus publicly declared and identifiable, so as to ensure that for all trials, a minimum set of results will be reported and made publicly available." By registering ongoing and forthcoming clinical trials, researchers are encouraged to provide general information and particular details about the methods of the study protocol and interventions for human participants related to prevention, screening, diagnosis, treatment, health promotion, rehabilitation, or organization and financing of care.
The ICTRP suggested that trial sponsors or researchers should record in a certified and publicly accessible registry a minimum data set prior to participant recruitment, and should then convey this information to ICTRP (Table 1). This platform would produce a unique identification number once it has checked that the trial is unique and criteria are fulfilled, and feed back to the registers. Trialists can apply for a unique identification number at the time of application to their institutional review board, but they would always need to provide a final copy of the protocol as approved by the institutional review board. As defined by the WHO technical consultation on Clinical Trials Registration Standards, "[a]ny research project that prospectively assigns human participants or groups to one or more health-related interventions to evaluate the effects on health outcomes should be registered" (4).
In addition, the Ottawa Group recommends prospective registration of all trials, public disclosure of standard key trial details, registration of any subsequent amendments, and registration and public disclosure of all results once the analyses are completed and verified. However, some stakeholders have asked for delayed public disclosure of some data items (official scientific title of the study, interventions, primary outcome, key secondary outcomes, and target sample size) in instances where they may be considered sensitive for competitive commercial reasons (31, 32). In addition, they consider that early phase trials do not result in information that can inform clinical practice and should not be registered (32). But as Michael Goodyear recently noted, "[t]he recent tragic events in a phase I trial at Northwick Park Hospital in London, and the subsequent lack of available information, highlight the necessity of registration of the details of all trials" (33).
Several majors barriers have been recognized to development of a comprehensive register of clinical trials: the lack of funding appropriation for sustained effort, resistance from different parties including the pharmaceutical industry, lack of mechanisms for enforcement, and lack of awareness of the importance of the problem (1). Also, different registers address the needs of different constituencies (e.g., countries and regions; disease-specific or sponsor-specific aspects) and have different purposes (e.g., administrative, enrollment, scientific database), making it unrealistic to consider a single, unique register. In addition, registers frequently fail to identify and register trials conducted in developing countries or with local funding, partly because of a lack of awareness, language barriers, costs, or simply because trialists remain unaware of the reasons for registering and how to do it, can't afford it, or fail to identify clear benefits of registration (1).
CLINICAL TRIALS IN LATIN AMERICA
In 2003, Latin America and the Caribbean published 35 299 (3.3%) references in Science Citation Index and 12 359 (2.2%) references in MEDLINE, representing an increase in the number of citations of 14% and 30%, respectively, compared to 1999. About 80% of all Latin American citations in these two databases during 2003 came from Brazil, Argentina, or Mexico (34).
Based on a MEDLINE search, 2 149 articles published between January 2004 and December 2004 were identified as dealing with or having authors from the Latin American countries Argentina, Barbados, Belize, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, Dominican Republic, Ecuador, El Salvador, Guatemala, Guyana, Haiti, Honduras, Jamaica, Mexico, Nicaragua, Panama, Paraguay, Peru, Puerto Rico, Trinidad and Tobago, Uruguay or Venezuela. The search was limited to references of RCTs using the PubMed filter recommended by Robinson (35).
The Latin American and Caribbean Center on Health Sciences Information (LILACS) database is available in Latin America and Caribbean countries. This database is a regional collection of the health science-related literature published since 1982. LILACS gathers data from about 670 medical journals and has over 150 000 records. Roughly 104 000 records were found using the clinical trial filter recommended on its website (www.bireme.br/bvs/I/ ibd.htm) (36). However, this filter may have low specificity for detecting RCTs (37, 38)
LATIN AMERICAN ONGOING TRIALS REGISTER (LATINREC)
The Colombian Branch of The Iberoamerican Cochrane Network has been developing the Latin American Ongoing Clinical Trials Register (LATINREC) to collect information on clinical trials undertaken in Latin American countries, make this information available to the public, and provide methodological information and other tools as required by trialists. This project was funded by the International Clinical Epidemiology Network (INCLEN) in 2003, and its website is www.latin rec.org.
LATINREC will be a freely available and searchable register. To register a study, trialists will submit information including the basic data required by the ICTRP and will receive a WHO-assigned unique identification number. In addition, LATINREC will allow trialists to include subsequent protocol amendments and preliminary results.
To populate the register, LATINREC will work with government entities, research centers, universities, companies producing and developing medicines, diagnostic tests and other healthcare interventions, foundations, and funding agencies. A search will be performed periodically on MEDLINE, the Cochrane Library and LILACS to identify researchers possibly involved in clinical trials in Latin America, and these researchers will be contacted. LATINREC is also considering e-mail contact with known researchers to learn about other ongoing trials (7). Protocols can be completed in the trialist's native language (Spanish, French, English, or Portuguese) but information will be translated to English to apply for a unique identifier.
The Cochrane Collaboration (www.cochrane. org) is committed to providing the most reliable evidence of the effectiveness of health care through systematic reviews of RCTs. The Collaboration recognizes the importance of prospectively registering trials to ensure that the evidence assessed is valid and comprehensive, and to minimize the risk of publication and reporting bias. The Iberoamerican Cochrane Center is based at the Hospital de la Santa Creu i Sant Pau in Barcelona, Spain. This Center has evolved into a network and has assisted twelve Collaborative Centers in ten Latin American countries (Argentina, Chile, Colombia, Costa Rica, Cuba, Ecuador, Guatemala, Mexico, Peru and Venezuela), all of which participate in LATINREC.
The Iberoamerican Cochrane Network of the Cochrane Collaboration has set up the Latin American Ongoing Clinical Trials Register (LATINREC) to facilitate sharing basic information for ongoing clinical trials. This register will comply with the Ottawa Statement criteria and WHO's ICTRP. The register will be launched in June 2006 (visit www. latinrec.org), registration will be free of charge, and the contents will be freely available to the public. It should be noted that pharmaceutical industry regulations sometimes restrict public access to some information to ensure intellectual property protection and avoid the financial implications of making proprietary knowledge available to competitors. However, registration of clinical trials is expected to help enhance the general public's trust in medical science and the pharmaceutical industry.
Acknowledgment. The authors wish to acknowledge the valuable contributions made by Luis Gabriel Cuervo towards enriching and improving this manuscript.
1. Dickersin K, Rennie D. Registering clinical trials. JAMA. 2003;290:51623.
2. DeAngelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351: 12501.
3. Krleza-Jeric K, Chan AW, Dickersin K, Sim I, Grimshaw J, Gluud C (for the Ottawa group). Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ. 2005;330(7497):9568.
4. World Health Organization. The International Clinical Trials Registry Platform (ICTRP). Available from: http:// www.who.int/ictrp/en/. Accessed 23 March 2006.
5. Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR. Empirical assessment of effect of publication bias on meta-analyses. BMJ. 2000;320:15747.
6. Reveiz L, Cardona AF, Ospina EG. Using e-mail for identifying Randomized Controlled Trials that are difficult to locate for systematic reviews. J Clin Epidemiol. (in press).
7. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7:176.
8. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. Publication and related biases. Health Technol Assess. 2000;4: 1115.
9. McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356:122831.
10. Antes G. Registering clinical trial is necessary for ethical, scientific and economic reasons. Bull World Health Organ. 2004; 82:321.
11. Antes G, Chalmers I. Under-reporting of clinical trials is unethical. Lancet. 2003, 361:9789.
12. Reveiz L, Cardona AF, Ospina EG. Clinical trial registration. N Engl J Med. 2005 13;352:1989.
13. Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials by the Canadian Institutes of Health Research. CMAJ. 2004; 171:73540.
14. Chan AW, Haahr MT, Hróbjartsson A, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:245765.
15. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Available from: http:// www.icmje.org. Accessed 5 April 5 2006.
16. von Elm E, Poglia G, Walder B, Tramer MR. Different patterns of duplicate publication: an analysis of articles used in systematic reviews. JAMA. 2004;25; 291(8):97480.
17. Egger M, Zellweger-Zahner T, Antes G. Randomised trials in German-language journals. Lancet. 1996;347:10478.
18. Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet. 1997;350:3269.
19. Juni P, Holenstein F, Sterne J, Barlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol. 2002;31:11523
20. Bereczki D, Gesztelyi G: A. A Hungarian example for handsearching specialized national healthcare journals of small countries for controlled trials. Is it worth the trouble? Health Libr Rev. 2000;17:1447.
21. Rafalski V, Andreeva I, Ryabkova E. Searching for randomized clinical trials and controlled clinical trials in Russian medical journals: Handsearching vs. Medline. Available from: http://www. antibiotic.ru/en/cc/tudor.shtml. Accessed 3 April 2006.
22. Marti J, Bonfil X, Urrutia G, Lacalle J, Bravo R. Identicación y descripción de los ensayos clínicos publicados en re-vistas españolas de medicina general e interna durante el periodo 19711995. [The identification and description of clinical trials published in Spanish journals of general and internal medicine during the period of 19711995]. Med Clin (Barc) 1999;112(Suppl 1):2834.
23. Moses H, Martin JB. Academic relationships with industry: a new model for biomedical research. JAMA. 2001;285: 9335.
24. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):45465.
25. Begg C, Cho M, Esatwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:6379.
26. Publication standards for clinical research sponsored by the pharmaceutical industry. Rev Panam Salud Publica. 2003;14:626.
27. Wager E. Special article. Good publication practice for pharmaceutical companies: where are we now? Medscape Gen Med, Posted 04/18/2005. Available from: http://www.medscape.com/ view article/501752. Accessed 3 April 2006.
28. World Association of Medical Editors. WAME policy statement: the registration of clinical trials 2005. Available from: www.wame.org/wamestmt.htm# trialreg. Accessed 3 April 2006.
29. Cochrane Collaboration statement on registering clinical trials prospectively. 2004. Available from: http://www. cochrane.org/news/articles/2004.07.26. htm. Accessed 3 April 2006.
30. Tonks A. A clinical trials register for Europe. BMJ. 2002;326: 13145.
31. Grimes L (Campbell Alliance). The era of clinical trial registries, a white paper. Available from: http://www. campbell alliance.com/articles/Final_Clinical_ Trial_Registries_3-23-05.pdf. Accessed 23 March 2006.
32. Abbott response to World Health Organization International Clinical Trials Registry Platform (ICTRP) request for open comments on disclosure timing. Posted 31 March 2006. Available from: http://www.who.int/ictrp/005-Abbott_31March06.pdf. Accessed 2 April 2006.
33. Goodyear M. Closing the gaps: moving closer to a collaborative culture. Comments on disclosure timing: balancing increased transparency and competitive advantage. Submission to WHO International Clinical Trials Registry Platform March 2006. Available from: http:// www.who.int/ictrp/007-Michael_ Goodyear_31March06.pdf. Accessed 2 April 2006.
34. Red de indicadores de ciencia y tecnología (RICYT). Datos de producción científica en bases de datos internacionales. Available from: http://www.ricyt.org/ interior/interior.asp?Nivel1=3&Nivel2 =1&Idioma=#21. Accessed 25 September 2005.
35. Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. Int J Epidemiol. 2002;31(1):1503.
36. Virtual Health Library. LILACSLatin American and Caribbean Health Sciences. Available from: http://bases. bireme.br/cgi-bin/wxislind.exe/ iah/online/?IsisScript=iah/iah.xis& base=LILACS&lang=i. Accessed 25 September 2005.
37. Garcia Dieguez M, Esandi ME, Khoury M, Nishishinia MB, Arribas A Comparison of handsearching with electronic databases of clinical trials published in Argentinean indexed journals (abstract). 2003. Cochrane Library issue 1 2006.
38. Clark OA, Castro AA. Searching the Literatura Latino Americana e do Caribe em Ciencias da Saude (LILACS) database improves systematic reviews. Int J Epidemiol. 2002 Feb;31(1):1124.
* Send correspondence and reprint requests to: Ludovic Reveiz, Edificio de Consultorios Clínica Reina Sofía Diag 127 A # 348 cons 221, Santa Fe de Bogotá, Colombia; fax: (57) 1 6159435; e-mail: email@example.com