Editorials

Maintaining the integrity of the clinical evidence base

Jonathan Quick¹

In this issue of the Bulletin José Esparza highlights the promise of modern clinical research in combating a major killer disease (pp. 1133–1137). During the last decade, at least 15 antiretroviral drugs have come onto the market, bringing longer life and vastly improved quality of life to AIDS patients. In recent months there has been vigorous debate about making those medicines affordable in the South, and about the procurement of ciprofloxacin for the treatment of anthrax in the North. The technical and moral challenge of both ensuring access to existing medicines today and providing incentives for the discovery of new ones for tomorrow has never been more intense.

Clinical trials form the basis of effective research and development, but their reliability is currently imperilled by three major flaws: conflicts of interest on the part of the investigators; inappropriate involvement of research sponsors in their design and management; and publication bias in disseminating their results.

On financial conflicts of interest, Bodenheimer has reviewed studies showing that authors who supported use of certain cardiovascular treatments were significantly more likely to have a financial relationship with the drug's makers than those who did not; that studies funded by the manufacturer of a new therapy were more likely than others to find in favour of that therapy; and that independently funded pharmacoeconomic studies of cancer drugs were seven times more likely than industry-sponsored studies to reach unfavourable conclusions about a product (1).

On inappropriate involvement, recent reviews have documented how industry sponsors influence clinical trials to produce desired results (2). Investigators may have little or no input into trial design, no access to the raw data, and limited participation in data interpretation. This may result in flawed design or invalid practices such as "data dredging" (performing multiple post hoc analyses until some positive results show up). A major cardiovascular trial used eight combinations of drug versus placebo, ensuring a 23% probability of at least one good outcome by chance alone. The share of contract research grew from 40% to 80% during the 1990s, making it easier for commercial sponsors to directly influence clinical trials (3).

Bias in publicizing positive results and underreporting negative ones is the third threat to the clinical evidence base (4). One study of university-industry research centres found that 35% of signed agreements allowed the sponsor to delete information from publication, 53% allowed delay of publication, and 30% allowed both (5). A series of high profile cases have shown how investigators who publish or otherwise communicate results contrary to the wishes of the sponsor face intimidation, efforts to discredit them professionally, and threats of legal action to recover the value of "lost sales" (1).

What can be done? Most clinical research is still conducted to highly exacting standards of objectivity. Yet concern over current trends led the editors of 13 leading medical journals to publish a joint editorial about it in September 2001 (6). Their statement is unequivocal: "[Research] contracts should give the researchers a substantial say in trial design, access to the raw data, responsibility for data analysis and interpretation, and the right to publish." The former editor of the New England Journal of Medicine argues in a separate piece that the editors did not go far enough. "The entire system of clinical investigation is driven by profit," he writes: "we are seeing the corruption of a system of research that used to have high ideals and be clearly in the public interest" (7). Lo and colleagues propose that university- based investigators and researcher staff should be prohibited from holding stock, stock options or decision-making positions in a company that may be affected by the results of their clinical research (8). The World Health Organization is tightening its rules for staff and expert advisers on conflicts of interest, and has established procedures to maintain a "firewall" between commercial interests and normative, regulatory and research decisions.

In a highly competitive world, the pressures may be simply too great for individual researchers, universities, medical journals, or public agencies to stem the tide of commercial influence. Decades ago, when too many clinical trials were putting patients unacceptably at risk, the Helsinki Declaration was drawn up to protect trial subjects. Perhaps it is time for a similar declaration on the rights and obligations of clinical investigators and on how to manage the entire clinical trials evidence base. In addition to the measures proposed by journal editors in September, such a declaration could stipulate: certification by sponsors that specified rules have been kept to ensure the intellectual independence of investigators; inclusion of all details of all trials in a registry which is accessible to third parties such as the Cochrane Collaboration (9); prohibition of legal action by sponsors against investigators except in the case of fraud; and protection of whistle-blowers who report unscientific and unethical research practices (10).

Investment always involves risk, and in clinical research unfavourable results are part of that risk. If clinical trials become a commercial venture in which self-interest overrules public interest and desire overrules science, then the social contract which allows research on human subjects in return for medical advances is broken.

In the last 50 years the world has seen a stunning output of new medicines and vaccines. Continued progress depends critically on the quality of clinical trials. It is in the interest of all stakeholders, including pharmaceutical firms, that the evidence on which clinical and policy decisions are based meets the highest standards of scientific and ethical integrity.

1. Bodenheimer T. Uneasy alliance: Clinical investigators and the pharmaceutical industry. New England Journal of Medicine, 2000, 342: 1539–1544.

2. Bero LA, Rennie D. Influences on the quality of published drug studies. International Journal of Technology Assessment in Health Care, 1996, 12: 209–237.

3. Rettig RA. The industrialization of clinical research. Health Affairs, 2000, 19: 129–146.

4. Chalmers I. Underreporting research is scientific misconduct. JAMA, 1990, 263: 1405–1408.

5. Cohen W, Florida R, Goe WR. University-industry research centers in the United States. Pittsburgh, Carnegie-Mellon University Press, 1994.

6. Davidoff F et al. Sponsorship, authorship, and accountability. Lancet, 2001, 325: 854–856 (published simultaneously in 12 other medical journals).

7. Relman A. Trust me, I'm a scientist. New Scientist, 22 September 2001: 46–47.

8. Lo B, Wolf LE, Berkeley A. Conflict-of-interest policies for investigators in clinical trials. New England Journal of Medicine, 2000, 343: 1616–1620.

9. Maynard A, Bloor K. Regulating the pharmaceutical industry. BMJ, 1997, 315: 200–201.

10. Shuchman M. Consequences of blowing the whistle in medical research. Annals of Internal Medicine, 2000, 132: 1013–1014.

¹Director, Essential Drugs and Medicines Policy, World Health Organization, 1211 Geneva 27, Switzerland.

Ref. No. 01-1602