Are randomized clinical trials good for us (in the short term)? Evidence for a "trial effect"

Objective: To assess whether there is evidence that randomized controlled t rials are systematically beneficial, or harmful, for patients. In other wor ds, is there a "trial effect"? If so, to examine whether the evidence sheds light on the likely sources of the difference in outcomes. Methods: System atic review of the literature. Results: We set out in some detail potential sources of a "trial effect" and potential biases. We found only 14 researc h articles (covering more than 21 trials) with relevant primary data. We ex tracted, with difficulty, quantitative data-sets from the articles, and cla ssified these according to likely source of any apparent trial effect. The categories used were: differences in prognosis; superior treatment in the t rial; and "protocol/Hawthorne effect" (benefit from improved routine care w ithin a trial). Analysis: The evidence available is limited in breadth (com ing largely from cancer trials) and quality, as well as quantity. There is weak evidence to suggest that clinical trials have a positive effect on the outcome of participants. This does not appear to depend strongly on the tr ial demonstrating that an experimental treatment is superior. However, bene fit to participants is less evident where scope for a "protocol/Hawthorne e ffect" was apparently limited (because there was no effective routine treat ment or because the comparison group also received protocol care). A form o f bias, arising if clinicians who tend to recruit to trials also tend to be better clinicians, could also explain these results. Conclusion: While the evidence is not conclusive, it is more likely that clinical trials have a positive rather than a negative effect on the outcome of patients. In the l imited data available, the effect seems to be larger in trials where an eff ective treatment already exists and is included in the trial protocol. Reco mmendation: That carefully researched treatment protocols, and monitoring o f outcomes, be used for all patients, not just those in trials. (C) 2001 El sevier Science Inc. All rights reserved.