A comparison of observational studies and randomized, controlled trials.

Background: For many years it has been claimed that observational studies f ind stronger treatment effects than randomized, controlled trials. We compa red the results of observational studies with those of randomized, controll ed trials. Methods: We searched the Abridged Index Medicus and Cochrane data bases to identify observational studies reported between 1985 and 1998 that compared two or more treatments or interventions for the same condition. We then se arched the Medline and Cochrane data bases to identify all the randomized, controlled trials and observational studies comparing the same treatments f or these conditions. For each treatment, the magnitudes of the effects in t he various observational studies were combined by the Mantel-Haenszel or we ighted analysis-of-variance procedure and then compared with the combined m agnitude of the effects in the randomized, controlled trials that evaluated the same treatment. Results: There were 136 reports about 19 diverse treatments, such as calciu m-channel-blocker therapy for coronary artery disease, appendectomy, and in terventions for subfertility. In most cases, the estimates of the treatment effects from observational studies and randomized, controlled trials were similar. In only 2 of the 19 analyses of treatment effects did the combined magnitude of the effect in observational studies lie outside the 95 percen t confidence interval for the combined magnitude in the randomized, control led trials. Conclusions: We found little evidence that estimates of treatment effects i n observational studies reported after 1984 are either consistently larger than or qualitatively different from those obtained in randomized, controll ed trials. (N Engl J Med 2000;342:1878-86.) (C)2000, Massachusetts Medical Society.