Randomized, controlled trials, observational studies, and the hierarchy ofresearch designs.

Background: In the hierarchy of research designs, the results of randomized , controlled trials are considered to be evidence of the highest grade, whe reas observational studies are viewed as having less validity because they reportedly overestimate treatment effects. We used published meta-analyses to identify randomized clinical trials and observational studies that exami ned the same clinical topics. We then compared the results of the original reports according to the type of research design. Methods: A search of the Medline data base for articles published in five m ajor medical journals from 1991 to 1995 identified meta-analyses of randomi zed, controlled trials and meta-analyses of either cohort or case-control s tudies that assessed the same intervention. For each of five topics, summar y estimates and 95 percent confidence intervals were calculated on the basi s of data from the individual randomized, controlled trials and the individ ual observational studies. Results: For the five clinical topics and 99 reports evaluated, the average results of the observational studies were remarkably similar to those of t he randomized, controlled trials. For example, analysis of 13 randomized, c ontrolled trials of the effectiveness of bacille Calmette-Guerin vaccine in preventing active tuberculosis yielded a relative risk of 0.49 (95 percent confidence interval, 0.34 to 0.70) among vaccinated patients, as compared with an odds ratio of 0.50 (95 percent confidence interval, 0.39 to 0.65) f rom 10 case-control studies. In addition, the range of the point estimates for the effect of vaccination was wider for the randomized, controlled tria ls (0.20 to 1.56) than for the observational studies (0.17 to 0.84). Conclusions: The results of well-designed observational studies (with eithe r a cohort or a case-control design) do not systematically overestimate the magnitude of the effects of treatment as compared with those in randomized , controlled trials on the same topic. (N Engl J Med 2000;342:1887-92.) (C) 2000, Massachusetts Medical Society.