STATISTICAL POWER, SAMPLE-SIZE, AND THEIR REPORTING IN RANDOMIZED CONTROLLED TRIALS

Objective.-To describe the pattern over time in the level of statistic al power and the reporting of sample size calculations in published ra ndomized controlled trials (RCTs) with negative results. Design.-Our s tudy was a descriptive survey. Power to detect 25% and 50% relative di fferences was calculated for the subset of trials with negative result s in which a simple two-group parallel design was used. Criteria were developed both to classify trial results as positive or negative and t o identify the primary outcomes. Power calculations were based on resu lts from the primary outcomes reported in the trials. Population.-We r eviewed all 383 RCTs published in JAMA, Lancet, and the New England Jo urnal of Medicine in 1975, 1980, 1985, and 1990. Results.-Twenty-seven percent of the 383 RCTs (n=102) were classified as having negative re sults. The number of published RCTs more than doubled from 1975 to 199 0, with the proportion of trials with negative results remaining fairl y stable. Of the simple two-group parallel design trials having negati ve results with dichotomous or continuous primary outcomes (n=70), onl y 16% and 36% had sufficient statistical power (80%) to detect a 25% o r 50% relative difference, respectively. These percentages did not con sistently increase overtime. Overall, only 32% of the trials with nega tive results reported sample size calculations, but the percentage doi ng so has improved over time from 0% in 1975 to 43% in 1990. Only 20 o f the 102 reports made any statement related to the clinical significa nce of the observed differences. Conclusions.-Most trials with negativ e results did not have large enough sample sizes to detect a 25% or a 50% relative difference. This result has not changed over time. Few tr ials discussed whether the observed differences were clinically import ant. There are important reasons to change this practice. The reportin g of statistical power and sample size also needs to be improved.