Size is everything - large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects

Variability in patients' response to interventions in pain and other clinic al settings is large. Many explanations such as trial methods, environment or culture have been proposed, but this paper sets out to show that the mai n cause of the variability may be random chance, and that if trials are sma ll their estimate of magnitude of effect may be incorrect, simply because o f the random play of chance. This is highly relevant to the questions of 'H ow large do trials have to be for statistical accuracy?' and 'How large do trials have to be for their results to be clinically valid?' The true under lying control event rate (CER) and experimental event rate (EER) were deter mined from single-dose acute pain analgesic trials in over 5000 patients. T rial group size required to obtain statistically significant and clinically relevant (0.95 probability of number-needed-to-treat within +/- 0.5 of its true value) results were computed using these values. Ten thousand trials using these CER and EER values were simulated using varying group sizes to investigate the variation due to random chance alone. Most common analgesic s have EERs in the range 0.4-0.6 and CER of about 0.19, With such efficacy, to have a 90% chance of obtaining a statistically significant result in th e correct direction requires group sizes in the range 30-60, For clinical r elevance nearly 500 patients are required in each group. Only with an extre mely effective drug (EER > 0.8) will we be reasonably sure of obtaining a c linically relevant NNT with commonly used group sizes of around 40 patients per treatment arm. The simulated trials showed substantial variation in CE R and EER, with the probability of obtaining the correct values improving a s group size increased. We contend that much of the variability in control and experimental event rates is due to random chance alone. Single small tr ials are unlikely to be correct. If we want to be sure of getting correct ( clinically relevant) results in clinical trials we must study more patients , Credible estimates of clinical efficacy are only likely to come from larg e trials or from pooling multiple trials of conventional (small) size. (C) 1998 International Association for the Study of Pain. Published by Elsevier Science B.V.