Sample size calculation for clinical trials: the impact of clinician beliefs

The UK Medical Research Council (MRC) randomized trial of gastric surgery, ST01, compared conventional (Dl)with radical (D2) surgery. Sample size esti mation was based upon the consensus opinion of the surgical members of the design team, which suggested that a change in 5-year survival from 20% (D1) to 34% (D2) could be realistic and medically important. On the basis of th ese survival rates, the sample size for the trial was 400 patients, However , this trial was exceptional in the way that a survey of surgeons' opinions was made at the start of the trial, in 1986, and again before results were analysed but after termination of the trial in 1994. At the initial survey , the three surgeons from the trial steering committee and 23 other surgeon s experienced in treating gastric carcinoma were given detailed questionnai res. They were asked about the expected survival rate in the D1 group, anti cipated difference in survival from D2 surgery, and what difference would b e medically important and influence future treatment of patients. The conse nsus opinion of those surveyed was that there might be a survival improveme nt of 9.4%. In 1994, prior to closure of the trial, and before any survival information was disclosed, the survey was repeated with 21 of the original 26 surgeons. At this second survey, the opinion of the trial steering comm ittee was that 9.5% difference was more realistic. This was in accord with the opinion of the larger group, which remained little changed since 1986. The baseline 5-year D1 survival was thought likely to be about 32%, which c orresponded closely to the actual survival of recruited patients. Revised s ample size calculations suggested that, on the basis of these more recent o pinions, between 800 and 1200 patients would have been required. Both surve ys assessed the level of treatment benefit that was deemed to be sufficient for causing surgeons to change their practice. This showed that the 13% di fference in survival used as the study target was clinically relevant, but also indicated that many clinicians would remain unwilling to change their practice if the difference is only 9.5%. The experience of this carefully d esigned trial illustrates the problems of designing long-term, randomized t rials. It raises interesting questions about the common practice of basing sample size estimates upon the beliefs of a trial design committee that may include a number of enthusiasts for the trial treatment. If their opinion of anticipated effect sizes drives the design of the trial, rather than the opinion of a larger community of experts that includes sceptics as well as enthusiasts, there is likely to be a serious miscalculation of sample size requirements. (C) 2000 Cancer Research Campaign.