Controlling test size while gaining the benefits of an internal pilot design

To compensate for a power analysis based on a poor estimate of variance, in ternal pilot designs use some fraction of the planned observations to reest imate error variance and modify the final sample size. Ignoring the randomn ess of the final sample size may bias the final variance estimate and infla te test size. We propose and evaluate three different tests that control te st size for an internal pilot in a general linear univariate model with fix ed predictors and Gaussian errors. Test 1 uses the first sample plus those observations guaranteed to be collected in the second sample for the final variance estimate. Test 2 depends mostly on the second sample for the final variance estimate. Test 3 uses the unadjusted variance estimate and modifi es the critical value to bound test size. We also examine three sample-size modification rules. Only test 2 can control conditional test sire. align w ith a modification rule, and provide simple power calculations. We recommen d it if the minimum second (incremental) sample is at least moderate (perha ps 20). Otherwise, the bounding test appears to have the highest power in s mall samples. Reanalyzing published data highlights some advantages and dis advantages of the various tests.