Testing treatment effects in repeated measures designs: Trimmed means and bootstrapping

Non-normality and covariance heterogeneity between groups affect the validi ty of the traditional repeated measures methods of analysis, particularly w hen group sizes are unequal. A non-pooled Welch-type statistic (WJ) and the Huynh Improved General Approximation (ICA) test generally have been found to be effective in controlling rates of Type I error in unbalanced non-sphe rical repeated measures designs even though data are non-normal in form and covariance matrices are heterogeneous. However, under some conditions of d eparture from multisample sphericity and multivariate normality their rates of Type I error have been found to be elevated. Westfall and Young's resul ts suggest that Type I error control could be improved by combining bootstr ap methods with methods based on trimmed means. Accordingly, in our investi gation we examined four methods for testing for main and interaction effect s in a between- by within-subjects repeated measures design: (a) the IGA an d WJ tests with least squares estimators based on theoretically determined critical values; (b) the IGA and WJ tests with least squares estimators bas ed on empirically determined critical values; (c) the IGA and WJ tests with robust estimators based on theoretically determined critical values; and ( d) the IGA and WJ tests with robust estimators based on empirically determi ned critical values. We found that the IGA tests were always robust to assu mption violations whether based on least squares or robust estimators or wh ether critical values were obtained through theoretical or empirical method s. The WJ procedure, however, occasionally resulted in liberal rates of err or when based on least squares estimators but always proved robust when app lied with robust estimators. Neither approach particularly benefited from a dopting bootstrapped critical values. Recommendations are provided to resea rchers regarding when each approach is best.