An examination of the robustness of the empirical Bayes and other approaches for testing main and interaction effects in repeated measures designs

In a previous paper, Boik presented an empirical Bayes (EB) approach to the analysis of repeated measurements. The EB approach is a blend of the conve ntional univariate and multivariate approaches. Specifically, in the EB app roach, the underlying covariance matrix is estimated by a weighted sum of t he univariate and multivariate estimators. In addition to demonstrating tha t his approach controls test size and frequently is more powerful than eith er the E-adjusted univariate or multivariate approaches, Folk showed how co nventional multivariate software can be used to conduct EB analyses. Our in vestigation examined the Type I error properties of the EB approach when it s derivational assumptions were not satisfied as well as when other factors known to affect the conventional tests of significance were varied. For co mparative purposes we also investigated procedures presented by Huynh and b y Keselman, Carriere, and Lix, procedures designed for nonspherical data an d covariance heterogeneity, as well as an adjusted univariate and multivari ate test statistic. Our results indicate that when the response variable is normally distributed and group sizes are equal, the EB approach was robust to violations of its derivational assumptions and therefore is recommended due to the power findings reported by Boik. However, we also found that bo th the EB approach and the adjusted univariate and multivariate procedures were prone to depressed or elevated rates of Type I error when data were no n-normally distributed and covariance matrices and group sizes were either positively or negatively paired with one another. On the other hand, the Hu ynh and Keselman et al. procedures were generally robust to these same pair ings of covariance matrices and group sizes.