Permutation tests for univariate or multivariate analysis of variance and regression

The most appropriate strategy to be used to create a permutation distributi on for tests of individual terms in complex experimental designs is current ly unclear. There are often many possibilities, including restricted permut ation or permutation of some form of residuals. This paper provides a summa ry of recent empirical and theoretical results concerning available methods and gives recommendations for their use in univariate and multivariate app lications. The focus of the paper is on complex designs in analysis of vari ance and multiple regression (i.e., linear models). The assumption of excha ngeability required for a permutation test is assured by random allocation of treatments to units in experimental work. For observational data, exchan geability is tantamount to the assumption of independent and identically di stributed errors under a null hypothesis. For partial regression, the metho d of permutation of residuals under a reduced model has been shown to provi de the best test. For analysis of variance, one must first identify exchang eable units by considering expected mean squares. Then, one may generally p roduce either (i) an exact test by restricting permutations or (ii) an appr oximate test by permuting raw data or some form of residuals. The latter ca n provide a more powerful test in many situations.