Asymptotic behavior of linear permutation tests under general alternatives, with application to test selection and study design

Tests based on the permutation of observations are a common and attractive method of comparing two groups of outcomes. in part because they retain pro per test size with minimal assumptions and can have high efficiency toward specific alternatives of interest. In addition, permutation tests may be us ed with discrete or categorical outcomes, for which linear rank tests are n ot designed. Permutation tests are now increasingly used to analyze discret e or continuous responses that themselves are functions of several statisti cs. Examples of such summary statistics include the area under the curve ge nerated by repeated measures of a laboratory marker or an overall composite score from a quality of life study. Here even simple structures for the jo int distribution of the component statistics can lead to complex difference s between the distributions of summary statistics of the comparison groups. Despite their attractive features, surprisingly little is known about the behavior of linear permutation tests when the two groups differ even in sim ple ways. This lack of knowledge Limits an assessment of the relative effic iency of different tests or the planning of the size of a study based on a permutation test. To address these issues, we derive the: asymptotic distri bution of permutation tests under a general contiguous alternative, and the n investigate the implications for test selection and study design for seve ral diverse areas of application. For discrete outcomes, areas of applicati on include permutation tests for ordinal responses and for count data. For continuous outcomes, we explore several applications, including general res ults for location-scale families, a comparison of different data transforma tions, and a comparison to linear rank tests.