The pairwise multiple comparison multiplicity problem: An alternative approach to familywise and comparisonwise type I error control

When undertaking many tests of significance, researchers are faced with the problem of how best to control the probability of committing a Type I erro r. The familywise approach deals directly with multiplicity problems by set ting a level of significance for an entire set of related hypotheses; the c omparison approach ignores the issue by setting the rate of error on each i ndividual hypothesis. A new formulation of control, the false discovery rat e, does not provide control as stringent as that of the familywise rate, bu t concomitant with this relaxation in stringency is an increase in sensitiv ity to detect effects relative to the sensitivity of familywise control. Ty pe I error and power rates for 4 relatively powerful and easily computed pa irwise multiple comparison procedures were compared with the false discover y rate procedure for various 1-way layouts by use of test statistics that d o not assume variance homogeneity.