PER-EXPERIMENT ERROR RATES - THE HIDDEN COSTS OF SEVERAL MULTIPLE COMPARISON PROCEDURES

For a number of common multiple comparison procedures controlling Type I error at .05 refers to what Ryan called die experiment wise error r ate. This expression represents the chance of making at least one Type I error within a given experiment. This approach ignores, however, th e potential detriments of multiple errors within a single experiment, that is, it fails to acknowledge what Ryan referred to as the error ra te per experiment. The current study uses a computer simulation to eva luate the differences between experiment wise error rates and error ra tes per experiment for a variety of multiple comparison procedures. Fo r pairwise comparisons, Newman-Keuls's and Tukey's tests are examined, as is Dunnett's test for comparisons with a control group and Scheffe 's test for all possible post hoc comparisons. For planned contrasts, a standard Bonferroni and Shaffer's sequentially rejective Bonferroni are simulated using both a nonorthogonal and an orthogonal set.