MULTIPLE TESTING AND STATISTICAL POWER WITH MODIFIED BONFERRONI PROCEDURES

The difference in statistical power between the original Bonferroni an d five modified Bonferroni procedures that control the overall Type I error rate is examined in the context of a correlation matrix where mu ltiple null hypotheses, H-0: rho(ij) = 0 for all i not equal j, are te sted. Using 50 real correlation matrices reported in educational and p sychological journals, a difference in the number of hypotheses reject ed of less than 4% was observed among the procedures. When simulated d ata were used, very small differences were found among the six procedu res in detecting at least one true relationship, but in detecting all true relationships the power of the modified Bonferroni procedures exc eeded that of the original Bonferroni procedure by at least .18 and by as much as .55 when all null hypotheses were false. The power differe nce decreased as the number of true relationships decreased. Power dif ferences obtained for the average power were of a much smaller magnitu de but still favored the modified Bonferroni procedures. For the five modified Bonferroni procedures, power differences less than .05 were t ypically observed; the Holm procedure had the lowest power, and the Ro m procedure had the highest.