FURTHER COMMENTS ON RELIABILITY AND POWER OF SIGNIFICANCE TESTS

The controversy about the relationship between reliability and the pow er of significance tests exists because statisticians obtain numerical solutions by varying independently the parameters of the power of sta tistical tests. In contrast, researchers have empirical limitations pl aced on them in varying the same parameters. Reliability and power can legitimately be decoupled by selection of the population from which t o sample (Zimmerman & Williams, 1986), but this is an undependable way to increase power (Humphreys, 1991). Reducing population variance by selection of the sample can be considered a special case of (and a cru de approximation to) the analysis of covariance, which is also a more effective way of controlling individual differences in true scores tha n the use of difference scores. Both the regressed differences and the raw differences are less reliable within treatments than their compon ents, but can have more power in statistical tests. As the reliability of derived scores increases, however, power increases.