Jl. Williams et al., LOW-POWER, TYPE-II ERRORS, AND OTHER STATISTICAL PROBLEMS IN RECENT CARDIOVASCULAR RESEARCH, American journal of physiology. Heart and circulatory physiology, 42(1), 1997, pp. 487-493
Frequently in biomedical literature, measurements are considered ''not
statistically different'' if a statistical test fails to achieve a P
value that is less than or equal to 0.05. This conclusion may be misle
ading because the size of each group is too small or the variability i
s large, and a type II error (false negative) is committed. In this st
udy, we examined the probabilities of detecting a real difference (pow
er) and type II errors in unpaired t-tests in Volumes 246 and 266 of t
he American Journal of Physiology: Heart and Circulatory Physiology. I
n addition, we examined all articles for other statistical errors. The
median power of the t-tests was similar in these volumes (similar to
0.55 and similar to 0.92 to detect a 20% and a 50% change, respectivel
y). In both volumes, similar to 80% of the studies with nonsignificant
unpaired t-tests contained at least one t-test with a type II error p
robability >0.30. Our findings suggest that low power and a high incid
ence of type II errors are common problems in this journal. In additio
n, the presentation of statistics was often vague, t-tests were misuse
d frequently and assumptions for inferential statistics usually were n
ot mentioned or examined.