Reporting of statistical inference in the Journal of Applied Psychology: Little evidence of reform

Reformers have long argued that misuse of Null Hypothesis Significance Test ing (NHST) is widespread and damaging. The authors analyzed 150 articles fr om the Journal of Applied Psychology (JAP) covering 1940 to 1999. They exam ined statistical reporting practices related to misconceptions about NHST, American Psychological Association (APA) guidelines, and reform recommendat ions. The analysis reveals (a) inconsistency in reporting alpha and p value s, (b) the use of ambiguous language in describing NHST, (c) frequent accep tance of null hypotheses without consideration of power, (d) that power est imates are rarely reported, and (e) that confidence intervals were virtuall y never used. APA guidelines have been followed only selectively. Research methodology reported in JAP has increased greatly in sophistication over 60 years, but inference practices have shown remarkable stability. There is l ittle sign that decades of cogent critiques by reformers had by 1999 led to changes in statistical reporting practices in JAP.