Gs. Howard et al., The proof of the pudding: An illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis, PSYCHOL MET, 5(3), 2000, pp. 315-332
Some methodologists have recently suggested that scientific psychology's ov
erreliance on null hypothesis significance testing (NHST) impedes the progr
ess of the discipline. In response, a number of defenders have maintained t
hat NHST continues to play a vital role in psychological research. Both sid
es of the argument to date have been presented abstractly. The authors take
a different approach to this issue by illustrating the use of NHST along w
ith 2 possible alternatives (metaanalysis as a primary data analysis strate
gy and Bayesian approaches) in a series of 3 studies. Comparing and contras
ting the approaches on actual data brings out the strengths and weaknesses
of each approach. The exercise demonstrates that the approaches are not mut
ually exclusive but instead can be used to complement one another.