Kd. Carlson et Fl. Schmidt, Impact of experimental design on effect size: Findings from the research literature on training, J APPL PSYC, 84(6), 1999, pp. 851-862
This article examines whether differences in the equations commonly used to
calculate effect size for single group pretest-posttest (SGPP) designs ver
sus those for control group designs can account for the finding that SGPP d
esigns yield larger mean effect sizes (e.g., M. S. Lipsey & D. B. Wilson, 1
993). It was found that the assumptions of no control group effect and the
equivalence of pretraining and posttraining dependent variable standard dev
iations required for these equations to produce equivalent estimates of eff
ect size were violated for some dependent variable types. Results indicate
that control group effects and inflation in the standard deviation of the p
osttraining dependent variable measure account for most of the observed dif
ference in effect size. The most severe violations occurred when the depend
ent variable was a knowledge assessment. Methods for including data from SG
PP designs in meta-analyses that minimize potential biases are discussed.