Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses

Articles published in several prominent educational journals were examined to investigate the use of data analysis tools by researchers in four resear ch paradigms: between-subjects univariate designs, between-subjects multiva riate designs, repeated measures designs, and covariance designs. In additi on to examining specific details pertaining to the research design (e.g., s ample size, group size equality/inequality) and methods employed for data a nalysis, the authors also catalogued whether (a) validity assumptions were examined, (b) effect size indices were reported (c) sample sizes were selec ted on the basis of power considerations, and (d) appropriate textbooks and /or articles were cited to communicate the nature of the analyses that were performed. The present analyses imply that researchers rarely verify that validity assumptions are satisfied and that, accordingly, they typically us e analyses that are nonrobust to assumption violations. In addition, resear chers rarely report effect size statistics, nor do they routinely perform p ower analyses to determine sample size requirements. Recommendations are of fered to rectify these shortcomings.