HOMOGENEITY TESTS IN METAANALYSIS - A MONTE-CARLO COMPARISON OF STATISTICAL POWER AND TYPE-I ERROR

The statistical power and Type I error rate of several homogeneity tes ts, usually applied in meta-analysis, are compared using Monte Carlo s imulation: (1) The chi-square test applied to standardized mean differ ences, correlation coefficients, and Fisher's r-to-Z transformations, and (2) S&H-75 (and 90 percent) procedure applied to standardized mean differences and correlation coefficients. Chi-square tests adjusted c orrectly Type I error rates to the nominal significance level while th e S&H procedures showed higher rates; consequently, the S&H procedures presented greater statistical power. In all conditions, the statistic al power was very low, particularly when the sample had few studies, s mall sample sizes, and presented short differences between the paramet ric effect sizes. Finally, the criteria for selecting homogeneity test s are discussed.