Fl. Oswald et Jw. Johnson, ON THE ROBUSTNESS, BIAS, AND STABILITY OF STATISTICS FROM METAANALYSIS OF CORRELATION-COEFFICIENTS - SOME INITIAL MONTE-CARLO FINDINGS, Journal of applied psychology, 83(2), 1998, pp. 164-178
Each of several Monte Carlo simulations generated 100 sets of observed
study correlations based on normal, heteroscedastic, or slightly nonl
inear bivariate distributions, with one population correlation coeffic
ient and true variance of 0. A version of J. E. Hunter and F. L. Schmi
dt's (1990b) meta-analysis was applied to each study set. Within simul
ations, <(rho)over cap> was accurate on average. <(sigma)over cap>(2)(
rho) was biased; one would correctly conclude more than half the time
that no moderator effects existed. However, cases of variation in <(rh
o)over cap> and especially in <(sigma)over cap>(2)(rho) indicated that
results from individual meta-analyses could deviate substantially fro
m what was found on average. Findings for these no-moderator cases off
er applied psychologists some guidelines and cautions when drawing con
clusions about true population correlations and true moderator effects
(e.g., situational specificity, validity generalization) from meta-an
alytic results.