Fixed effects vs. random effects meta-analysis models: Implications for cumulative research knowledge

Research conclusions in the social sciences are increasingly based on meta- analysis, making questions of the accuracy of meta-analysis critical to the integrity of the base of cumulative knowledge. Both fixed effects (FE) and random effects (RE) meta-analysis models have been used widely in publishe d meta-analyses. This article shows that FE models typically manifest a sub stantial Type I bias in significance tests for mean effect sizes and for mo derator variables (interactions), while RE models do not. Likewise, FE mode ls, but not RE models, yield confidence intervals for mean effect sizes tha t are narrower than their nominal width, thereby overstating the degree of precision in meta-analysis findings. This article demonstrates analytically that these biases in FE procedures are large enough to create serious dist ortions in conclusions about cumulative knowledge in the research literatur e. We therefore recommend that RE methods routinely be employed in meta-ana lysis in preference to FE methods.