A comparison of methods to detect publication bias in meta-analysis

Meta-analyses are subject to bias for many of reasons, including publicatio n bias. Asymmetry in a funnel plot of study size against treatment effect i s often used to identify such bias. We compare the performance of three sim ple methods of testing for bias: the rank correlation method; a simple line ar regression of the standardized estimate of treatment effect on the preci sion of the estimate; and a regression of the treatment effect on sample si ze. The tests are applied to simulated meta-analyses in the presence and ab sence of publication bias. Both one-sided and two-sided censoring of studie s based on statistical significance was used. The results indicate that non e of the tests performs consistently well. Test performance varied With the magnitude of the true treatment effect, distribution of study size and whe ther a one- or two-tailed significance test was employed. Overall, the powe r of the tests was low when the number of studies per meta-analysis was clo se to that often observed in practice. Tests that showed the highest power also had type I error rates higher than the nominal level. Based on the emp irical type I error rates, a regression of treatment effect on sample size, weighted by the inverse of the variance of the logit of the pooled proport ion (using the marginal total) is the preferred method. Copyright (C) 2001 John Wiley & Sons, Ltd.