Empirical assessment of effect of publication bias on meta-analyses

Objective To assess the effect of publication bias on the results and concl usions of systematic reviews and meta-analyses. Design Analysis of published meta-analyses by trim and fill method, Studies 48 reviews in Cochrane Database of Systematic Reviews that consider ed a binary endpoint and contained 10 or more individual studies. Main outcome measures Number of reviews with missing studies and effect on conclusions of meta-analyses. Results The trim and fill fixed effects analysis method estimated that 26 ( 54%) of reviews had missing studies and in 10 the number missing was signif icant. The corresponding figures with a random effects model were 23 (48%) and eight. In four cases, statistical inferences regarding the effect of th e intervention were changed after the overall estimate for publication bias was adjusted for. Conclusions Publication or related biases were common within the sample of meta-analyses assessed. In most cases these biases did not affect the concl usions, Nevertheless, researchers should check routinely whether conclusion s of systematic reviews are robust to possible non-random selection mechani sms.