Random-effects meta-analyses are not always conservative

It is widely held that random-effects summary effect estimates are more con servative than fixed-effects summaries in epidemiologic meta-analysis. This view is based on the fact that random-effects summaries have higher estima ted variances and, consequently, wider confidence intervals than fixed-effe cts summaries when there is evidence of appreciable heterogeneity among the results from the individual studies. In such instances, however, the rando m-effects point estimates are not invariably closer to the null value nor a re their p values invariably larger than those of fixed-effects summaries. Thus, random-effects summaries are not predictably conservative according t o either of these two connotations of the term. The authors give an example from a meta-analysis of water chlorination and cancer in which the random- effects summaries are less conservative in both of these alternative senses and possibly more biased than the fixed-effects summaries. The discussion of when to use random effects and when to use fixed effects in computing su mmary estimates should be replaced by a discussion of whether summary estim ates should be computed at all when the studies are not methodologically co mparable, when their results are discernibly heterogeneous, or when there i s evidence of publication bias.