Meta-analysis of binary data: which within study variance estimate to use?

We applied a mixed effects model to investigate between- and within-study v ariation in improvement rates of 180 schizophrenia outcome studies. The bet ween-study variation was explained by the fixed study characteristics and a n additional random study effect. Both rate difference and logit models wer e used. For a binary proportion outcome p(i) with sample size n(i) in the i th study, ((p) over cap (i)(1 - (p) over cap (i))n(i))(-1) is the usual est imate of the within-study variance sigma (2)(i) in the logit model, where ( p) over cap (i) is the sample mean of the binary outcome for subjects in st udy i. This estimate can be highly correlated with logit((p) over cap (i)). we used ((p) over bar (i)(1 - (p) over bar )n(i))(-1) as an alternative es timate of sigma (2)(i), where (p) over bar is the weighted mean of (p) over cap (i)'s. We estimated regression coefficients (beta) of the fixed effect s and the variance (tau (2)) of the random study effect using a quasi-likel ihood estimating equations approach. Using the schizophrenia meta-analysis data, we demonstrated how the choice of the estimate of sigma (2)(i) affect s the resulting estimates of beta and tau (2). We also conducted a simulati on study to evaluate the performance of the two estimates of sigma (2)(i) i n different conditions, where the conditions vary by number of studies and study size. Using the schizophrenia meta-analysis data, the estimates of be ta and tau (2) were quite different when different estimates of sigma (2)(i ) were used in the logit model. The simulation study showed that the estima tes of beta and tau (2) were less biased, and the 95 per cent CI coverage w as closer to 95 per cent when the estimate of sigma (2)(i) was ((p) over ba r (1 - (p) over bar )n(i))(-1) rather than ((p) over cap (i)(1 - (p) over c ap (i))n(i))(-1). Finally, we showed that a simple regression analysis is n ot appropriate unless tau (2) is much larger than sigma (2)(i), or a robust variance is used. Copyright (C) 2001 John Wiley & Sons, Ltd.