Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm

Two new implementations of the EM algorithm are proposed for maximum likeli hood fitting of generalized linear mixed models. Both methods use random (i ndependent and identically distributed) sampling to construct Monte Carlo a pproximations at the E-step. One approach involves generating random sample s from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate . The second method uses a multivariate t importance sampling approximation . In many applications the two methods are complementary. Rejection samplin g is more efficient when sample sizes are small, whereas importance samplin g is better with larger sample sizes. Monte Carte approximation using rando m samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Sp ecifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is s wamped by Monte Carlo error. In contrast, techniques for assessing Monte Ca rlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step appr oximations. Three different data sets, including the infamous salamander da ta of McCullagh and Nelder, are used to illustrate the techniques and to co mpare them with the alternatives. The results show that the methods propose d can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the in tractable integrals in the likelihood function are of high dimension.