Fitting semiparametric random effects models to large data sets

For large data sets, it can be difficult or impossible to fit models with random effects using standard algorithms due to memory limitations or high computational burdens.In addition, it would be advantageous to use the abundant information to relax assumptions, such as normality of random effects.Motivated by data from an epidemiologic study of childhood growth, we propose a 2-stage method for fitting semiparametric random effects models to longitudinal data with many subjects.In the first stage, we use a multivariate clustering method to identify G<<N groups of subjects whose data have no scientifically important differences, as defined by subject matter experts.Then, in stage 2, group-specific random effects are assumed to come from an unknown distribution, which is assigned a Dirichlet process prior, further clustering the groups from stage 1.We use our approach to model the effects of maternal smoking during pregnancy on growth in 17 518 girls.