Estimating data transformations in nonlinear mixed effects models

A routine practice in the analysis of repeated measurement data is to repre sent individual responses by a mixed effects model on some transformed scal e. For example, for pharmacokinetic, growth, and other data, both the respo nse and the regression model are typically transformed to achieve approxima te within-individual normality and constant variance on the new scale; howe ver, the choice of transformation is often made subjectively or by default, with adoption of a standard choice such as the log. We propose a mixed eff ects framework based on the transform-both-sides model, where the transform ation is represented by a monotone parametric function and is estimated fro m the data. For this model, we describe a practical fitting strategy based on approximation of the marginal likelihood. Inference is complicated by th e fact that estimation of the transformation requires modification of the u sual standard errors for estimators of fixed effects; however, we show that , under conditions relevant to common applications, this complication is as ymptotically negligible, allowing straightforward implementation via standa rd software.