Parameter expansion for data augmentation

Viewing the observed data of a statistical model as incomplete and augmenti ng its missing parts are useful for clarifying concepts and central to the invention of two well-known statistical algorithms: expectation-maximizatio n (EM) and data augmentation. Recently, Liu, Rubin, and Wu demonstrated tha t expanding the parameter space along with augmenting the missing data is u seful for accelerating iterative computation in an EM algorithm. The main p urpose of this article is to rigorously define a parameter expanded data au gmentation (PX-DA) algorithm and to study its theoretical properties. The P X-DA is a special way of using auxiliary variables to accelerate Gibbs samp ling algorithms and is closely related to reparameterization techniques. We obtain theoretical results concerning the convergence rate of the PX-DB al gorithm and the choice of prior for the expansion parameter. To understand the role of the expansion parameter, we establish a new theory for iterativ e conditional sampling under the transformation group formulation, which ge neralizes the standard Gibbs sampler. Using the new theory, we show that th e PX-DA algorithm with a Haar measure prior (often improper) for the expans ion parameter is always proper and is optimal among a class of such algorit hms including reparameterization.