Optimization transfer using surrogate objective functions

The well-known EM algorithm is an optimization transfer algorithm that depe nds on the notion of incomplete or missing data. By invoking convexity argu ments, one can construct a variety of other optimization transfer algorithm s that do not involve missing data. These algorithms all rely on a majorizi ng or minorizing function that serves as a surrogate for the objective func tion. Optimizing the surrogate function drives the objective function in th e correct direction. This article illustrates this general principle by a n umber of specific examples drawn from the statistical literature. Because o ptimization transfer algorithms often exhibit the slow convergence of EM al gorithms, two methods of accelerating optimization transfer are discussed a nd evaluated in the context of specific problems.