SMEM algorithm for mixture models

We present a split-and-merge expectation-maximization (SMEM) algorithm to o vercome the local maxima problem in parameter estimation of finite mixture models. In the case of mixture models, local maxima often involve having to o many components of a mixture model in one part of the space and too few i n another, widely separated part of the space. To escape from such configur ations, we repeatedly perform simultaneous split-and-merge operations using a new criterion for efficiently selecting the split-and-merge candidates. We apply the proposed algorithm to the training of gaussian mixtures and mi xtures of factor analyzers using synthetic and real data and show the effec tiveness of using the split-and-merge operations to improve the likelihood of both the training data and of held-out test data. We also show the pract ical usefulness of the proposed algorithm by applying it to image compressi on and pattern recognition problems.