Resolution-based complexity control for gaussian mixture models

In the domain of unsupervised learning, mixtures of gaussians have become a popular tool for statistical modeling. For this class of generative models , we present a complexity control scheme, which provides an effective means for avoiding the problem of overfitting usually encountered with unconstra ined (mixtures of) gaussians in high dimensions. According to some prespeci fied level of resolution as implied by a fixed variance noise model, the sc heme provides an automatic selection of the dimensionalities of some local signal subspaces by maximum likelihood estimation. Together with a resoluti on-based control scheme for adjusting the number of mixture components, we arrive at an incremental model refinement procedure within a common determi nistic annealing framework, which enables an efficient exploration of the m odel space. The advantages of the resolution-based framework are illustrate d by experimental results on synthetic and high-dimensional real-world data .