STATISTICAL-MECHANICS OF THE MAXIMUM-LIKELIHOOD DENSITY-ESTIMATION

Estimating the density of data generated by Gaussian mixtures, using t he maximum-likelihood criterion, is investigated. Solving the statisti cal mechanics of this problem we evaluate the quality of the estimatio n as a function of the number of data points, P = alpha N, N being the dimensionality of the points, in the limit of large N. Below a critic al value of alpha, the estimated density consists of Gaussian centers that have zero overlap with the structure of the true mixture. We show numerically that estimating the centers by slowly reducing the estima ted Gaussian width yields a good agreement with the theory even in the presence of many local minima.