A. Buhot et Mb. Gordon, PHASE-TRANSITIONS IN OPTIMAL UNSUPERVISED LEARNING, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics, 57(3), 1998, pp. 3326-3333
We determine the optimal performance of learning the orientation of th
e symmetry axis of a set of P = alpha N points that are uniformly dist
ributed in all the directions but one on the N-dimensional space. The
components along the symmetry breaking direction, of unitary vector B,
are sampled from a mixture of two Gaussians of variable separation an
d width. The typical optimal performance is measured through the overl
ap R-opt = B.J, where J* is the optimal guess of the symmetry breakin
g direction. Within this general scenario, the learning curves R-opt(a
lpha) may present first order transitions if the clusters an narrow en
ough. Close to these transitions. high performance states can be obtai
ned through the minimization of the corresponding optimal potential, a
lthough these solutions are metastable, and therefore not learnable, w
ithin the usual Bayesian scenario.