UNSUPERVISED LEARNING BY EXAMPLES - ONLINE VERSUS OFF-LINE

We study both on-line and off-line unsupervised learning from p random patterns which are uniformly distributed on the N-sphere with the exc eption of a single symmetry breaking orientation B, along which they m ay be arbitrarily distributed. Supervised learning from the same kind of patterns is included as a special case. In the thermodynamic limit N --> infinity with alpha = p/N fixed we calculate the overlap R(alpha )= B . J/\J\\B\ between the unknown ''true'' B and the optimal ''Bayes '' hypothesis J with particular emphasis on the small and large cu asy mptotics and the phenomenon of retarded learning. Finally, we identify a cost function whose minimum reproduces the off-line Bayes overlap.