LEARNING BY EXAMPLES FROM A NONUNIFORM DISTRIBUTION

We present a general replica calculation for learning from examples ge nerated by a nonuniform pattern distribution with a single symmetry-br eaking orientation. Our results cover the three main learning scenario s: storage of patterns with random classifications by a perceptron, su pervised learning from a teacher, and unsupervised learning. We show t hat for a perceptron the critical storage capacity alpha(c) = 2 is com pletely independent of the pattern distribution provided it is point s ymmetric or provided the classification as +/-1 is unbiased. In a part icular model for supervised learning we find that an ideal (Bayes) stu dent learns most from a few examples if they are easy and from a large number if they are difficult. Learning based on the minimization of a specific class of (quadratic) cost functions is solved completely for all three scenarios.