SYSTEMS THAT CAN LEARN FROM EXAMPLES - REPLICA CALCULATION OF UNIFORM-CONVERGENCE BOUNDS FOR PERCEPTRONS

The generalization abilities of neural networks for inferring a rule o n the basis of examples can be characterized by the convergence of the learning error to the generalization error with increasing size of th e training set. Using the replica technique, we calculate the maximum difference between training and generalization error for the ensemble of all perceptrons trained by a teacher perceptron and the maximal gen eralization error for the perceptrons that have a training error equal to zero. The results axe compared with the rigorous bounds provided b y the Vapnik-Chervonenkis theorem.