LEARNING ALGORITHM THAT GIVES THE BAYES GENERALIZATION LIMIT FOR PERCEPTRONS

A variational approach to the study of learning a linearly separable r ule by a single-layer perceptron leads to a gradient descent learning algorithm with exactly tile same generalization ability as the Bayes l imit calculated by Opper and Haussler [Phys. Rev. Lett. 66, 2677 (1991 )]. This is done by finding, through the Gardner-Derrida replica metho d, the student-teacher overlap R as a functional of the algorithm cost function and maximizing this functional The resulting cost function i s closely related to the optimal cost function derived for on-line lea rning.