Information theory approach to learning of the perceptron rule - art. no. 046106

By recourse to a method based on information theory, we have studied the ge neralization problem in perceptrons. We considered different a priori distr ibutions about the weights of the teacher perceptron. Our approach allows u s to define the information gain from the examples used in the training pro cedure. The information gain can be used to choose a convenient example set for training the perceptron and to select the transfer function of the stu dent perceptron.