WEIGHT SPACE STRUCTURE AND GENERALIZATION IN THE REVERSED-WEDGE PERCEPTRON

The generalization ability of the reversed-wedge perceptron serving as a toy model for multilayer neural networks is investigated. We analys e the decomposition of the version space into disjoint cells belonging to different internal representations defined by the signs of the ali gning fields. The version space is characterized by the number and siz e of these cells and their typical overlap with the teacher network. F or a small training set the system is unable to detect the structure o f the patterns induced by the teacher. Accordingly it performs as if s toring random input-output patterns with very low generalization abili ty and a large misfit in the internal representation. With increasing training set size, cells with large misfit are eliminated at a much hi gher rate than those with internal representation similar to that of t he teacher. This results eventually in the discontinuous phase transit ion to good generalization typical for multilayer neural networks.