FINITE-SIZE EFFECTS IN LEARNING AND GENERALIZATION IN LINEAR PERCEPTRONS

Most properties of learning and generalization in linear perceptrons c an be derived from the average response function G. We present a metho d for calculating G using only simple matrix identities and partial di fferential equations. Using this method, we first rederive the known r esult for G in the thermodynamic limit of perceptrons of infinite size N, which has previously been calculated using replica and diagrammati c methods. We also show explicitly that the response function is self- averaging in the thermodynamic limit. Extensions of our method to more general learning scenarios with anisotropic teacher-space priors, inp ut distributions, and weight-decay terms are discussed. Finally, finit e-size effects are considered by calculating the O(1/N) correction to G. We verify the result by computer simulations and discuss the conseq uences for generalization and learning dynamics in linear perceptrons of finite size.