Complex dynamics in learning systems

To describe the dynamics of learning an input-output relation from a set of examples, the evolution of an appropriate choice of macroscopic dynamical variables have to be found. Recent progress in on-line learning only addres ses the often unrealistic case of an infinite training set. For restricted training sets, previous studies have so far been limited to asymptotic dyna mics or simple learning rules. Using the cavity method and diagrammatic tec hniques, we introduce a new framework to model batch learning of restricted sets of examples, widely applicable to any learning cost function, and ful ly taking into account the temporal correlations introduced by the recyclin g of the examples. (C) 2000 Published by Elsevier Science B.V. All rights r eserved.