On-line learning from restricted training sets in multilayer neural networks

We analyse the dynamics of on-line learning in multilayer neural networks w here training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is based o n monitoring a set of macroscopic variables from which the training and gen eralisation errors can be calculated. A closed set of dynamical equations i s derived using the dynamical replica method and is solved numerically. The theoretical results are consistent with those obtained by computer simulat ions.