LEARNING IN 2-LAYERED NETWORKS WITH CORRELATED EXAMPLES

On-line learning in layered perceptrons is often hampered by plateaus in the time dependence of the performance. Studies on backpropagation in networks with a small number of input units have revealed that corr elations between subsequently presented patterns shorten the length of such plateaus. We show how to extend the statistical mechanics framew ork to quantitatively check the effect of correlations on learning in networks with a large number of input units. The surprisingly compact description we obtain makes it possible to derive properties of on-lea rning with correlations directly from studies on on-line learning with out correlations.