DYNAMICS OF ONLINE LEARNING IN RADIAL BASIS FUNCTION NETWORKS

On-line learning is examined for the radial basis function network, an important and practical type of neural network. The evolution of gene ralization error is calculated within a framework which allows the phe nomena of the learning process, such as the specialization of the hidd en units, to be analyzed. The distinct stages of training are elucidat ed, and the role of the learning rate described. The three most import ant stages of training, the symmetric phase, the symmetry-breaking pha se, and the convergence phase, are analyzed in detail; the convergence phase analysis allows derivation of maximal and optimal learning rate s. As well as finding the evolution of the mean system parameters, the variances of these parameters are derived and shown to be typically s mall. Finally, the analytic results an strongly confirmed by simulatio ns.