On-line learning in RBF neural networks: a stochastic approach

The on-line learning of Radial Basis Function neural networks (RBFNs) is an alyzed. Our approach makes use of a master equation that describes the dyna mics of the weight space probability density. An approximate solution of th e master equation is obtained in the limit of a small learning rate. In thi s limit, the on line learning dynamics is analyzed and it is shown that, si nce fluctuations are small, dynamics can be well described in terms of evol ution of the mean. This allows us to analyze the learning process of RBFNs in which the number of hidden nodes K is larger than the typically small nu mber of input nodes N. The work represents a complementary analysis of on-l ine RBFNs, with respect to the previous works (Phys. Rev. E 56 (1997a) 907; Neur. Comput. 9 (1997) 1601), in which RBFNs with N >> K have been analyze d. The generalization error equation and the equations of motion of the wei ghts are derived for generic RBF architectures, and numerically integrated in specific cases. Analytical results are then confirmed by numerical simul ations. Unlike the case of large N > K we find that the dynamics in the cas e N < K is not affected by the problems of symmetric phases and subsequent symmetry breaking. (C) 2000 Elsevier Science Ltd. All rights reserved.