ONLINE TRAINING OF RECURRENT NEURAL NETWORKS WITH CONTINUOUS TOPOLOGYADAPTATION

This paper presents a novel on-line procedure for training dynamic neu ral networks with input-output recurrences whose topology is continuou sly adjusted to the complexity of the target system dynamics. The latt er is accomplished by changing the number of the elements of the netwo rk hidden layer whenever the existing topology cannot capture the dyna mics presented by the new data. The training mechanism developed in th is work is based on the suitably altered extended Kalman filter (EKF) algorithm which is simultaneously used for the network parameter adjus tment and for its state estimation. The network itself consists of a s ingle hidden layer with a Gaussian radial basis functions (GRBF's) and of a linear output layer. The choice of the GRBF is induced by the re quirements of the online learning. The latter implies the network arch itecture which permits only local influence of the new data point in o rder not to forget the previously learned dynamics. The continuous top ology adaptation is implemented in our algorithm to avoid memory and c omputational problems of using a regular grid of GRBF's which covers t he network input space. Furthermore, we show that the resulting parame ter increase can be handled ''smoothly'' without interfering with the already acquired information. In the case when the target system dynam ics are changing over time, we show that a suitable forgetting factor can he used to ''unlearn'' the no-longer-relevant dynamics. The qualit y of the presented recurrent network training algorithm is demonstrate d on the identification of nonlinear dynamic systems.