Improvement of generalization ability for identifying dynamical systems byusing universal learning networks

This paper studies how the generalization ability of models of dynamical sy stems can be improved by taking advantage of the second order derivatives o f the outputs with respect to the external inputs. The proposed method can be regarded as a direct implementation of the well-known regularization tec hnique using the higher order derivatives of the Universal Learning Network s (ULN's). ULNs consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and e ach pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs, in which both the first order derivatives (gradients) and the higher order de rivatives are incorporated. First, the method for computing the second orde r derivatives of ULNs is discussed. Then, a new method for implementing the regularization term is presented. Finally, simulation studies on identific ation of a nonlinear dynamical system with noises are carried out to demons trate the effectiveness of the proposed method. Simulation results show tha t the proposed method can improve the generalization ability of neural netw orks significantly, especially in terms that (1) the robust network can be obtained even when the branches of trained ULNs are destructed, and (2) the obtained performance does not depend on the initial parameter values. (C) 2001 Elsevier Science Ltd. All rights reserved.