BACK-PROPAGATION THROUGH ADJOINTS FOR THE IDENTIFICATION OF NONLINEARDYNAMIC-SYSTEMS USING RECURRENT NEURAL MODELS

In this paper, back propagation is reinvestigated for an efficient eva luation of the gradient in arbitrary interconnections of recurrent sub systems. It is shown that the error has to be back-propagated through the adjoint model of the system and that the gradient can only be obta ined after a delay. A faster version, accelerated back propagation, th at eliminates this delay, is also developed. Various schemes including the sensitivity method are studied to update the weights of the netwo rk using these gradients. Motivated by the Lyapunov approach and the a djoint model, the predictive back propagation and its variant, targete d back propagation, are proposed. A further refinement, predictive bac k propagation with filtering is then developed, where the states of th e model are also updated. The convergence of this scheme is assured. I t is shown that it is sufficient to back propagate as many time steps as the order of the system for convergence. As a preamble, convergence of on-line batch and sample-wise updates in feedforward models is ana lyzed using the Lyapunov approach.