Initial state training procedure improves dynamic recurrent networks with time-dependent weights

The problem of learning multiple continuous trajectories by means of recurr ent neural networks with (in general) time-varying weights is addressed in this study. The learning process is transformed into an optimal control fra mework where both the weights and the initial network state to be found are treated as controls. For such a task, a new learning algorithm is proposed which is based on a variational formulation of Pontryagin's maximum princi ple. The convergence of this algorithm, under reasonable assumptions, is al so investigated. Numerical examples of learning nontrivial two-class proble ms are presented which demonstrate the efficiency of the approach proposed.