A SIGNAL-PROCESSING FRAMEWORK BASED ON DYNAMIC NEURAL NETWORKS WITH APPLICATION TO PROBLEMS IN ADAPTATION, FILTERING, AND CLASSIFICATION

We present in this paper a coherent neural network-based framework for solving a variety of difficult signal processing problems. The framew ork relies on the assertion that time-lagged recurrent networks posses s the necessary representational capabilities to act as universal appr oximators of nonlinear dynamical systems. This property applies to mod eling problems posed as system identification, time-series prediction, nonlinear filtering adaptive filtering, and temporal pattern classifi cation. We address the development of models of nonlinear dynamical sy stems, in the form of time-lagged recurrent neural networks, which can be used without further training (i.e., as fixed-weight networks). We employ a weight update procedure based on the extended Kalman filter (EKF); as a solution to the recency effect, which is the tendency for a network to forget earlier learning as it processes new examples, we have developed a technique called multistream training. We demonstrate our training framework by applying it to four problems. First, we sho w, that a single time-lagged recurrent neural network can be trained n ot only to produce excellent one-time-step predictions for two differe nt time;series, but also to be robust to severe errors in the provided input sequence. The second problem involves the modeling of a complex system containing significant process noise, which was shown in [1] t o lead to unstable trained models. We illustrate how multistream train ing may be used to enhance the stability of such models. The remaining two problems are drawn from real-world automotive applications. The f irst of these involves input-output modeling of the dynamic behavior o f a catalyst-sensor system which is exposed to art operating engine's exhaust stream. Finally we consider real-time and continuous detection of engine misfire, which is cast as a dynamic pattern classification problem.