Y. Iiguni et al., A REAL-TIME LEARNING ALGORITHM FOR A MULTILAYERED NEURAL NETWORK BASED ON THE EXTENDED KALMAN FILTER, IEEE transactions on signal processing, 40(4), 1992, pp. 959-966
The extended Kalman filter (EKF) is well known as a state estimation m
ethod for a nonlinear system, and can be used as a parameter estimatio
n method by augmenting the state with unknown parameters. A multilayer
ed neural network is a nonlinear system having a layered structure, an
d its learning algorithm is regarded as parameter estimation for such
a nonlinear system. In this paper, a new real-time learning algorithm
for a multilayered neural network is derived from the EKF. Since this
EKF-based learning algorithm approximately gives the minimum variance
estimate of the linkweights, the convergence performance is improved i
n comparison with the backwards error propagation algorithm using the
steepest descent techniques. Furthermore, tuning parameters which cruc
ially govern the convergence properties are not included, which makes
its application easier. Simulation results for the XOR and parity prob
lems are provided.