STOCHASTIC CONVERGENCE ANALYSIS OF A PARTIALLY ADAPTIVE 2-LAYER PERCEPTRON USING A SYSTEM-IDENTIFICATION MODEL

This paper studies the stationary points of the output layer of a two- layer perceptron which attempts to identify the parameters of a specif ic nonlinear system. The training sequence is modeled as the binary ou tput of the nonlinear system when the input is comprised of an indepen dent sequence of zero-mean Gaussian vectors with independent component s. The training rule for the output layer weights is a modified versio n of Rosenblatt's algorithm. Equations are derived which define the st ationary points of the algorithm for an arbitrary output nonlinearity g(x). For the subsequent analysis, the output nonlinearity is speciali zed to g(x) = sgn(x). The solutions to these equations show that the o nly stationary points occur when the hidden weights of the perceptron are constrained to lie on the plane spanned by the nonlinear system mo del, In this plane, the angles of the perceptron weights and of the no nlinear system model weights satisfy a pair of homogeneous linear equa tions with an infinity of solutions. However, there is a unique soluti on for algorithm convergence (i.e., zero error) such that the paramete rs of the two-layer perceptron must exactly match that of the nonlinea r system.