ADVANCED NEURAL-NETWORK TRAINING ALGORITHM WITH REDUCED COMPLEXITY BASED ON JACOBIAN DEFICIENCY

In this paper we introduce an advanced supervised training method for neural networks. It is based on Jacobian rank deficiency and it is for mulated, in some sense, in the spirit of the Gauss-Newton algorithm. T he Levenberg-Marquardt algorithm, as a modified Gauss-Newton, has been used successfully in solving nonlinear least squares problems includi ng neural-network training. It outperforms (in terms of training accur acy, convergence properties, overall training time, etc.) the basic ba ckpropagation and its variations with variable learning rate significa ntly, however, with higher computation and memory complexities within each iteration. The new method developed in this paper is aiming at im proving convergence properties, while reducing the memory and computat ion complexities in supervised training of neural networks. Extensive simulation results are provided to demonstrate the superior performanc e of the new algorithm over the Levenberg-Marquardt algorithm.