AN ADAPTIVE CONJUGATE-GRADIENT LEARNING ALGORITHM FOR EFFICIENT TRAINING OF NEURAL NETWORKS

An adaptive conjugate gradient learning algorithm has been developed f or training of multilayer feedforward neural networks. The problem of arbitrary trail-and-error selection of the learning and momentum ratio s encountered in the momentum backpropagation algorithm is circumvente d in the new adaptive algorithm. Instead of constant learning and mome ntum ratios, the step length in the inexact line search is adapted dur ing the learning process through a mathematical approach. Thus, the ne w adaptive algorithm provides a more solid mathematical foundation for neural network learning. The algorithm has been implemented in C on a SUN-SPARCstation and applied to two different domains: engineering de sign and image recognition. It is shown that the adaptive neural netwo rks algorithm has superior convergence property compared with the mome ntum backpropagation algorithm.