AN ANALYSIS OF PREMATURE SATURATION IN BACK-PROPAGATION LEARNING

The back propagation (BP) algorithm is widely used for finding optimum weights of multilayer neural networks in many pattern recognition app lications. However, the critical drawbacks of the algorithm are its sl ow learning speed and convergence to local minima. One of the major re asons for these drawbacks is the ''premature saturation '' which is a phenomenon that the error of the neural network stays significantly hi gh constant for some period of time during learning. It is known to be caused by an inappropriate set of initial weights. In this paper, the probability of premature saturation at the beginning epoch of learnin g procedure in the BP algorithm has been derived in terms of the maxim um value of initial weights, the number of nodes in each layer, and th e maximum slope of the sigmoidal activation function; it has been veri fied by the Monte Carlo simulation. Using this result, the premature s aturation can be avoided with proper initial weight settings.