NEURODYNAMICS OF LEARNING AND NETWORK PERFORMANCE

A simple dynamic model of a neural network is presented. Using the dyn amic model of a neural network, we improve the performance of a three- layer multilayer perceptron (MLP). The dynamic model of a MLP is used to make fundamental changes in the network optimization strategy. Thes e changes are: Neuron activation functions are used, which reduce the probability of singular Jacobians; Successive regularization is used t o constrain the volume of the weight space being minimized; Boltzmann pruning is used to constrain the dimension of the weight space; and pr ior class probabilities are used to normalize all error calculations, so that statistically significant samples of rare but important classe s can be included without distortion of the error surface. All four of these changes are made in the inner loop of a conjugate gradient opti mization iteration and are intended to simplify the training dynamics of the optimization. On handprinted digits and fingerprint classificat ion problems, these modifications improve error-reject performance by factors between 2 and 4 and reduce network size by 40 to 60%. (C) 1997 SPIE and IS&T.