A SYSTEMATIC AND EFFECTIVE SUPERVISED LEARNING MECHANISM-BASED ON JACOBIAN RANK DEFICIENCY

Most neural network applications rely on the fundamental approximation property of feedforward networks. Supervised learning is a means of i mplementing this approximate mapping. In a realistic problem setting, a mechanism is needed to devise this learning process based on availab le data, which encompasses choosing an appropriate set of parameters i n order to avoid overfitting, using an efficient learning algorithm me asured by computation and memory complexities, ensuring the accuracy o f the training procedure as measured by the training error, and testin g and cross-validation for generalization. We develop a comprehensive supervised learning algorithm to address these issues. The algorithm c ombines training and pruning into one procedure by utilizing a common observation of Jacobian rank deficiency in feedforward networks. The a lgorithm not only reduces the training time and overall complexity but also achieves training accuracy and generalization capabilities compa rable to more standard approaches. Extensive simulation results are pr ovided to demonstrate the effectiveness of the algorithm.