G. Zhou et J. Si, A SYSTEMATIC AND EFFECTIVE SUPERVISED LEARNING MECHANISM-BASED ON JACOBIAN RANK DEFICIENCY, Neural computation, 10(4), 1998, pp. 1031-1045
Most neural network applications rely on the fundamental approximation
property of feedforward networks. Supervised learning is a means of i
mplementing this approximate mapping. In a realistic problem setting,
a mechanism is needed to devise this learning process based on availab
le data, which encompasses choosing an appropriate set of parameters i
n order to avoid overfitting, using an efficient learning algorithm me
asured by computation and memory complexities, ensuring the accuracy o
f the training procedure as measured by the training error, and testin
g and cross-validation for generalization. We develop a comprehensive
supervised learning algorithm to address these issues. The algorithm c
ombines training and pruning into one procedure by utilizing a common
observation of Jacobian rank deficiency in feedforward networks. The a
lgorithm not only reduces the training time and overall complexity but
also achieves training accuracy and generalization capabilities compa
rable to more standard approaches. Extensive simulation results are pr
ovided to demonstrate the effectiveness of the algorithm.