In the present paper we develop two algorithms, subset-based training (SBT)
and subset-based training and pruning (SBTP), using the fact that the Jaco
bian matrices in sigmoid network training problems are usually rank deficie
nt. The weight vectors are divided into two parts during training, accordin
g to the Jacobian rank sizes. Both SBT and SBTP are trust-region methods. C
ompared with the standard Levenberg-Marquardt (LM) method, these two algori
thms can achieve similar convergence properties as the LM but with fewer me
mory requirements. Furthermore the SBTP combines training and pruning of a
network into one comprehensive procedure. The effectiveness of the two algo
rithms is evaluated using three examples. Comparisons are made with some ex
isting algorithms. Some convergence properties of the two algorithms are gi
ven to qualitatively evaluate the performance of the algorithms. (C) 1999 E
lsevier Science Ltd. All rights reserved.