ON FAULT-TOLERANT TRAINING OF FEEDFORWARD NEURAL NETWORKS

This paper presents an extensive study of fault tolerant training of f eedforward artificial neural networks. We present several versions of a very robust training algorithm and report the results of their simul ations. Our algorithm is shown to outperform all existing training alg orithms in its ability to tolerate different fault types and larger nu mber of hidden unit failures. We show that the generalization ability of the proposed algorithm is substantially better than that of the sta ndard backpropagation algorithm and is comparable with that of other e xisting fault tolerant algorithms. The algorithm is based on the backp ropagation algorithm with built-in measures for extensive fault tolera nt training. A novel concept presented in this paper is that of traini ng the network for fault types beyond the limits of the activation fun ction. We demonstrate that training for such unrealistic fault types e nables the network to be more tolerant to realistic fault types within the limits of the activation function. Further, tradeoffs between tra ining time, enhanced fault tolerance, and generalization properties ar e studied. (C) 1997 Elsevier Science Ltd.