The annealing robust backpropagation (ARBP) learning algorithm

Multilayer feedforward neural networks are often referred to as universal a pproximators, Nevertheless, if the used training data are corrupted by larg e noise, such as outliers, traditional backpropagation learning schemes may not always come up with acceptable performance. Even though various robust learning algorithms have been proposed in the literature, those approaches still suffer from the initialization problem. In those robust learning alg orithms, the so-called M-estimator is employed. For the hi-estimation type of learning algorithms, the loss function is used to play the role in discr iminating against outliers from the majority by degrading the effects of th ose outliers in learning. However, the loss function used in those algorith ms may not correctly discriminate against those outliers, In this paper, th e annealing robust backpropagation learning algorithm (ARBP) that adopts th e annealing concept into the robust learning algorithms is proposed to deal with the problem of modeling under the existence of outliers, The proposed algorithm has been employed in various examples. Those results all demonst rated the superiority over other robust learning algorithms independent of outliers, In the paper not only is the annealing concept adopted into the r obust learning algorithms but also the annealing schedule kit mas found exp erimentally to achieve the best performance among other annealing schedules , where k is a constant and t is the epoch number.