A SUCCESSIVE PROJECTION METHOD FOR BINARY PATTERN-RECOGNITION WITH MULTILAYER FEEDFORWARD NEURAL NETWORKS

Error back-propagation (BP) is one of the most popular ideas used in l earning algorithms for multilayer neural networks. In BP algorithms, t here are two types of learning schemes, online learning and batch lear ning. The online BP has been applied to various problems in practice, because of its simplicity of implementation. However; efficient implem entation of the online BP usually requires an ad hoc rule for determin ing the learning rate of the algorithm. In this paper, we propose a ne w learning algorithm called SPM, which is derived from the successive projection method for solving a system of nonlinear inequalities. Alth ough SPM can be regarded as a modification of online sp, the former al gorithm determines the learning rate (step-size) adaptively based on t he output for each input pattern. SPM may also be considered a modific ation of the globally guided back-propagation (GGBP) proposed by Tang and Koehler. Although no theoretical proof of the convergence for SPM is given, some simulation results on pattern classification problems i ndicate that SPM is more effective and robust than the standard online BP and GGBP.