FAST CONVERGENT GENETIC-TYPE SEARCH FOR MULTILAYERED NETWORK

The classical supervised learning algorithms for optimizing multi-laye red feedforward neural networks, such as the original back-propagation algorithm, suffer from several weaknesses. First, they have the possi bility of being trapped at local minima during learning, which may lea d to failure in finding the global optimal solution. Second, the conve rgence rate is typically too slow even if the learning can be achieved . This paper introduces a new learning algorithm which employs a genet ic-type search during the learning phase of back-propagation algorithm so that the above problems can be overcome. The basic idea is to evol ve the network weights in a controlled manner so as to jump to the reg ions of smaller mean squared error whenever the back-propagation stops at a local minimum. By this, the local minima can always be escaped a nd a much faster learning with global optimal solution can be achieved . A mathematical framework on the weight evolution of the new algorith m is also presented in this paper, which gives a careful analysis on t he requirements of weight evolution (or perturbation) during learning in order to achieve a better error performance in the weights between different hidden layers. Simulation results on three typical problems including XOR, 3-bit parity and the counting problem are described to illustrate the fast learning behaviour and the global search capabilit y of the new algorithm in improving the performance of back-propagated network.