NEURAL NETWORKS IN COMPUTATIONAL MATERIALS SCIENCE - TRAINING ALGORITHMS

Neural networks can be used in principle in an unbiased way for a mult itude of pattern recognition and interpolation problems within computa tional material science. Reliably finding the weights of large feed-fo rward neural networks with both accuracy and speed is crucial to their use. In this paper, the rate of convergence of numerous optimization techniques that can be used to determine the weights is compared for t wo problems related to the construction of atomistic potentials. Techn iques considered were back propagation (steepest descent), conjugate g radient methods, real-string genetic algorithms, simulated annealing a nd a new swarm search algorithm. For small networks, where only a few optimal solutions exist, we find that conjugate-gradient methods are m ost successful. However, for larger networks where the parameter space to be searched is more complex, a hybrid scheme is most effective; ge netic algorithm or simulated annealing to End a good initial starting set of weights, followed by a conjugate-gradient approach to home in o n a final solution. These hybrid approaches are now our method of choi ce for training large networks.