OPTIMAL LINEAR-COMBINATIONS OF NEURAL NETWORKS

Neural network-based modeling often involves trying multiple networks with different architectures and training parameters in order to achie ve acceptable model accuracy. Typically, one of the trained networks i s chosen as best, while the rest are discarded. Hashem and Schmeiser ( 1995) proposed using optimal linear combinations of a number of traine d neural networks instead of using a single best network. Combining th e trained networks may help integrate the knowledge acquired by the co mponents networks and thus improve model accuracy. In this paper, we e xtend the idea of optimal linear combinations (OLCs) of neural network s and discuss issues related to the generalization ability of the comb ined model. We then present two algorithms for selecting the component networks for the combination to improve the generalization ability of OLCs. Our experimental results demonstrate significant improvements i n model accuracy, as a result of using OLCs, compared to using the app arent best network. (C) 1997 Elsevier Science Ltd.