BOOSTING AND OTHER ENSEMBLE METHODS

We compare the performance of three types of neural network-based ense mble techniques to that of a single neural network. The ensemble algor ithms are two versions of boosting and committees of neural networks' trained independently. For each of the four algorithms, we experimenta lly determine the test and training error curves in an optical charact er recognition (OCR) problem as both a function of training set size a nd computational cost using three architectures. We show that a single machine is best for small training set size while for large training set size some version of boosting is best. However, for a given comput ational cost, boosting is always best. Furthermore, we show a surprisi ng result for the original boosting algorithm: namely, that as the tra ining set size increases, the training error decreases until it asympt otes to the test error rate. This has potential implications in the se arch for better training algorithms.