STRATEGIES FOR IMPROVING NEURAL-NET GENERALIZATION

We address the problem of training multilayer perceptrons to instantia te a target function. In particular, we explore the accuracy of the tr ained network on a test set of previously unseen patterns - the genera lisation ability of the trained network. We systematically evaluate al ternative strategies designed to improve the generalisation performanc e. The basic idea is to generate a diverse set of networks, each of wh ich is designed to be an implementation of the target function. We the n have a set of trained, alternative versions - a version set. The goa l is to achieve 'useful diversity' within this set, and thus generate potential for improved generalisation performance of the set as a whol e when compared to the performance of any individual version. We defin e this notion of 'useful diversity', we define a metric for it, we exp lore a number of ways of generating it, and we present the results of an empirical study of a number of strategies for exploiting it to achi eve maximum generalisation performance. The strategies encompass stati stical measures as well as a 'selectornet' approach which proves to be particularly promising. The selector net is a form of 'metanet' that operates in conjunction with a version set.