ESTIMATING MLP GENERALIZATION ABILITY WITHOUT A TEST SET USING FAST, APPROXIMATE LEAVE-ONE-OUT CROSS-VALIDATION

When using MLP regression models, some method for estimating the gener alisation ability is required to identify badly over- and under-fitted models. If data is limited, it may be impossible to spare sufficient data for a test set, and leave-one-out cross-validation may be conside red as an alternative method for estimating generalisation ability. Ho wever, this method is very computer intensive, and we suggest a faster , approximate version suitable for use with the MLP. This approximate method is tested using an artificial test problem, and is then applied to a real modelling problem from the paper-making industry. It is sho wn that the basic method appears to work quite well, but that the appr oximation may be poor under certain conditions. These conditions and p ossible means of improving the approximation are discussed in some det ail.