Learning criteria for training neural network classifiers

This paper presents a study of two learning criteria and two approaches to using them for training neural network classifiers, specifically a Multi-La yer Perceptron (MLP) and Radial Basis Function (RBF) networks. The first ap proach, which is a traditional one, relies on the use of two popular learni ng criteria i.e. learning via minimising a Mean Squared Error (MSE) functio n or a Cross Entropy (CE) function. It is shown that the two criteria have different charcteristics in learning speed and outlier effects, and that th is approach does not necessarily result in a minimal classification error. To be suitable for classification tasks, in our second approach an empirica l classification criterion is introduced for the testing process while usin g the MSE or CE function for the training. Experimental results on several benchmarks indicate that the second approach, compared with the first, lead s to an improved generalisation performance, and that the use of the CE fun ction, compared with the MSE function, gives a faster training speed and im proved or equal generalisation performance.