VARIABLE SELECTION FOR NEURAL NETWORKS IN MULTIVARIATE CALIBRATION

The problem of variable selection for neural network modeling is discu ssed in this paper. Two methods that gave the best results in a previo us comparative study are presented. One of these methods is a modified version of the Hinton diagrams, the other method is based on saliency estimation and is part of the Optimal Brain Surgeon algorithm for pru ning unimportant weights in a neural network. We also propose two new methods, based on the estimation of the contribution of each input var iable to the variance of the predicted response. These new methods are designed for situations where input variables are orthogonal, such as the PC scores often used in multivariate calibration. The four method s are tested on synthetic examples, and on real industrial data sets f or multivariate calibration. The main characteristics of each method a re discussed. In particular, we underline the strong theoretical and e xperimental limitations of methods like the modified Hinton diagrams, based on weight magnitude estimation. We also demonstrate that althoug h the saliency estimation approach is theoretically more stringent, it gives unstable results on repeated trials. The advantage of the two v ariance-based approaches is that they are much less dependent on the i nitial weight randomization than the two other methods, and therefore, the results they produce are more stable and reliable. (C) 1998 Elsev ier Science B.V. All rights reserved.