New fast algorithms for error rate-based stepwise variable selection in discriminant analysis

Variable selection is an important technique for reducing the dimensionalit y in multivariate predictive discriminant analysis and classification. In t he past, direct evaluation of the subsets by means of a classier has been c omputationally too expensive, rendering necessary the use of heuristic meas ures of class separation, such as Wilk's Lambda or the Mahalanobis distance between class means. We present new fast algorithms for stepwise variable selection based on quadratic and linear classifiers with time complexities which, to within a constant, are the same as those applying measures of cla ss separation. Comparing the new algorithms to previous implementations of classifier-based variable selection, we show that dramatic speed-ups are ac hieved.