A REVIEW AND EMPIRICAL-EVALUATION OF FEATURE WEIGHTING METHODS FOR A CLASS OF LAZY LEARNING ALGORITHMS

Many lazy learning algorithms are derivatives of the k-nearest neighbo r (Ic-NN) classifier, which uses a distance function to generate predi ctions from stored instances. Several studies have shown that K-NN's p erformance is highly sensitive to the definition of its distance funct ion. Many K-NN variants have been proposed to reduce this sensitivity by parameterizing the distance function with feature weights. However, these variants have not been categorized nor empirically compared. Th is paper reviews a class of weight-setting methods for lazy learning a lgorithms. We introduce a framework for distinguishing these methods a nd empirically compare them. We observed four trends from our experime nts and conducted further studies to highlight them. Our results sugge st that methods which use performance feedback to assign weight settin gs demonstrated three advantages over other methods: they require less pre-processing, perform better in the presence of interacting feature s, and generally require less training data to learn good settings. We also found that continuous weighting methods tend to outperform featu re selection algorithms for tasks where some features are useful but l ess important than others.