CONTEXT-SENSITIVE FEATURE-SELECTION FOR LAZY LEARNERS

High sensitivity to irrelevant features is arguably the main shortcomi ng of simple lazy learners. In response to it, many feature selection methods have been proposed, including forward sequential selection (FS S) and backward sequential selection (BSS). Although they often produc e substantial improvements in accuracy, these methods select the same set of relevant features everywhere in the instance space, and thus re present only a partial solution to the problem. In general, some featu res will be relevant only in some parts of the space; deleting them ma y hurt accuracy in those parts, but selecting them will have the same effect in parts where they are irrelevant. This article introduces RC, a new feature selection algorithm that uses a clustering-like approac h to select sets of locally relevant features (i.e., the features it s elects may vary from one instance to another). Experiments in a large number of domains from the UCI repository show that RC almost always i mproves accuracy with respect to FSS and BSS, often with high signific ance, A study using artificial domains confirms the hypothesis that th is difference in performance is due to RC's context sensitivity, and a lso suggests conditions where this sensitivity will and will not be an advantage. Another feature of RC is that it is faster than FSS and BS S, often by an order of magnitude or more.