Information-theoretic algorithm for feature selection

Feature selection is used to improve the efficiency of learning algorithms by finding an optimal subset of features. However, most feature selection t echniques can handle only certain types of data. Additional limitations of existing methods include intensive computational requirements and inability to identify redundant variables. In this paper, we present a novel, inform ation-theoretic algorithm for feature selection, which finds an optimal set of attributes by removing both irrelevant and redundant features. The algo rithm has a polynomial computational complexity and is applicable to datase ts of a mixed nature. The method performance is evaluated on several benchm ark datasets by using a standard classifier (C4.5). (C) 2001 Elsevier Scien ce B.V. All rights reserved.