An integrated instance-based learning algorithm

The basic nearest-neighbor rule generalizes well in many domains but has se veral shortcomings, including inappropriate distance functions, large stora ge requirements, slow execution time, sensitivity to noise, and an inabilit y to adjust its decision boundaries after storing the training data. This p aper proposes methods for overcoming each of these weaknesses and combines the methods into a comprehensive learning system called the Integrated Decr emental Instance-Based Learning Algorithm (IDIBL) that seeks to reduce stor age, improve execution speed, and increase generalization accuracy, when co mpared to the basic nearest neighbor algorithm and other learning models. I DIBL tunes its own parameters using a new measure of fitness that combines confidence and cross-validation accuracy in order to avoid discretization p roblems with more traditional leave-one-out cross-validation. Tn our experi ments IDIBL achieves higher generalization accuracy than other less compreh ensive instance-based learning algorithms, while requiring less than one-fo urth the storage of the nearest neighbor algorithm and improving execution speed by a corresponding factor. In experiments on twenty-one data sets, ID IBL also achieves higher generalization accuracy than that reported for six teen major machine learning and neural network models.