USING APPRENTICESHIP TECHNIQUES TO GUIDE CONSTRUCTIVE INDUCTION

Constructive induction is a means of improving classification accuracy in difficult domains by transforming a difficult domain into a form a menable to standard induction techniques by constructing new features. When performing constructive induction, though, a learning system fac es a combinatorial explosion of potential features to construct, but o nly a small fraction of these will prove to be useful. The challenge l ies in identifying enough of these useful constructed features to achi eve sufficient accuracy while examining as little as possible of the s pace of potential constructed features. This paper presents how appren ticeship techniques (Mitchell et al., 1985; Hall, 1988; Wilkins, 1988; Tecuci & Kodratoff, 1990) can be used to guide the feature constructi on process by focusing attention on weak areas of a learned knowledge base. The method used is to run a splitting algorithm (such as CART, P LS1, or C4.5) to build a knowledge base, employ apprenticeship techniq ues to detect and localize knowledge base deficiencies, use this infor mation to construct new features, and then repeat the cycle as necessa ry (i.e. until a desired accuracy is reached). We show how this method improves accuracy on a range of classification problems and discuss w hy the combination of apprenticeship as a knowledge acquisition method and constructive induction as a machine learning method overcomes key weaknesses of each of these methods used separately.