SELECTING THE RIGHT-SIZE MODEL FOR PREDICTION

We evaluate the effectiveness of cross-validation in selecting the rig ht-size model for decision tree and k-nearest neighbor learning method s. For samples with at least 200 cases, extensive empirical evidence s upports the following conclusions relative to complexity-fit selection : (a) 10-fold cross-validation is nearly unbiased; (b) ignoring model complexity-fit and picking the ''standard'' model is highly biased; (c ) 10-fold cross-validation is consistent with optimal complexity-fit s election for large sample sizes and (d) the accuracy of complexity-fit selection by 10-fold cross-validation is largely dependent on sample size, irrespective of the population distribution.