Model comparisons and model selections based on generalization criterion methodology

The purpose of this article is to formalize the generalization criterion me thod for model comparison. The method has the potential to provide powerful comparisons of complex and nonnested models that may also differ in terms of numbers of parameters. The generalization criterion differs From the bet ter known cross-validation criterion in the following critical procedure. A lthough both employ a calibration stage to estimate parameters, cross-valid ation employs a replication sample from the same design for the validation stage, whereas generalization employs a new design for the critical stage. Two examples of the generalization criterion method are presented that demo nstrate its usefulness for selecting a model based on sound scientific prin ciples out of a set that also contains models lacking sound scientific prin ciples that are either overly complex or oversimplified. The main advantage of the generalization criterion is its reliance on extrapolations to new c onditions. After all, accurate a priori predictions to new conditions are t he hallmark of a good scientific theory. (C) 2000 Academic Press.