The importance of complexity in model selection

Model selection should be based not solely on goodness-of-fit, but must als o consider model complexity. While the goal of mathematical modeling in cog nitive psychology is to select one model from a set of competing models tha t best captures the underlying mental process. choosing the model that best fits a particular set of data will not achieve this goal. This is because a highly complex model can provide a good lit without necessarily bearing a ny interpretable relationship with he underlying process. it is shown that model selection based solely on the fit to observed data will result in the choice of all unnecessarily complex model that overfits the data, and thus generalizes poorly. The effect of over-fitting must be properly of first b y model selection methods. An application example of selection methods usin g artificial data is also presented. (C) 2000 Academic Press.