NETWORK INFORMATION CRITERION - DETERMINING THE NUMBER OF HIDDEN UNITS FOR AN ARTIFICIAL NEURAL-NETWORK MODEL

The problem of model selection, or determination of the number of hidd en units, can be approached statistically, by generalizing Akaike's in formation Criterion (AIC) to be applicable to unfaithful (i.e., unreal izable) models with general loss criteria including regularization ter ms. The relation between the training error and the generalization err or is studied in terms of the number of the training examples and the complexity of It network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a n ew Network Information Criterion (NIC) which is useful for selecting t he optimal network model based on a given training set.