Bayesian nonlinear model selection and neural networks: A conjugate prior approach

In order to select the best predictive neural-network architecture in a set of several candidate networks, me propose a general Bayesian nonlinear reg ression model comparison procedure, based on the maximization of an expecte d utility criterion. This criterion selects the model under which the train ing set achieves the highest level of internal consistency, through the pre dictive probability distribution of each model, The density of this distrib ution is computed as the model posterior predictive density and is asymptot ically approximated from the assumed Gaussian likelihood of the data set an d the related conjugate prior density of the parameters. The use of such a conjugate prior allows the analytic calculation of the parameter posterior and predictive posterior densities, in an empirical-Bayes-like approach. Th is Bayesian selection procedure allows us to compare general nonlinear regr ession models and in particular feedforward neural networks, in addition to embedded models as usual with asymptotic comparison tests.