GENERAL GAUSSIAN PRIORS FOR IMPROVED GENERALIZATION

We explore the dependence of performance measures, such as the general ization error and generalization consistency. on the structure and the parametrization of the prior on ''rules'', instanced here by the nois y linear perceptron. Using a statistical mechanics framework, we show how one may assign values to the parameters of a model for a ''rule'' on the basis of data instancing the rule. Information about the data, such as input distribution, noise distribution and other ''rule'' char acteristics may be embedded in the form of general Gaussian priors for improving net performance. We examine explicitly two types of general Gaussian priors which are useful in some simple cases. We calculate t he optimal values for the parameters of these priors and show their ef fect in modifying the most probable, MAP, values for the rules. Copyri ght (C) 1996 Elsevier Science Ltd.