DOES EXTRA KNOWLEDGE NECESSARILY IMPROVE GENERALIZATION

The generalization error is a widely used performance measure employed in the analysis of adaptive learning systems. This measure is general ly critically dependent on the knowledge that the system is given abou t the problem it is trying to learn. In this paper we examine to what extent it is necessarily the case that an increase in the knowledge th at the system has about the problem will reduce the generalization err or. Using the standard definition of the generalization error, we pres ent simple cases for which the intuitive idea of ''reducivity''-that m ore knowledge will improve generalization-does not hold. Under a simpl e approximation, however, we find conditions to satisfy ''reducivity.' ' Finally, we calculate the effect of a specific constraint on the gen eralization error of the linear perceptron, in which the signs of the weight components are fixed. This particular restriction results in a significant improvement in generalization performance.