Withdrawing an example from the training set: An analytic estimation of its effect on a non-linear parameterised model

For a non-linear parameterised model, the effects of withdrawing an example from the training set can be predicted. We focus on the prediction of the error on the left-out example, and of the confidence interval for the predi ction of this example. We derive a rigorous expression of the first-order e xpansion, in parameter space, of the gradient of a quadratic cost function, and specify its validity conditions. As a consequence, we derive approxima te expressions of the prediction error on a given example, and of the confi dence interval thereof, had this example been withdrawn from the training s et. We show that the influence of an example on the model can be summarised by a single parameter. These results are applicable to leave-one-out cross -validation, with a considerable decrease in computation time with respect to conventional leave-one-out. The paper focuses on the theoretical aspects of the question; both academic illustrations and large-scale industrial ex amples are described in [9]. (C) 2000 Elsevier Science B.V. All rights rese rved.