Predictive approaches for choosing hyperparameters in Gaussian processes

Gaussian processes are powerful regression models specified by parameterize d mean and covariance functions. Standard approaches to choose these parame ters (known by the name hyperparameters) are maximum likelihood and maximum a posteriori. In this article, we propose and investigate predictive appro aches based on Geisser's predictive sample reuse (PSR) methodology and the related Stone's cross-validation ICV) methodology. More specifically, we de rive results for Geisser's surrogate predictive probability (GPP), Geisser' s predictive mean square error (GPE), and the standard CV error and make a comparative study. Within an approximation we arrive at the generalized cro ss-validation (GCV) and establish its relationship with the GPP and GPE app roaches. These approaches are tested on a number of problems. Experimental results show that these approaches are strongly competitive with the existi ng approaches.