Criterion-based methods for Bayesian model assessment

We propose a general Bayesian criterion fur model assessment. The criterion is constructed from the posterior predictive distribution of the data, and can he written as a sum of two components, one involving the means of the posterior predictive. distribution and the other involving the variances. I t can be viewed as a Bayesian goodness-of-fit statistic which measures the performance of a model by a combination of how close its predictions are to the observed data and the variability of the predictions. We call this pro posed predictive criterion the L measure, it is motivated hy earlier work o f Ibrahim and Laud (1994) and related to a criterion of Gelfand and Ghosh ( 1998). We examine the L measure in detail for the class of generalized line ar models and survival models with right censored or interval censored data . We also propose a calibration of the L measure, defined as the prior pred ictive distribution of the difference between the L measures of the candida te model and the criterion minimizing model, and call it the calibration di stribution. The calibration distribution will allow us to formally compare tao competing models based on their L measure values. We discuss theoretica l properties of the calibration distribution in detail, and provide Monte C arlo methods for computing it. For the linear model, we derive an analytic closed form expression for the L measure and the calibration distribution, and;also derive a closed form expression for the mean of the calibration di stribution. These novel developments will enable us to fully characterize t he properties of the L measure for each model under consideration and will facilitate a direct formal comparison between several models, including non -nested models. Informative priors based on historical data and computation al techniques are discussed. Several simulated and real datasets are used t o demonstrate the proposed methodology.