The choice of variables in multivariate regression: A non-conjugate Bayesian decision theory approach

We consider the choice of explanatory variables in multivariate linear regr ession. Our approach balances prediction accuracy against costs attached to variables:in a multivariate version of a decision theory approach pioneere d by Lindley (1968). We also employ a non-conjugate proper prior distributi on for the parameters of the regression model, extending the standard norma l-inverse Wishart by adding a component of error which is unexplainable by any number of predictor variables, thus avoiding the determinism identified by Dawid (1988). Simulated annealing and fast updating algorithms are used to search for good subsets when there are very many regressors. The techni que is illustrated on a near infrared spectroscopy example involving 39 obs ervations and 300 explanatory variables. This demonstrates the effectivenes s of multivariate regression as opposed to separate univariate regressions. It also emphasises that within a Bayesian framework more variables than ob servations can be utilised.