Applications of hybrid Monte Carlo to Bayesian generalized linear models: Quasicomplete separation and neural networks

The "leapfrog" hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separatio n. This method offers a very attractive solution to this common problem, pr oviding a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. Th e method is also quite successful in fitting generalized linear models in w hich the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajec tory can become very demanding. In this case, it is best to mix the algorit hm with multivariate random walk Metropolis-Hastings. However, this entails very little additional programming work.