Mean-field methods for a special class of Belief Networks

The chief aim of this paper is to propose mean-field approximations for a b road class of Belief networks, of which sigmoid and noisy-or networks can b e seen as special cases. The approximations are based on a powerful mean-fi eld theory suggested by Plefka. We show that Saul, Jaakkola, and Jordan's a pproach is the first order approximation in Plefka's approach, via a variat ional derivation. The application of Plefka's theory to belief networks is not computationally tractable. To tackle this problem we propose new approx imations based on Taylor series. Small scale experiements show that the pro posed schemes are attractive.