Model-independent mean-field theory as a local method for approximate propagation of information

We present a systematic approach to mean-field theory (MFT) in a general pr obabilistic setting without assuming a particular model. The mean-field equ ations derived here may serve as a local, and thus very simple, method for approximate inference in probabilistic models such as Boltzmann machines or Bayesian networks. Our approach is 'model-independent' in the sense that w e do not assume a particular type of dependences; in a Bayesian network, fo r example, we allow arbitrary tables to specify conditional dependences. In general, there are multiple solutions to the mean-field equations. We show that improved estimates can be obtained by forming a weighted mixture of t he multiple mean-field solutions. Simple approximate expressions for the mi xture weights are given. The general formalism derived so far is evaluated for the special case of Bayesian networks. The benefits of taking info acco unt multiple solutions are demonstrated by using MFT for inference in a sma ll and in a very large Bayesian network. The results are compared with the exact results.