Keith Humphreys
Department of Statistics
University of Glasgow,
Glasgow, G12 8QQ,
Scotland, U.K.
E-mail: keith@stats.gla.ac.uk
Phone: +44-141-330-4852
Fax: +44-141-330-4814
D.M. Titterington
Department of Statistics
University of Glasgow,
Glasgow, G12 8QQ,
Scotland, U.K.
E-mail: mike@stats.gla.ac.uk
Phone: +44-141-330-5022
Fax: +44-141-330-4814
Exact inference for Boltzmann machines is computationally expensive. One approach to improving tractability is to approximate the gradient algorithm. We describe a new way of doing this which is based on Bahadur's representation of the multivariate binary distribution (Bahadur, 1961). We compare the approach, for networks with no unobserved variable, to the ``mean field'' approximation of Peterson and Anderson (1987) and the approach of Kappen and Rodriguez (1998), which is based on the linear response theorem. We also investigate the use of the pairwise association cluster method (Tanaka and Morita, 1995).
Boltzmann machines, learning, Bahadur, mean field