Approximate inference in large and densely connected graph-ical models is a challenging but highly relevant problem. Belief propaga-tion, as a method for performing approximate inference in loopy graphs,has shown empirical success in many applications. However, convergenceof belief propagation can only be guaranteed for simple graphs. Whetherbelief propagation converges depends strongly on the applied messageupdate scheme, and specialized schemes can be highly beneficial. Yet,residual belief propagation is the only established method utilizing thisfact to improve convergence properties. In experiments, we observe thatresidual belief propagation fails to converge if local oscillations occur andthe same sequence of messages is repeatedly updated. To overcome thisissue, we propose two novel message update schemes. In the first schemewe add noise to oscillating messages. In the second scheme we applyweight decay to gradually reduce the influence of these messages and con-sequently enforce convergence. Furthermore, in contrast to previous work,we consider the correctness of the obtained marginals and observe sig-nificant performance improvements when applying the proposed messageupdate schemes to various Ising models with binary random variables.

Top