ON LANGEVIN UPDATING IN MULTILAYER PERCEPTRONS

The Langevin updating rule, in which noise is added to the weights dur ing learning, is presented and shown to improve learning on problems w ith initially ill-conditioned Hessians. This is particularly important for multilayer perceptrons with many hidden layers, that often have i ll-conditioned Hessians. In addition, Manhattan updating is shown to h ave similar effect.