DISTRIBUTED DYNAMICS IN NEURAL NETWORKS

We analyze the dynamics and statistical mechanics of attractor neural networks with ''distributed'' updating rules in which groups of one or more neurons are updated simultaneously. Such partially parallel upda ting schemes are a central feature of neural-network architectures tha t use many processors, implemented either on special multiprocessor ha rdware, or among many computers linked over a network. Several updatin g rules are classified and discussed; these rules generalize the paral lel dynamics of the Little model and the one-at-a-time dynamics of the Hopfield model. Analytic results presented herein include a stability criterion that specifies sufficient conditions under which distribute d dynamics lead to fixed-point attractors. For binary neurons with blo ck-sequential updating and a Hebbian learning rule, the storage capaci ty is found as a function of the number of update groups. Several open problems are also discussed.