LEARNING IN NEURAL NETWORKS WITH MATERIAL SYNAPSES

We discuss the long term maintenance of acquired memory in synaptic co nnections of a perpetually learning electronic device. This is affecte d by ascribing each synapse a finite number of stable states in which it can maintain for indefinitely long periods. Learning uncorrelated s timuli is expressed as a stochastic process produced by the neural act ivities on the synapses. In several interesting cases the stochastic p rocess can be analyzed in detail, leading to a clarification of the pe rformance of the network, as an associative memory, during the process of uninterrupted learning. The stochastic nature of the process and t he existence of an asymptotic distribution for the synaptic values in the network imply generically that the memory is a palimpsest but capa city is as low as log N for a network of N neurons. The only way we fi nd for avoiding this tight constraint is to allow the parameters gover ning the learning process (the coding level of the stimuli; the transi tion probabilities for potentiation and depression and the number of s table synaptic levels) to depend on the number of neurons. It is shown that a network with synapses that have two stable states can dynamica lly learn with optimal storage efficiency, be a palimpsest, and mainta in its (associative) memory for an indefinitely long time provided the coding level is low and depression is equilibrated against potentiati on. We suggest that an option so easily implementable in material devi ces would not have been overlooked by biology. Finally we discuss the stochastic learning on synapses with variable number of stable synapti c states.