A learning attractor neural network (LANN) With a double dynamics of n
eural activities and synaptic efficacies, operating on two different t
imescales is studied by simulations in preparation for an electronic i
mplementation. The present network includes several quasirealistic fea
tures: neurons are represented by their afferent currents and output s
pike rates; excitatory and inhibitory neurons are separated; attractor
spike rates as well as coding levels in arriving stimuli are low; lea
rning takes place only between excitatory units. Synaptic dynamics is
an unsupervised, analogue Hebbian process, but long term memory in the
absence of neural activity is maintained by a refresh mechanism which
on long timescales discretizes the synaptic values, converting learni
ng into asynchronous stochastic process induced by the stimuli on the
synaptic efficacies. This network is intended to learn a set of attrac
tors from the statistics of freely arriving stimuli, which are represe
nted by external synaptic inputs injected into the excitatory neurons.
In the simulations different types of sequences of many thousands of
stimuli are presented to the network, without distinguishing in the dy
namics a learning phase from retrieval. Stimulus sequences differ in p
re-assigned global statistics (including time-dependent statistics); i
n orders of presentation of individual stimuli within a given statisti
cs; in lengths of time intervals for each presentation and in the inte
rvals separating one stimulus from another. We find that the network e
ffectively learns a set of attractors representing the statistics of t
he stimuli, and is able to modify its attractors when the input statis
tics change. Moreover, as the global input statistics changes the netw
ork can also forget attractors related to stimulus classes no longer p
resented. Forgetting takes place only due to the arrival of new stimul
i. The performance of the network and the statistics of the attractors
are studied as a function of the input statistics. Most of the large-
scale characteristics of the learning dynamics can be captured theoret
ically. This model modifies a previous implementation of a LANN compos
ed of discrete neurons, in a network of more realistic neurons. The di
fferent elements have been designed to facilitate their implementation
in silicon.