REINFORCEMENT LEARNING BY HEBBIAN SYNAPSES WITH ADAPTIVE THRESHOLDS

A central problem in learning theory is how the vertebrate brain proce sses reinforcing stimuli in order to master complex sensorimotor tasks . This problem belongs to the domain of supervised learning, in which errors in the response of a neural network serve as the basis for modi fication of synaptic connectivity in the network and thereby train it on a computational task. The model presented here shows how a reinforc ing feedback can modify synapses in a neuronal network according to th e principles of Hebbian learning. The reinforcing feedback steers syna pses towards long-term potentiation or depression by critically influe ncing the rise in postsynaptic calcium, in accordance with findings on synaptic plasticity in mammalian brain. An important feature of the m odel is the dependence of modification thresholds on the previous hist ory of reinforcing feedback processed by the network. The learning alg orithm trained networks successfully on a task in which a population v ector in the motor output was required to match a sensory stimulus vec tor presented shortly before. In another task, networks were trained t o compute coordinate transformations by combining different visual inp uts. The model continued to behave well when simplified units were rep laced by single-compartment neurons equipped with several conductances and operating in continuous time. This novel form of reinforcement le arning incorporates essential properties of Hebbian synaptic plasticit y and thereby shows that supervised learning can be accomplished by a learning rule similar to those used in physiologically plausible model s of unsupervised learning. The model can be crudely correlated to the anatomy and electrophysiology of the amygdala, prefrontal and cingula te cortex and has predictive implications for further experiments on s ynaptic plasticity and learning processes mediated by these areas. (C) 1997 IBRO. Published by Elsevier Science Ltd.