Plasticity in value systems and its role in adaptive behavior

Adaptive behavior requires the sensing of salient behavioral consequences w hich can act to modulate changes in neural connections linking sensory and motor structures. In previous work, we proposed that salient sensory events trigger neuronal value systems capable of modulating synaptic plasticity. Here, we investigate the capacity of value systems to modulate their own re sponses in the context of various conditioning tasks. To this end, we imple ment a modifiable value system incorporating anatomical and physiological p roperties within Darwin V, a neuronal model embedded in a mobile real world device. While exploring an environment containing stimulus objects, Darwin V's visual maps develop object-related neuronal responses. Phasic response s of a value system initially triggered only by object-"taste" (innate valu e) modulate changes in connections between visual and motor neurons, thus l inking specific visual responses to appropriate motor outputs. Over time, D arwin V is able behaviorally to discriminate between "striped" objects with positive value (appetitive behavior) and objects with "blobs" with negativ e value (aversive behavior) based on vision alone. In parallel with modific ation of visuo-motor connections, value-dependent modification also occurs in connections from visual sensory maps to the value system itself. As a re sult, visual activity patterns become able directly to trigger value signal s (acquired value). If acquired value is disabled, transfer of the value si gnal to stimuli preceding innately salient events does not occur, and behav ioral responses due to aversive conditioning are subject to rapid extinctio n. If an auditory signal reliably precedes the visual appearance of an aver sive object, Darwin V could be conditioned first to reject the object based on vision (primary conditioning), and subsequently based on sound alone (s econdary conditioning). We compare the functional characteristics of value- dependent learning to formal notions of reinforcement learning. We suggest that plasticity in sensory afferents to value systems may provide a neurobi ological basis for mediating the changing effects of saliency on adaptive b ehavioral responses.