A Hebbian feedback covariance learning paradigm for self-tuning optimal control

We propose a novel adaptive optimal control paradigm inspired by Hebbian co variance synaptic adaptation, a preeminent model of learning and memory as well as other malleable functions in the brain. The adaptation is driven by the spontaneous fluctuations in the system input and output, the covarianc e of which provides useful information about the changes in the system beha vior. The control structure represents a novel form of associative reinforc ement learning in which the reinforcement signal is implicitly given by the covariance of the input-output (I/O) signals. Theoretical foundations for the paradigm are derived using Lyapunov theory and are verified by means of computer simulations. The learning algorithm is applicable to a general cl ass of nonlinear adaptive control problems, This on-line direct adaptive co ntrol method benefits from a computationally straightforward design, proof of convergence, no need for complete system identification, robustness to n oise and uncertainties, and the ability to optimize a general performance c riterion in terms of system states and control signals. These attractive pr operties of Hebbian feedback covariance learning control lend themselves to future investigations into the computational functions of synaptic plastic ity in biological neurons.