Most problems in learning to control dynamic systems involve learning
under uncertainty, noise, and the lack of explicit instructional infor
mation about how to perform a task. Under these circumstances, techniq
ues developed by artificial intelligence researchers for 'learning fro
m examples,' including the 'supervised learning' techniques studied by
neural network researchers, are impractical because of the difficulty
of obtaining training information (the 'examples') in the form of sit
uation-action training pairs. A useful alternative in such situations
is a learning technique that can discover appropriate actions in vario
us situations through a search process that is guided by evaluative pe
rformance feedback. Reinforcement learning methods developed by neural
network researchers are examples of such techniques. This paper focus
es on direct reinforcement learning techniques and discusses their rol
e in learning control by relating them to similar adaptive control met
hods. Several examples are also presented to illustrate the power and
utility of direct reinforcement learning techniques for learning contr
ol.