We present an approach to support effective learning and adaptation of
behaviors for autonomous agents with reinforcement learning algorithm
s. These methods can identify control systems that optimize a reinforc
ement program, which is, usually, a straightforward representation of
the designer's goals. Reinforcement learning algorithms usually are to
o slow to be applied in real time on embodied agents, although they pr
ovide a suitable way to represent the desired behavior. We have tackle
d three aspects of this problem: the speed of the algorithm, the learn
ing procedure, and the control system architecture. The learning algor
ithm we have developed includes features to speed up learning, such as
niche-based learning, and a representation of the control modules in
terms of fuzzy rules that reduces the search space and improves robust
ness to noisy data. Our learning procedure exploits methodologies such
as learning from easy missions and transfer of policy from simpler en
vironments to the more complex. The architecture of our control system
is layered and modular, so that each module has a low complexity and
can be learned in a short time. The composition of the actions propose
d by the modules is either learned or predefined. Finally, we adopt an
anytime learning approach to improve the qualify of She control syste
m on-line and to adapt it to dynamic environments. The experiments we
present in this article concern learning to reach another moving agent
in a real, dynamic environment that includes nontrivial situations su
ch as that in which the moving target is faster than the agent and tha
t in which the target is hidden by obstacles.