Cooperative behavior acquisition for mobile robots in dynamically changingreal worlds via vision-based reinforcement learning and development

In this paper, we first discuss the meaning of physical embodiment and the complexity of the environment in the context of multi-agent learning. We th en propose a vision-based reinforcement learning method that acquires coope rative behaviors in a dynamic environment. We use the robot soccer game ini tiated by RoboCup (Kitano et al., 1997) to illustrate the effectiveness of our method. Each agent works with other team members to achieve a common go al against opponents. Our method estimates the relationships between a lear ner's behaviors and those of other agents in the environment through intera ctions (observations and actions) using a technique from system identificat ion. In order to identify the model of each agent, Akaike's Information Cri terion is applied to the results of Canonical Variate Analysis to clarify t he relationship between the observed data in terms of actions and future ob servations. Next, reinforcement learning based on the estimated state vecto rs is performed to obtain the optimal behavior policy. The proposed method is applied to a soccer playing situation. The method successfully models a rolling ball and other moving agents and acquires the learner's behaviors. Computer simulations and real experiments are shown and a discussion is giv en. (C) 1999 Elsevier Science B.V. All rights reserved.