INCREMENTAL EVOLUTION OF COMPLEX GENERAL BEHAVIOR

Several researchers have demonstrated how complex action sequences can be learned through neuroevolution (i.e., evolving neural networks wit h genetic algorithms). However, complex general behavior such as evadi ng predators or avoiding obstacles, which is not tied to specific envi ronments, turns out to be very difficult to evolve. Often the system d iscovers mechanical strategies, such as moving back and forth, that he lp the agent cope but are not very effective, do not appear believable , and do not generalize to new environments. The problem is that a gen eral strategy is too difficult for the evolution system to discover di rectly. This article proposes an approach wherein such complex general behavior is learned incrementally, by starting with simpler behavior and gradually making the task more challenging and general The task tr ansitions are implemented through successive stages of Delta coding (i .e., evolving modifications), which allows even converged populations to adapt to the new task. The method is tested in the stochastic, dyna mic task of prey capture and is compared with direct evolution. The in cremental approach evolves more effective and more general behavior an d should also scale up to harder tasks.