INSTRUCTIONS, INTENTIONS AND EXPECTATIONS

Based on an ongoing attempt to integrate Natural Language instructions with human figure animation, we demonstrate that agents' understandin g and use of instructions can complement what they can derive from the environment in which they act. We focus on two attitudes that contrib ute to agents' behavior-their intentions and their expectations-and sh own how Natural Language instructions contribute to such attitudes in ways that complement the environment. We also show that instructions c an require more than one context of interpretation and thus that agent s' understanding of instructions can evolve as their activity progress es. A significant consequence is that Natural Language understanding i n the context of behavior cannot simply be treated as ''front end'' pr ocessing, but rather must be integrated more deeply into the processes that guide an agent's behavior and respond to its perceptions.