Deictic believability: Coordinated gesture, locomotion, and speech in lifelike pedagogical agents

Lifelike animated agents for. knowledge-based learning environments can pro vide timely, customized advice to support students' problem solving. Becaus e of their strong visual presence, they hold significant promise for substa ntially increasing students' enjoyment of their learning experiences. A key problem posed by lifelike agents that inhabit artificial worlds is deictic believability. In the same manner that humans refer to objects in their en vironment through judicious combinations of speech, locomotion, and gesture , animated agents should be able to move through their environment and poin t to and refer to objects appropriately as they provide problem-solving adv ice. In this paper we describe a framework for achieving deictic believabil ity in animated agents. A deictic behavior planner exploits a world model a nd the evolving explanation plan as it selects and coordinates locomotive, gestural, and speech behaviors. The resulting behaviors and utterances are believable, and the references exhibit a lack of ambiguity. This approach t o spatial deixis has been implemented in a lifelike animated agent, COSMO, who inhabits a learning environment for the domain of Internet packet routi ng. COSMO provides real-time advice to students as they escort packets thro ugh a virtual world of interconnected routers. Results of an informal focus group study with the COSMO agent suggest that the spatial deixis framework produces clear explanatory animated behaviors.