The VirBot: a virtual reality robot driven with multimodal commands

In this paper we show how symbolic Artificial Intelligence (AI) techniques can be used to develop intelligent Virtual Reality (VR) environments. We ha ve developed an expert system architecture for building intelligent agents that respond to voice and gesture commands in virtual environments. To demo nstrate the utility of this we present a simple application that allows a u ser to drive a virtual robot (the VirBot) around a virtual environment with multimodal commands. This software agent is able to accept spoken commands from the user, interpret them using natural language understanding and int eract with and modify the virtual environment. Command recognition is furth er improved by the use of contextual information which reduces recognition error rates by 31%. Context is encoded using scripts, making it easy to com pose inferences about events for which there is incomplete or fuzzy informa tion. (C) 1998 Elsevier Science Ltd. All rights reserved.