Multimedia human-computer interaction for presence and exploration in a telemuseum

It is not always feasible or practical to be present at a location of inter est. Telepresence makes such possible. This paper presents the design and i mplementation of a telepresence system in which a mobile manipulator robot is utilized to explore a museum at a remote site, sending live video and au dio on demand to a user at the local site. The user's sense of presence is also enhanced through the capability to manipulate objects remotely via the robot's gripper. The display system is a slightly modified, off-the-shelf mediascope, a television display system that can be worn like a pair of gla sses. The head movements of the user are monitored using color tracking, an d the displayed images are updated according to the motions of the head. Th is is achieved by interpreting the human head movements and transforming th em into control signals for appropriate movements of the robot and its moun ted color camera. Novel features of the presented system include utilizatio n of human implicit commands, availability and cost-effectiveness of the sy stem, feeling of presence, and intelligence fusion of the human user (where to go and what to do) with the robot's intelligence (local obstacle avoida nce and navigation). The multimedia interface and control of the system are described in detail. Additionally, a number of research issues that requir e further investigation in order to realize attainable, high fidelity telep resence systems are discussed in this paper.