Visual and haptic collaborative tele-presence

The core of a successful sense of presence is a visually, aurally, and hapt ically compelling experience. In this paper, we introduce the integration o f vision and haptics for the purposes of remote collaboration. A remote sta tion acquires a 3D-model of an object of interest which is transmitted to a local station. A user in the local station manipulates a virtual and the r emote object as if he/she is haptically and visually at the remote station. This tele-presence feeling is achieved by visually registering the head-mo unted display of the local user to the remote world and by dynamically regi stering the local object both visually and haptically with respect to the r emote world. This can be achieved by adequate modeling and feedforward comp ensation including gravity compensation for the robotic manipulator with wh ich the operator interacts. We present multiple scenarios where such a capa bility will be useful. One is remote design where a user tests a remotely d esigned docking station by inserting a virtual laptop into a model of the 3 D docking station transmitted from a remote site. Medical robotics provides another possible scenario in which a resident is given surgical training t o perform a virtual laparoscopy on a 3D exterior model of a patient, includ ing tomographic registration of anatomical structures. We present results f rom numerous experiments from both the visual and haptic aspects as well as in integrated form. (C) 2001 Elsevier Science Ltd. All rights reserved.