WYSIWYF display: A visual/haptic interface to virtual environment

To build a VR training system for visuomotor skills, an image displayed by a visual interface should be correctly registered to a haptic interface so that the visual sensation and the haptic sensation are both spatially and t emporally consistent. In other words, it is desirable that what you see is what you feel (WYSIWYF). In this paper, we propose a method that can realize correct visual/haptic r egistration, namely WYSIWYF, by using a vision-based, object-tracking techn ique and a video-keying technique. Combining an encountered-type haptic dev ice with a motion-command-type haptic rendering algorithm makes it possible to deal with two extreme cases (free motion and rigid constraint). This ap proach provides realistic haptic sensations, such as free-to-touch and move -and-collide. We describe a first prototype and illustrate its use with sev eral demonstrations. The user encounters the haptic device exactly when his or her hand reaches a virtual object in the display. Although this prototy pe has some remaining technical problems to be solved, it serves well to sh ow the validity of the proposed approach.