Optically based direct manipulation for augmented reality

Augmented reality (AR) constitutes a very powerful three-dimensional user i nterface for many "hands-on" application scenarios. To fully exploit the AR paradigm, the computer must not only augment the real world, but also acce pt feedback from it. In this paper, we present an optical approach for coll ecting such feedback by analyzing video sequences to track users and the ob jects they work with. Our system can be set up in any room after quickly pl acing a few known optical targets in the scene. We present two demonstratio n scenarios to illustrate the overall concept and potential of our approach and then discuss the research issues involved. (C) 1999 Elsevier Science L td. All rights reserved.