Programming contact tasks using a reality-based virtual environment integrated with vision

We present an integrated system in which an operator uses a simulated envir onment to program part-mating and contact tasks, Generation of models withi n this virtual environment is facilitated using a fast, occlusion-tolerant, three-dimensional (3-D) grey-scale vision system which can recognize and a ccurately locate objects within the work site. A major goal of this work is to make robotic programming easy and intuitive for untrained users working with standard desktop hardware. Simulation can help accomplish this, offer ing the ease-of-use benefits of "programming by demonstration," coupled wit h the ability to create a programmer-friendly virtual environment. Within a simulated environment, it is also straightforward to track and interpret a n operator's actions. The simulator models objects as polyhedra and impleme nts full 3-D contact dynamics, making is easy to place and manipulate objec ts using input from a simple two-dimensional (2-D) mouse. When a manipulati on task is completed, local planning techniques are used to turn the virtua l environment's motion sequence history into a set of robot motion commands capable of realizing the prescribed task.