Providing synthetic views for teleoperation using visual pose tracking in multiple cameras

This paper describes a visual tool for teleoperative experimentation involv ing remote manipulation and contact tasks, Using modest hardware, it recove rs in real time the pose of moving polyhedral objects, and presents a synth etic view of the scene to the operator of a teleoperated robot using any ch osen viewpoint and viewing direction. To recover pose, the method of line t racking first introduced by Harris is extended to multiple calibrated camer as, and its dynamic performance Improved using robust methods and iterative filtering, Experiments are reported which determine the static and dynamic performance of the vision system, and its use in teleoperation is illustra ted in two experiments, a peg in hole manipulation task and an impact contr ol task.