This paper describes current work on a cooperative teleassistance syst
em for semiautonomous control of mobile robots. This system combines a
robot architecture for limited autonomous perceptual and motor contro
l with a knowledge-based operator assistant that provides strategic se
lection and enhancement of relevant data. It extends recent developmen
ts in artificial intelligence in modeling the role of visual interacti
ons in problem solving for application to an interface permitting the
human and remote to cooperate in cognitively demanding tasks such as r
ecovering from execution failures, mission planning, and learning. The
design of the system is presented, together with a number of exceptio
n-handling scenarios that were constructed as a result of experiments
with actual sensor data collected from two mobile robots.