VISUAL TRACKING OF A MOVING TARGET BY A CAMERA MOUNTED ON A ROBOT - ACOMBINATION OF CONTROL AND VISION

In this paper, we present algorithms for robotic (eye-in-hand configur ation) real-time visual tracking of arbitrary 3-D objects traveling at unknown velocities in a 2-D space (depth is given as known). We formu late the problem of visual tracking as a problem of combining control with computer vision. We present a mathematical formulation of a contr ol problem that includes the sensory information of a novel and import ant feedback sensor (vision sensor). This formulation represents every thing with respect to the camera frame and not with respect to the wor ld frame. Due to this fact, we have the ability to quickly and accurat ely control the camera. We propose using the sum-of-squared difference s (SSD) optical flow for the computation of the vector of discrete dis placements each instant of time. These displacements can be fed either directly to a PI controller or to a pole assignment controller or to a discrete steady-state Kalman filter. In the latter case, the Kalman filter calculates the estimated values of the system's states and the exogenous disturbances, and a discrete LQG controller computes the des ired motion of the robotic system. The outputs of the controllers are sent to a Cartesian robotic controller that drives the robot. The perf ormance of the proposed algorithms has been tested on the CMU Direct-D rive (DD) Arm II, and the results are presented in this paper.