Np. Papanikolopoulos et al., VISUAL TRACKING OF A MOVING TARGET BY A CAMERA MOUNTED ON A ROBOT - ACOMBINATION OF CONTROL AND VISION, IEEE transactions on robotics and automation, 9(1), 1993, pp. 14-35
In this paper, we present algorithms for robotic (eye-in-hand configur
ation) real-time visual tracking of arbitrary 3-D objects traveling at
unknown velocities in a 2-D space (depth is given as known). We formu
late the problem of visual tracking as a problem of combining control
with computer vision. We present a mathematical formulation of a contr
ol problem that includes the sensory information of a novel and import
ant feedback sensor (vision sensor). This formulation represents every
thing with respect to the camera frame and not with respect to the wor
ld frame. Due to this fact, we have the ability to quickly and accurat
ely control the camera. We propose using the sum-of-squared difference
s (SSD) optical flow for the computation of the vector of discrete dis
placements each instant of time. These displacements can be fed either
directly to a PI controller or to a pole assignment controller or to
a discrete steady-state Kalman filter. In the latter case, the Kalman
filter calculates the estimated values of the system's states and the
exogenous disturbances, and a discrete LQG controller computes the des
ired motion of the robotic system. The outputs of the controllers are
sent to a Cartesian robotic controller that drives the robot. The perf
ormance of the proposed algorithms has been tested on the CMU Direct-D
rive (DD) Arm II, and the results are presented in this paper.