This paper reports a visual tracking system that can track moving obje
cts in real-time with a modest workstation equipped with a pan-tilt de
vice. The algorithm essentially has three parts: (1) feature detection
, (2) tracking and (3) control of the robot head. Corners are viewpoin
t invariant, hence being utilised as the beacon for tracking. Tracking
is performed in two stages of Kalman filtering and affine transformat
ion. A technique of reducing greatly the computational time for the co
rrelation is also described. The Kalman filter predicts intelligently
the fovea window and reduced computation dramatically. The affine tran
sformation deals with the unexpected events when there is partial occl
usion.