TRACKING AND RECOGNIZING HAND GESTURES, USING STATISTICAL SHAPE MODELS

Hand gesture recognition from video images is of considerable interest as a means of providing simple and intuitive man-machine interfaces. Possible applications range from replacing the mouse as a pointing dev ice to virtual reality and communication with the deaf. We describe an approach to tracking a hand in an image sequence and recognising, in each video frame. which of five gestures it has adopted. A statistical ly based Point Distribution Model (PDM) is used to provide a compact p arametrised description of the shape of the hand for any of the gestur es or the transitions between them. The values of the resulting shape parameters are used in a statistical classifier to identify gestures. The model can be used as a deformable template to track a hand through a video sequence but this proves unreliable. We describe how a set of models, one for each of the five gestures, can be used for tracking w ith the appropriate model selected automatically. We show that this re sults in reliable tracking and gesture recognition for two 'unseen' vi deo sequences in which all the gestures are used.