Real time facial expression recognition system with applications to facialanimation in MPEG-4

This paper discusses recognition up to intensities of mix of primary facial expressions in real time. The proposed recognition method is compatible wi th the MPEG-4 high level expression Facial Animation Parameter (PAP). In ou r method, the whole facial image is considered as a single pattern without any block segmentation. As model features, an expression vector, viz. low g lobal frequency coefficient (DCT) changes relative to neutral facial image of a person is used. These features are robust and good enough to deal with real time processing. To construct a person specific model, apex images of primary facial expression categories are utilized as references. Personal facial expression space (PFES) is constructed by using multidimensional sca ling. PFES with its generalization capability maps an unknown input image r elative to known reference images. As PFES possesses linear mapping charact eristics, MPEG-4 high level expression FAP can be easily calculated by the location of the input face on PFES. Also, temporal variations of facial exp ressions can be seen on PFES as trajectories. Experimental results are show n to demonstrate the effectiveness of the proposed method.