Classifying facial actions

The Facial Action Coding System (FACS) [23] is an objective method for quan tifying facial movement in terms of component actions. This system is widel y used in behavioral investigations of emotion, cognitive processes, and so cial interaction. The coding is presently performed by highly trained human experts. This paper explores and compares techniques for automatically rec ognizing facial actions in sequences of images. These techniques include an alysis of facial motion through estimation of optical flow; holistic spatia l analysis, such as principal component analysis, independent component ana lysis, local feature analysis, and linear discriminant analysis; and method s based on the outputs of local filters, such as Gabor wavelet representati ons and local principal components. Performance of these systems is compare d to naive and expert human subjects. Best performances were obtained using the Gabor wavelet representation and the independent component representat ion, both of which achieved 96 percent accuracy for classifying 12 facial a ctions of the upper and lower face. The results provide converging evidence for the importance of using local filters, high spatial frequencies, and s tatistical independence for classifying facial actions.