Recognizing action units for facial expression analysis

Most automatic expression analysis systems attempt to recognize a small set of prototypic expressions, such as happiness, anger, surprise, and fear. S uch prototypic expressions, however, occur rather infrequently. Human emoti ons and intentions are more often communicated by changes in one or a few d iscrete facial features. In this paper, we develop an Automatic Face Analys is (AFA) system to analyze facial expressions based on both permanent facia l features (brows, eyes, mouth) and transient facial features (deepening of facial furrows) in a nearly frontal-view face image sequence. The AFA syst em recognizes fine-grained changes in facial expression into action units ( AUs) of the Facial Action Coding System (FACS), instead of a few prototypic expressions. Multistate face and facial component models are proposed for tracking and modeling the various facial features, including lips, eyes, br ews, cheeks, and furrows. During tracking, detailed parametric descriptions of the facial features are extracted. With these parameters as the inputs, a group of action units (neutral expression, six upper face AUs and 10 low er face AUs) are recognized whether they occur alone or in combinations. Th e system has achieved average recognition rates of 96.4 percent (95.4 perce nt if neutral expressions are excluded) for upper face AUs and 96.7 percent (95.6 percent with neutral expressions excluded) for lower face AUs. The g eneralizability of the system has been tested by using independent image da tabases collected and FAGS-coded for ground-truth by different research tea ms.