The combined perception of emotion from voice and face: early interaction revealed by human electric brain responses

Judgement of the emotional tone of a spoken utterance is influenced by a si multaneously presented face expression. The time course of this integration was investigated by measuring the mismatch negativity (MMN). In one condit ion, the standard stimulus was an angry voice fragment combined with a (con gruous) angry face expression. In the deviant pair, the voice expression wa s kept the same and only the face expression changed to an (incongruous) sa d face. The pairs with a deviant visual item evoked a negative electric bra in response showing the characteristics of the MMN, which is usually evoked only by auditory deviations. Similar results were obtained by employing in congruous standard and congruous deviant pairs. These findings provide comp elling evidence of an early integration of face with voice information in t he processing of affect. (C) 1999 Elsevier Science Ireland Ltd. All rights reserved.