Visual and audiovisual speech perception with color and gray-scale facial images

Research has shown that auditory speech recognition is influenced by the ap pearance of a talker's face, but the actual nature of this visual informati on has yet to be established. Here, we report three experiments that invest igated visual and audiovisual speech recognition using color, gray-scale, a nd point-light talking faces (which allowed comparison with the influence o f isolated kinematic information). Auditory and visual forms of the syllabl es /ba/, /bi/, /ga/, /gi/, /va/, and /vi/ were used to produce auditory, vi sual, congruent, and incongruent audiovisual speech stimuli. Visual speech identification and visual influences on identifying the auditory components of congruent and incongruent audiovisual speech were identical for color a nd gray-scale faces and were much greater than for point-light faces. These results indicate that luminance, rather than color, underlies visual and a udiovisual speech perception and that this information is more than the kin ematic information provided by point-light faces. Implications for processi ng visual and audiovisual speech are discussed.