Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex

Background: Integrating information from the different senses markedly enha nces the detection and identification of external stimuli. Compared with un imodal inputs, semantically and/or spatially congruent multisensory cues sp eed discrimination and improve reaction times. Discordant inputs have the o pposite effect, reducing performance and slowing responses. These behaviour al features of crossmodal processing appear to have parallels in the respon se properties of multisensory cells in the superior colliculi and cerebral cortex of non-human mammals. Although spatially concordant multisensory inp uts can produce a dramatic, often multiplicative, increase in cellular acti vity, spatially disparate cues tend to induce a profound response depressio n. Results: Using functional magnetic resonance imaging (fMRI), we investigate d whether similar indices of crossmodal integration are detectable in human cerebral cortex, and for the synthesis of complex inputs relating to stimu lus identity, Ten human subjects were exposed to varying epochs of Semantic ally congruent and incongruent audio-visual speech and to each modality in isolation. Brain activations to matched and mismatched audio-visual inputs were contrasted with the combined response to both unimodal conditions. Thi s strategy identified an area of heteromodal cortex in the left superior te mporal sulcus that exhibited significant supra-additive response enhancemen t to matched audio-visual inputs and a corresponding sub-additive response to mismatched inputs. Conclusions: The data provide fMRI evidence of crossmodal binding by conver gence in the human heteromodal cortex. They further suggest that response e nhancement and depression may be a general property of multisensory integra tion operating at different levels of the neuroaxis and irrespective of the purpose for which sensory inputs are combined.