“What” versus “Where” in the audiovisual domain: An fMRI study

• Published in: NeuroImage (Orlando, Fla.) Vol. 33; no. 2; pp. 672 - 680
• Main Authors: Sestieri, C., Di Matteo, R., Ferretti, A., Del Gratta, C., Caulo, M., Tartaro, A., Olivetti Belardinelli, M., Romani, G.L.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01-11-2006; Elsevier Limited
• Subjects: Acoustic Stimulation; Adult; Audiovisual; Auditory Perception; Brain; Brain Mapping - methods; Brain research; Crossmodal; fMRI; Functional Laterality; Humans; Magnetic Resonance Imaging - methods; Male; Photic Stimulation; Reaction Time; Recognition (Psychology); Semantics; Sound Localization - physiology; Space Perception - physiology; Specialization; Speech; Studies; Visual Perception; What; Where; Audiovisual; fMRI; What; Where; Crossmodal
• ISSN: 1053-8119; 1095-9572
• DOI: 10.1016/j.neuroimage.2006.06.045

Similar “what/where” functional segregations have been proposed for both visual and auditory cortical processing. In this fMRI study, we investigated if the same segregation exists in the crossmodal domain, when visual and auditory stimuli have to be matched in order to perform either a recognition or a localization task. Recent neuroimaging research highlighted the contribution of different heteromodal cortical regions during various forms of crossmodal binding. Interestingly, crossmodal effects during audiovisual speech and object recognition have been found in the superior temporal sulcus, while crossmodal effects during the execution of spatial tasks have been found over the intraparietal sulcus, suggesting an underlying “what/where” segregation. In order to directly compare the specific involvement of these two heteromodal regions, we scanned ten male right-handed subjects during the execution of two crossmodal matching tasks. Participants were simultaneously presented with a picture and an environmental sound, coming from either the same or the opposite hemifield and representing either the same or a different object. The two tasks required a manual YES/NO response respectively about location or semantic matching of the presented stimuli. Both group and individual subject analysis were performed. Task-related differences in BOLD response were observed in the right intraparietal sulcus and in the left superior temporal sulcus, providing a direct confirmation of the “what–where” functional segregation in the crossmodal audiovisual domain.