This thesis aimed to find hemodynamic and electrophysiological correlates of the perceptual binding of crossmodal audiovisual stimuli in the human brain. How the multisensory convergence areas and the temporal correlation (synchronization) of areas are involved in binding is currently a major discussion in multisensory research. Therefore, the question of central interest in the present work was the relationship between the correlates in the functional magnet resonance imaging (fMRI) and the simultaneously acquired electroencephalogram (EEG). Additionally to the separated analyses of both signals in a further analysis fMRI was examined guided by the EEG, i.e. the fMRI signal was parametrically modulated by the quantified EEG signal. The perceptual binding of crossmodal audiovisual stimuli was addressed in a paradigm using the ventriloquism-effect. Participants were asked to report the temporal congruency and location of auditory stimuli when observing both visual and auditory stimuli of low complexity that were switched on/off either synchronously or asynchronously. In synchronously presented spatially disparate audiovisual stimuli the ventriloquism-effect is indicated by a localization shift of the subjective origin of the auditory stimulus toward the visual stimulus. Crossmodal binding was introduced with temporally congruent audiovisual stimuli eliciting the ventriloquism-effect, whereas temporally incongruent audiovisual stimuli without a localization shift were the bimodal control. In a network of multisensory convergence areas hemodynamic activity correlating with crossmodal binding was found in the insula, the parieto-occipital sulcus (POS), and the superior temporal sulcus (STS). At parietal electrodes an electrophysiological correlate of crossmodal binding was identified in neural synchronization changes of theta and alpha band activity. The time-frequency representations showed different patterns of synchronization for the congruent and the incongruent stimulation. The perceptual binding of crossmodal audiovisual stimuli was reflected in hemodynamic and in electrophysiological brain activity. On a single trial basis, the quantified EEG correlate was used to predict the hemodynamic signal in the EEG-guided fMRI analysis. This modulation of the hemodynamic signal gave information about the association of cortical areas and EEG dynamic within the framework of crossmodal binding. The association is of special interest in the context of the two hypotheses of binding, binding by hierarchically organized convergence areas and binding by temporal correlation. As the main result of the EEG-guided fMRI analysis a very distinct area associated with increased synchronization was related to the perceptual binding of crossmodal audiovisual stimuli. When this activation in the STS is related to projections from or to unisensory areas as discussed in the literature, the two hypotheses of crossmodal binding could be combined for audiovisual binding. The association revealed in the STS provides evidence for the STS to serve as the modulating convergence area of audiovisual binding.
Verknüpfung zu Publikationen oder weiteren Datensätzen
