Visuo-haptic integration process during object size discrimination: an fMRI study

Abstract

When exploring objects during every day activities, visual and haptic cues provide information about their properties such as size, shape and texture. How these two streams of sensory information are integrated by the brain to form a single percept is still not fully understood. Specifically, there is still a debate which regions of the brain are activated during visuo-haptic perception and whether one can identify areas that are uniquely activated during visuo-haptic integration, but not during unimodal perception. Previous research indicated that the lateral occipital complex (LOC) might be such a region. In an attempt to fill this knowledge gap, this study investigated the cortical activation patterns that underlie object size perception based unimodal visual or haptic information, and when both forms of information were available (bimodal). Brain imaging data were obtained from 12 healthy participants in a size perception task using visual and haptic stimuli. In each trial, a 6 cm reference object was judged against a comparison object (heights: 5.2-6.8 cm) and participants verbally indicated which object was perceived as taller. Unimodal and bimodal stimulus presentations were tested. Based on their responses size discrimination thresholds (DT) at the 75% correct response rate were obtained. In addition, functional brain activation volumes were derived during each condition for contrast analysis. Thresholds and associated correct response rates were not significantly different between unimodal and bimodal conditions. Bimodal visuo-haptic size discrimination was associated with super-additive activation in the dorsolateral prefrontal cortex (DLPFC), supplementary motor area (SMA) and inferior parietal cortices, while the LOC revealed no significant activation. In addition, during crossmodal stimulation the left SMA and primary sensorimotor cortex revealed significant activations when the visual cue was presented first. That is, during crossmodal visuo-haptic object perception having a prior visual experience activates different areas than having a prior haptic experience of the object. To our knowledge, this is the first direct comparison of unimodal and bimodal visuohaptic size perception that combined brain imaging and psychophysical data to identify sites of multisensory integration. The finding of a fronto-parietal network involved in visuo-haptic processing challenges a previous notion of the LOC as the sole locus of visuo-haptic integration. Our results suggest that the underlying visuo-haptic perceptual process seems to be task dependent.