Semantic and spatial congruency mould audiovisual integration depending on perceptual awareness.
Ontology highlight
ABSTRACT:
SUBMITTER: Delong P
PROVIDER: S-EPMC8149651 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The ability to sense temperature is vital to our life. It signals the environmental condition, reflects the physiological conditions of our own body, and generates feelings of pleasantness or unpleasantness. Moreover, recent studies have demonstrated implicit associations between physical temperature and social/emotional concepts, suggesting the processing of temperature may even influence cognition. In this work, we examined the effect of physical warmth and coldness on semantic cognition. Participants performed speeded target categorization for thermal descriptors in the form of semantic words or illustrative figures representing the thermal qualities "warm" or "cold" while physical thermal stimulation was presented. We compared the average reaction time (RT) for the congruent and incongruent conditions managed by response key assignments. In the congruent condition, the response key for the symbol associated with warmth (coldness) was assigned to the hand with warm (cold) thermal stimulation, and in the incongruent condition the key assignment was reversed. Our results demonstrate that the average RT in the congruent condition was faster than in the incongruent one for both forms of thermal descriptors, suggesting that the experience of physical temperature facilitates the internal processing of the meaning of thermal quality.
| S-EPMC5724365 | biostudies-literature
Project description:Previous studies confirmed that the cognitive resources are limited for each person, and perceptual load affects the detection of stimulus greatly; however, how the visual perceptual load influences audiovisual integration (AVI) is still unclear. Here, 20 older and 20 younger adults were recruited to perform an auditory/visual discrimination task under various visual perceptual-load conditions. The analysis for the response times revealed a significantly faster response to the audiovisual stimulus than to the visual stimulus or auditory stimulus (all p < 0.001), and a significantly slower response by the older adults than by the younger adults to all targets (all p ≤ 0.024). The race-model analysis revealed a higher AV facilitation effect for older (12.54%) than for younger (7.08%) adults under low visual perceptual-load conditions; however, no obvious difference was found between younger (2.92%) and older (3.06%) adults under medium visual perceptual-load conditions. Only the AV depression effect was found for both younger and older adults under high visual perceptual-load conditions. Additionally, the peak latencies of AVI were significantly delayed in older adults under all visual perceptual-load conditions. These results suggested that visual perceptual load decreased AVI (i.e., depression effects), and the AVI effect was increased but delayed for older adults.
| S-EPMC8511317 | biostudies-literature
Project description:An audiovisual object may contain multiple semantic features, such as the gender and emotional features of the speaker. Feature-selective attention and audiovisual semantic integration are two brain functions involved in the recognition of audiovisual objects. Humans often selectively attend to one or several features while ignoring the other features of an audiovisual object. Meanwhile, the human brain integrates semantic information from the visual and auditory modalities. However, how these two brain functions correlate with each other remains to be elucidated. In this functional magnetic resonance imaging (fMRI) study, we explored the neural mechanism by which feature-selective attention modulates audiovisual semantic integration. During the fMRI experiment, the subjects were presented with visual-only, auditory-only, or audiovisual dynamical facial stimuli and performed several feature-selective attention tasks. Our results revealed that a distribution of areas, including heteromodal areas and brain areas encoding attended features, may be involved in audiovisual semantic integration. Through feature-selective attention, the human brain may selectively integrate audiovisual semantic information from attended features by enhancing functional connectivity and thus regulating information flows from heteromodal areas to brain areas encoding the attended features.
| S-EPMC4725371 | biostudies-literature
Project description:To recognize individuals, the brain often integrates audiovisual information from familiar or unfamiliar faces, voices, and auditory names. To date, the effects of the semantic familiarity of stimuli on audiovisual integration remain unknown. In this functional magnetic resonance imaging (fMRI) study, we used familiar/unfamiliar facial images, auditory names, and audiovisual face-name pairs as stimuli to determine the influence of semantic familiarity on audiovisual integration. First, we performed a general linear model analysis using fMRI data and found that audiovisual integration occurred for familiar congruent and unfamiliar face-name pairs but not for familiar incongruent pairs. Second, we decoded the familiarity categories of the stimuli (familiar vs. unfamiliar) from the fMRI data and calculated the reproducibility indices of the brain patterns that corresponded to familiar and unfamiliar stimuli. The decoding accuracy rate was significantly higher for familiar congruent versus unfamiliar face-name pairs (83.2%) than for familiar versus unfamiliar faces (63.9%) and for familiar versus unfamiliar names (60.4%). This increase in decoding accuracy was not observed for familiar incongruent versus unfamiliar pairs. Furthermore, compared with the brain patterns associated with facial images or auditory names, the reproducibility index was significantly improved for the brain patterns of familiar congruent face-name pairs but not those of familiar incongruent or unfamiliar pairs. Our results indicate the modulatory effect that semantic familiarity has on audiovisual integration. Specifically, neural representations were enhanced for familiar congruent face-name pairs compared with visual-only faces and auditory-only names, whereas this enhancement effect was not observed for familiar incongruent or unfamiliar pairs. Hum Brain Mapp 37:4333-4348, 2016. © 2016 Wiley Periodicals, Inc.
| S-EPMC6867328 | biostudies-literature
Project description:Multisensory perception is regarded as one of the most prominent examples where human behaviour conforms to the computational principles of maximum likelihood estimation (MLE). In particular, observers are thought to integrate auditory and visual spatial cues weighted in proportion to their relative sensory reliabilities into the most reliable and unbiased percept consistent with MLE. Yet, evidence to date has been inconsistent. The current pre-registered, large-scale (N = 36) replication study investigated the extent to which human behaviour for audiovisual localization is in line with maximum likelihood estimation. The acquired psychophysics data show that while observers were able to reduce their multisensory variance relative to the unisensory variances in accordance with MLE, they weighed the visual signals significantly stronger than predicted by MLE. Simulations show that this dissociation can be explained by a greater sensitivity of standard estimation procedures to detect deviations from MLE predictions for sensory weights than for audiovisual variances. Our results therefore suggest that observers did not integrate audiovisual spatial signals weighted exactly in proportion to their relative reliabilities for localization. These small deviations from the predictions of maximum likelihood estimation may be explained by observers' uncertainty about the world's causal structure as accounted for by Bayesian causal inference.
| S-EPMC6864592 | biostudies-literature
Project description:To clarify the role of sensory experience during early development for adult multisensory learning capabilities, we probed audiovisual spatial processing in human individuals who had been born blind because of dense congenital cataracts (CCs) and who subsequently had received cataract removal surgery, some not before adolescence or adulthood. Their ability to integrate audio-visual input and to recalibrate multisensory spatial representations was compared to normally sighted control participants and individuals with a history of developmental (later onset) cataracts. Results in CC individuals revealed both normal multisensory integration in audiovisual trials (ventriloquism effect) and normal recalibration of unimodal auditory localization following audiovisual discrepant exposure (ventriloquism aftereffect) as observed in the control groups. In addition, only the CC group recalibrated unimodal visual localization after audiovisual exposure. Thus, in parallel to typical multisensory integration and learning, atypical crossmodal mechanisms coexisted in CC individuals, suggesting that multisensory recalibration capabilities are defined during a sensitive period in development.
| S-EPMC9301879 | biostudies-literature
Project description:Learning to associate written letters/characters with speech sounds is crucial for reading acquisition. Most previous studies have focused on audiovisual integration in alphabetic languages. Less is known about logographic languages such as Chinese characters, which map onto mostly syllable-based morphemes in the spoken language. Here we investigated how long-term exposure to native language affects the underlying neural mechanisms of audiovisual integration in a logographic language using magnetoencephalography (MEG). MEG sensor and source data from 12 adult native Chinese speakers and a control group of 13 adult Finnish speakers were analyzed for audiovisual suppression (bimodal responses vs. sum of unimodal responses) and congruency (bimodal incongruent responses vs. bimodal congruent responses) effects. The suppressive integration effect was found in the left angular and supramarginal gyri (205-365 ms), left inferior frontal and left temporal cortices (575-800 ms) in the Chinese group. The Finnish group showed a distinct suppression effect only in the right parietal and occipital cortices at a relatively early time window (285-460 ms). The congruency effect was only observed in the Chinese group in left inferior frontal and superior temporal cortex in a late time window (about 500-800 ms) probably related to modulatory feedback from multi-sensory regions and semantic processing. The audiovisual integration in a logographic language showed a clear resemblance to that in alphabetic languages in the left superior temporal cortex, but with activation specific to the logographic stimuli observed in the left inferior frontal cortex. The current MEG study indicated that learning of logographic languages has a large impact on the audiovisual integration of written characters with some distinct features compared to previous results on alphabetic languages.
| S-EPMC6372538 | biostudies-literature
Project description:It has been reported that attending to stimuli in visual modality can spread to task-irrelevant but synchronously presented stimuli in auditory modality, a phenomenon termed the cross-modal spread of attention, which could be either stimulus-driven or representation-driven depending on whether the visual constituent of an audiovisual object is further selected based on the object representation. The stimulus-driven spread of attention occurs whenever a task-irrelevant sound synchronizes with an attended visual stimulus, regardless of the cross-modal semantic congruency. The present study recorded event-related potentials (ERPs) to investigate whether the stimulus-driven cross-modal spread of attention could be modulated by audio-visual emotional congruency in a visual oddball task where emotion (positive/negative) was task-irrelevant. The results first demonstrated a prominent stimulus-driven spread of attention regardless of audio-visual emotional congruency by showing that for all audiovisual pairs, the extracted ERPs to the auditory constituents of audiovisual stimuli within the time window of 200-300 ms were significantly larger than ERPs to the same auditory stimuli delivered alone. However, the amplitude of this stimulus-driven auditory Nd component during 200-300 ms was significantly larger for emotionally incongruent than congruent audiovisual stimuli when their visual constituents' emotional valences were negative. Moreover, the Nd was sustained during 300-400 ms only for the incongruent audiovisual stimuli with emotionally negative visual constituents. These findings suggest that although the occurrence of the stimulus-driven cross-modal spread of attention is independent of audio-visual emotional congruency, its magnitude is nevertheless modulated even when emotion is task-irrelevant.
| S-EPMC9497153 | biostudies-literature
Project description:The flourishing of studies on the neural correlates of decision-making calls for an appraisal of the relation between perceptual decisions and conscious perception. By exploiting the long integration time of noisy motion stimuli, and by forcing human observers to make difficult speeded decisions--sometimes a blind guess--about stimulus direction, we traced the temporal buildup of motion discrimination capability and perceptual awareness, as assessed trial by trial through direct rating. We found that both increased gradually with motion coherence and viewing time, but discrimination was systematically leading awareness, reaching a plateau much earlier. Sensitivity and criterion changes contributed jointly to the slow buildup of perceptual awareness. It made no difference whether motion discrimination was accomplished by saccades or verbal responses. These findings suggest that perceptual awareness emerges on the top of a developing or even mature perceptual decision. We argue that the middle temporal (MT) cortical region does not confer us the full phenomenic depth of motion perception, although it may represent a precursor stage in building our subjective sense of visual motion.
| S-EPMC3040746 | biostudies-literature
Project description:Behavioral research consistently shows that congruous events, that is, events whose constituent elements match along some specific dimension, are better remembered than incongruous events. Although it has been speculated that this "congruency subsequent memory effect" (cSME) results from enhanced semantic elaboration, empirical evidence for this account is lacking. Here, we report a set of behavioral and neuroimaging experiments demonstrating that congruous events engage regions along the left inferior frontal gyrus (LIFG)--consistently related to semantic elaboration--to a significantly greater degree than incongruous events, providing evidence in favor of this hypothesis. Critically, we additionally report 3 novel findings in relation to event congruency: First, congruous events yield superior memory not only for a given study item but also for associated source details. Second, the cSME is evident not only for events that matched a semantic context but also for those that matched a subjective aesthetic schema. Finally, functional magnetic resonance imaging brain/behavior correlation analysis reveals a strong link between 1) across-subject variation in the magnitude of the cSME and 2) differential right hippocampal activation, suggesting that episodic memory for congruous events is effectively bolstered by the extent to which semantic associations are generated and relationally integrated via LIFG-hippocampal-encoding mechanisms.
| S-EPMC2665161 | biostudies-other