Effects of age and left hemisphere lesions on audiovisual integration of speech.
Ontology highlight
ABSTRACT:
SUBMITTER: Michaelis K
PROVIDER: S-EPMC7379161 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Speech perception is influenced by vision through a process of audiovisual integration. This is demonstrated by the McGurk illusion where visual speech (for example /ga/) dubbed with incongruent auditory speech (such as /ba/) leads to a modified auditory percept (/da/). Recent studies have indicated that perception of the incongruent speech stimuli used in McGurk paradigms involves mechanisms of both general and audiovisual speech specific mismatch processing and that general mismatch processing modulates induced theta-band (4-8 Hz) oscillations. Here, we investigated whether the theta modulation merely reflects mismatch processing or, alternatively, audiovisual integration of speech. We used electroencephalographic recordings from two previously published studies using audiovisual sine-wave speech (SWS), a spectrally degraded speech signal sounding nonsensical to naïve perceivers but perceived as speech by informed subjects. Earlier studies have shown that informed, but not naïve subjects integrate SWS phonetically with visual speech. In an N1/P2 event-related potential paradigm, we found a significant difference in theta-band activity between informed and naïve perceivers of audiovisual speech, suggesting that audiovisual integration modulates induced theta-band oscillations. In a McGurk mismatch negativity paradigm (MMN) where infrequent McGurk stimuli were embedded in a sequence of frequent audio-visually congruent stimuli we found no difference between congruent and McGurk stimuli. The infrequent stimuli in this paradigm are violating both the general prediction of stimulus content, and that of audiovisual congruence. Hence, we found no support for the hypothesis that audiovisual mismatch modulates induced theta-band oscillations. We also did not find any effects of audiovisual integration in the MMN paradigm, possibly due to the experimental design.
| S-EPMC6634411 | biostudies-literature
Project description:This psychophysics study used musicians as a model to investigate whether musical expertise shapes the temporal integration window for audiovisual speech, sinewave speech, or music. Musicians and non-musicians judged the audiovisual synchrony of speech, sinewave analogs of speech, and music stimuli at 13 audiovisual stimulus onset asynchronies (±360, ±300 ±240, ±180, ±120, ±60, and 0 ms). Further, we manipulated the duration of the stimuli by presenting sentences/melodies or syllables/tones. Critically, musicians relative to non-musicians exhibited significantly narrower temporal integration windows for both music and sinewave speech. Further, the temporal integration window for music decreased with the amount of music practice, but not with age of acquisition. In other words, the more musicians practiced piano in the past 3 years, the more sensitive they became to the temporal misalignment of visual and auditory signals. Collectively, our findings demonstrate that music practicing fine-tunes the audiovisual temporal integration window to various extents depending on the stimulus class. While the effect of piano practicing was most pronounced for music, it also generalized to other stimulus classes such as sinewave speech and to a marginally significant degree to natural speech.
| S-EPMC4124486 | biostudies-literature
Project description:ObjectiveTo investigate the role of multiple distributed brain networks, including the default mode, fronto-temporo-parietal, and cingulo-opercular networks, which mediate domain-general and task-specific processes during speech production after aphasic stroke.MethodsWe conducted an observational functional MRI study to investigate the effects of a previous left hemisphere stroke on functional connectivity within and between distributed networks as patients described pictures. Study design included various baseline tasks, and we compared results to those of age-matched healthy participants performing the same tasks. We used independent component and psychophysiological interaction analyses.ResultsAlthough activity within individual networks was not predictive of speech production, relative activity between networks was a predictor of both within-scanner and out-of-scanner language performance, over and above that predicted from lesion volume, age, sex, and years of education. Specifically, robust functional imaging predictors were the differential activity between the default mode network and both the left and right fronto-temporo-parietal networks, respectively activated and deactivated during speech. We also observed altered between-network functional connectivity of these networks in patients during speech production.ConclusionsSpeech production is dependent on complex interactions among widely distributed brain networks, indicating that residual speech production after stroke depends on more than the restoration of local domain-specific functions. Our understanding of the recovery of function following focal lesions is not adequately captured by consideration of ipsilesional or contralesional brain regions taking over lost domain-specific functions, but is perhaps best considered as the interaction between what remains of domain-specific networks and domain-general systems that regulate behavior.
| S-EPMC4826341 | biostudies-literature
Project description:Visual information about lip and facial movements plays a role in audiovisual (AV) speech perception. Although this has been widely confirmed, previous behavioural studies have shown interlanguage differences, that is, native Japanese speakers do not integrate auditory and visual speech as closely as native English speakers. To elucidate the neural basis of such interlanguage differences, 22 native English speakers and 24 native Japanese speakers were examined in behavioural or functional Magnetic Resonance Imaging (fMRI) experiments while mono-syllabic speech was presented under AV, auditory-only, or visual-only conditions for speech identification. Behavioural results indicated that the English speakers identified visual speech more quickly than the Japanese speakers, and that the temporal facilitation effect of congruent visual speech was significant in the English speakers but not in the Japanese speakers. Using fMRI data, we examined the functional connectivity among brain regions important for auditory-visual interplay. The results indicated that the English speakers had significantly stronger connectivity between the visual motion area MT and the Heschl's gyrus compared with the Japanese speakers, which may subserve lower-level visual influences on speech perception in English speakers in a multisensory environment. These results suggested that linguistic experience strongly affects neural connectivity involved in AV speech integration.
| S-EPMC4980767 | biostudies-literature
Project description:Emotional speech perception is a multisensory process. When speaking with an individual we concurrently integrate the information from their voice and face to decode e.g., their feelings, moods, and emotions. However, the physiological reactions-such as the reflexive dilation of the pupil-associated to these processes remain mostly unknown. That is the aim of the current article, to investigate whether pupillary reactions can index the processes underlying the audiovisual integration of emotional signals. To investigate this question, we used an algorithm able to increase or decrease the smiles seen in a person's face or heard in their voice, while preserving the temporal synchrony between visual and auditory channels. Using this algorithm, we created congruent and incongruent audiovisual smiles, and investigated participants' gaze and pupillary reactions to manipulated stimuli. We found that pupil reactions can reflect emotional information mismatch in audiovisual speech. In our data, when participants were explicitly asked to extract emotional information from stimuli, the first fixation within emotionally mismatching areas (i.e., the mouth) triggered pupil dilation. These results reveal that pupil dilation can reflect the dynamic integration of audiovisual emotional speech and provide insights on how these reactions are triggered during stimulus perception.
| S-EPMC10073148 | biostudies-literature
Project description:Research in audiovisual speech perception has demonstrated that sensory factors such as auditory and visual acuity are associated with a listener's ability to extract and combine auditory and visual speech cues. This case study report examined audiovisual integration using a newly developed measure of capacity in a sample of hearing-impaired listeners. Capacity assessments are unique because they examine the contribution of reaction-time (RT) as well as accuracy to determine the extent to which a listener efficiently combines auditory and visual speech cues relative to independent race model predictions. Multisensory speech integration ability was examined in two experiments: an open-set sentence recognition and a closed set speeded-word recognition study that measured capacity. Most germane to our approach, capacity illustrated speed-accuracy tradeoffs that may be predicted by audiometric configuration. Results revealed that some listeners benefit from increased accuracy, but fail to benefit in terms of speed on audiovisual relative to unisensory trials. Conversely, other listeners may not benefit in the accuracy domain but instead show an audiovisual processing time benefit.
| S-EPMC4076931 | biostudies-literature
Project description:The brain improves speech processing through the integration of audiovisual (AV) signals. Situations involving AV speech integration may be crudely dichotomized into those where auditory and visual inputs contain (1) equivalent, complementary signals (validating AV speech) or (2) inconsistent, different signals (conflicting AV speech). This simple framework may allow the systematic examination of broad commonalities and differences between AV neural processes engaged by various experimental paradigms frequently used to study AV speech integration. We conducted an activation likelihood estimation metaanalysis of 22 functional imaging studies comprising 33 experiments, 311 subjects, and 347 foci examining "conflicting" versus "validating" AV speech. Experimental paradigms included content congruency, timing synchrony, and perceptual measures, such as the McGurk effect or synchrony judgments, across AV speech stimulus types (sublexical to sentence). Colocalization of conflicting AV speech experiments revealed consistency across at least two contrast types (e.g., synchrony and congruency) in a network of dorsal stream regions in the frontal, parietal, and temporal lobes. There was consistency across all contrast types (synchrony, congruency, and percept) in the bilateral posterior superior/middle temporal cortex. Although fewer studies were available, validating AV speech experiments were localized to other regions, such as ventral stream visual areas in the occipital and inferior temporal cortex. These results suggest that while equivalent, complementary AV speech signals may evoke activity in regions related to the corroboration of sensory input, conflicting AV speech signals recruit widespread dorsal stream areas likely involved in the resolution of conflicting sensory signals.
| S-EPMC4206592 | biostudies-literature
Project description:Previous studies confirmed that the cognitive resources are limited for each person, and perceptual load affects the detection of stimulus greatly; however, how the visual perceptual load influences audiovisual integration (AVI) is still unclear. Here, 20 older and 20 younger adults were recruited to perform an auditory/visual discrimination task under various visual perceptual-load conditions. The analysis for the response times revealed a significantly faster response to the audiovisual stimulus than to the visual stimulus or auditory stimulus (all p < 0.001), and a significantly slower response by the older adults than by the younger adults to all targets (all p ≤ 0.024). The race-model analysis revealed a higher AV facilitation effect for older (12.54%) than for younger (7.08%) adults under low visual perceptual-load conditions; however, no obvious difference was found between younger (2.92%) and older (3.06%) adults under medium visual perceptual-load conditions. Only the AV depression effect was found for both younger and older adults under high visual perceptual-load conditions. Additionally, the peak latencies of AVI were significantly delayed in older adults under all visual perceptual-load conditions. These results suggested that visual perceptual load decreased AVI (i.e., depression effects), and the AVI effect was increased but delayed for older adults.
| S-EPMC8511317 | biostudies-literature
Project description:Audiovisual (AV) speech perception is the process by which auditory and visual sensory signals are integrated and used to understand what a talker is saying during face-to-face communication. This form of communication is markedly superior to speech perception in either sensory modality alone. However, there are additional lexical factors that are affected by age-related cognitive changes that may contribute to differences in AV perception. In the current study, we extended an existing model of spoken word identification to the AV domain, and examined the cognitive factors that contribute to age-related and individual differences in AV perception of words varying in lexical difficulty (i.e., on the basis of competing items). Young (n = 49) and older adults (n = 50) completed a series of cognitive inhibition tasks and a spoken word identification task. The words were presented in auditory-only, visual-only, and AV conditions, and were equally divided into lexically hard (words with many competitors) and lexically easy (words with few competitors). Overall, young adults demonstrated better inhibitory abilities and higher identification performance than older adults. However, whereas no relationship was observed between inhibitory abilities and AV word identification performance in young adults, there was a significant relationship between Stroop interference and AV identification of lexically hard words in older adults. These results are interpreted within the framework of existing models of spoken-word recognition with implications for how cognitive deficits in older adults contribute to speech perception.
| S-EPMC5757834 | biostudies-literature
Project description:Background:Individuals with Autism Spectrum Disorders (ASD) have been shown to have multisensory integration deficits, which may lead to problems perceiving complex, multisensory environments. For example, understanding audiovisual speech requires integration of visual information from the lips and face with auditory information from the voice, and audiovisual speech integration deficits can lead to impaired understanding and comprehension. While there is strong evidence for an audiovisual speech integration impairment in ASD, it is unclear whether this impairment is due to low level perceptual processes that affect all types of audiovisual integration or if it is specific to speech processing. Method:Here, we measure audiovisual integration of basic speech (i.e., consonant-vowel utterances) and object stimuli (i.e., a bouncing ball) in adolescents with ASD and well-matched controls. We calculate a temporal window of integration (TWI) using each individual's ability to identify which of two videos (one temporally aligned and one misaligned) matches auditory stimuli. The TWI measures tolerance for temporal asynchrony between the auditory and visual streams, and is an important feature of audiovisual perception. Results:While controls showed similar tolerance of asynchrony for the simple speech and object stimuli, individuals with ASD did not. Specifically, individuals with ASD showed less tolerance of asynchrony for speech stimuli compared to object stimuli. In individuals with ASD, decreased tolerance for asynchrony in speech stimuli was associated with higher ratings of autism symptom severity. Conclusions:These results suggest that audiovisual perception in ASD may vary for speech and object stimuli beyond what can be accounted for by stimulus complexity.
| S-EPMC6135104 | biostudies-literature