Measurement and Prediction of Binaural-Temporal Integration of Speech Reflections.
Ontology highlight
ABSTRACT:
SUBMITTER: Rennies J
PROVIDER: S-EPMC6593929 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The auditory system has exquisite temporal coding in the periphery which is transformed into a rate-based code in central auditory structures, like auditory cortex. However, the cortex is still able to synchronize, albeit at lower modulation rates, to acoustic fluctuations. The perceptual significance of this cortical synchronization is unknown. We estimated physiological synchronization limits of cortex (in humans with electroencephalography) and brainstem neurons (in chinchillas) to dynamic binaural cues using a novel system-identification technique, along with parallel perceptual measurements. We find that cortex can synchronize to dynamic binaural cues up to approximately 10 Hz, which aligns well with our measured limits of perceiving dynamic spatial information and utilizing dynamic binaural cues for spatial unmasking, i.e. measures of binaural sluggishness. We also find that the tracking limit for frequency modulation (FM) is similar to the limit for spatial tracking, demonstrating that this sluggish tracking is a more general perceptual limit that can be accounted for by cortical temporal integration limits.
| S-EPMC10522716 | biostudies-literature
Project description:The equalization cancellation model is often used to predict the binaural masking level difference. Previously its application to speech in noise has required separate knowledge about the speech and noise signals to maximize the signal-to-noise ratio (SNR). Here, a novel, blind equalization cancellation model is introduced that can use the mixed signals. This approach does not require any assumptions about particular sound source directions. It uses different strategies for positive and negative SNRs, with the switching between the two steered by a blind decision stage utilizing modulation cues. The output of the model is a single-channel signal with enhanced SNR, which we analyzed using the speech intelligibility index to compare speech intelligibility predictions. In a first experiment, the model was tested on experimental data obtained in a scenario with spatially separated target and masker signals. Predicted speech recognition thresholds were in good agreement with measured speech recognition thresholds with a root mean square error less than 1?dB. A second experiment investigated signals at positive SNRs, which was achieved using time compressed and low-pass filtered speech. The results demonstrated that binaural unmasking of speech occurs at positive SNRs and that the modulation-based switching strategy can predict the experimental results.
| S-EPMC7734536 | biostudies-literature
Project description:This psychophysics study used musicians as a model to investigate whether musical expertise shapes the temporal integration window for audiovisual speech, sinewave speech, or music. Musicians and non-musicians judged the audiovisual synchrony of speech, sinewave analogs of speech, and music stimuli at 13 audiovisual stimulus onset asynchronies (±360, ±300 ±240, ±180, ±120, ±60, and 0 ms). Further, we manipulated the duration of the stimuli by presenting sentences/melodies or syllables/tones. Critically, musicians relative to non-musicians exhibited significantly narrower temporal integration windows for both music and sinewave speech. Further, the temporal integration window for music decreased with the amount of music practice, but not with age of acquisition. In other words, the more musicians practiced piano in the past 3 years, the more sensitive they became to the temporal misalignment of visual and auditory signals. Collectively, our findings demonstrate that music practicing fine-tunes the audiovisual temporal integration window to various extents depending on the stimulus class. While the effect of piano practicing was most pronounced for music, it also generalized to other stimulus classes such as sinewave speech and to a marginally significant degree to natural speech.
| S-EPMC4124486 | biostudies-literature
Project description:Temporal resolution needed for Japanese speech communication was measured. A new experimental paradigm that can reflect the spectro-temporal resolution necessary for healthy listeners to perceive speech is introduced. As a first step, we report listeners' intelligibility scores of Japanese speech with a systematically degraded temporal resolution, so-called "mosaic speech": speech mosaicized in the coordinates of time and frequency. The results of two experiments show that mosaic speech cut into short static segments was almost perfectly intelligible with a temporal resolution of 40 ms or finer. Intelligibility dropped for a temporal resolution of 80 ms, but was still around 50%-correct level. The data are in line with previous results showing that speech signals separated into short temporal segments of <100 ms can be remarkably robust in terms of linguistic-content perception against drastic manipulations in each segment, such as partial signal omission or temporal reversal. The human perceptual system thus can extract meaning from unexpectedly rough temporal information in speech. The process resembles that of the visual system stringing together static movie frames of ~40 ms into vivid motion.
| S-EPMC5928238 | biostudies-literature
Project description:Speech perception in noise is challenging and is improved by binaural hearing. Since signal processing of assistive hearing devices often modifies or masks the peripheral binaural head-shadow or better-ear effects, central binaural processing should be measured separately. In a prospective study, 10 listeners with normal hearing were tested with the German matrix sentence test in a set-up with two loudspeakers located at opposite angles in the horizontal plane with respect to S0N0. The speech reception threshold (SRT) was investigated depending on the separation angle between speech and noise. The lowest (best) SRT was obtained for a separation of target and interfering source from S0N0 at an angle of about S±60°N∓60°. The derived normative curve was comparable to SRTs predicted by the binaural-speech-intelligibility-model. The systematic separation of signal and noise showed a significant improvement in speech intelligibility for normal-hearing people even for small separation angles. This experimental setting was verified. This study aimed to assess the effect of small sound source separation on binaural hearing and speech perception.
| S-EPMC7406049 | biostudies-literature
Project description:Background:Individuals with Autism Spectrum Disorders (ASD) have been shown to have multisensory integration deficits, which may lead to problems perceiving complex, multisensory environments. For example, understanding audiovisual speech requires integration of visual information from the lips and face with auditory information from the voice, and audiovisual speech integration deficits can lead to impaired understanding and comprehension. While there is strong evidence for an audiovisual speech integration impairment in ASD, it is unclear whether this impairment is due to low level perceptual processes that affect all types of audiovisual integration or if it is specific to speech processing. Method:Here, we measure audiovisual integration of basic speech (i.e., consonant-vowel utterances) and object stimuli (i.e., a bouncing ball) in adolescents with ASD and well-matched controls. We calculate a temporal window of integration (TWI) using each individual's ability to identify which of two videos (one temporally aligned and one misaligned) matches auditory stimuli. The TWI measures tolerance for temporal asynchrony between the auditory and visual streams, and is an important feature of audiovisual perception. Results:While controls showed similar tolerance of asynchrony for the simple speech and object stimuli, individuals with ASD did not. Specifically, individuals with ASD showed less tolerance of asynchrony for speech stimuli compared to object stimuli. In individuals with ASD, decreased tolerance for asynchrony in speech stimuli was associated with higher ratings of autism symptom severity. Conclusions:These results suggest that audiovisual perception in ASD may vary for speech and object stimuli beyond what can be accounted for by stimulus complexity.
| S-EPMC6135104 | biostudies-literature
Project description:Background: Advances in machine learning (ML) technology have opened new avenues for detection and monitoring of cognitive decline. In this study, a multimodal approach to Alzheimer's dementia detection based on the patient's spontaneous speech is presented. This approach was tested on a standard, publicly available Alzheimer's speech dataset for comparability. The data comprise voice samples from 156 participants (1:1 ratio of Alzheimer's to control), matched by age and gender. Materials and Methods: A recently developed Active Data Representation (ADR) technique for voice processing was employed as a framework for fusion of acoustic and textual features at sentence and word level. Temporal aspects of textual features were investigated in conjunction with acoustic features in order to shed light on the temporal interplay between paralinguistic (acoustic) and linguistic (textual) aspects of Alzheimer's speech. Combinations between several configurations of ADR features and more traditional bag-of-n-grams approaches were used in an ensemble of classifiers built and evaluated on a standardised dataset containing recorded speech of scene descriptions and textual transcripts. Results: Employing only semantic bag-of-n-grams features, an accuracy of 89.58% was achieved in distinguishing between Alzheimer's patients and healthy controls. Adding temporal and structural information by combining bag-of-n-grams features with ADR audio/textual features, the accuracy could be improved to 91.67% on the test set. An accuracy of 93.75% was achieved through late fusion of the three best feature configurations, which corresponds to a 4.7% improvement over the best result reported in the literature for this dataset. Conclusion: The proposed combination of ADR audio and textual features is capable of successfully modelling temporal aspects of the data. The machine learning approach toward dementia detection achieves best performance when ADR features are combined with strong semantic bag-of-n-grams features. This combination leads to state-of-the-art performance on the AD classification task.
| S-EPMC8236853 | biostudies-literature
Project description:Simultaneity judgments were used to measure temporal binding windows (TBW) for brief binaural events (changes in interaural time and/or level differences [ITD and ILD]) and test the hypothesis that ITD and ILD contribute to perception via separate sensory dimensions subject to binding via slow (100+ ms)-presumably cortical-mechanisms as in multisensory TBW. Stimuli were continuous low-frequency noises that included two brief shifts of either type (ITD or ILD), both of which are heard as lateral position changes. TBW for judgments within a single cue dimension were narrower for ITD (mean = 444 ms) than ILD (807 ms). TBW for judgments across cue dimensions (i.e., one ITD shift and one ILD shift) were similar to within-cue ILD (778 ms). The results contradict the original hypothesis, in that cross-cue comparisons were no slower than within-cue ILD comparisons. Rather, the wide TBW values-consistent with previous estimates of multisensory TBW-suggest slow integrative processing for both types of judgments. Narrower TBW for ITD than ILD judgments suggests important cue-specific differences in the neural mechanisms or the perceptual correlates of integration across binaural-cue dimensions.
| S-EPMC5735209 | biostudies-literature
Project description:ObjectivesThis study was designed to evaluate binaural temporal processing in young and older adults using a binaural masking level difference (BMLD) paradigm. Using behavioral and electrophysiological measures within the same listeners, a series of stimulus manipulations was used to evaluate the relative contribution of binaural temporal fine-structure and temporal envelope cues. We evaluated the hypotheses that age-related declines in the BMLD task would be more strongly associated with temporal fine-structure than envelope cues and that age-related declines in behavioral measures would be correlated with cortical auditory evoked potential (CAEP) measures.DesignThirty adults participated in the study, including 10 young normal-hearing, 10 older normal-hearing, and 10 older hearing-impaired adults with bilaterally symmetric, mild-to-moderate sensorineural hearing loss. Behavioral and CAEP thresholds were measured for diotic (So) and dichotic (Sπ) tonal signals presented in continuous diotic (No) narrowband noise (50-Hz wide) maskers. Temporal envelope cues were manipulated by using two different narrowband maskers; Gaussian noise (GN) with robust envelope fluctuations and low-noise noise (LNN) with minimal envelope fluctuations. The potential to use temporal fine-structure cues was controlled by varying the signal frequency (500 or 4000 Hz), thereby relying on the natural decline in phase-locking with increasing frequency.ResultsBehavioral and CAEP thresholds were similar across groups for diotic conditions, while the masking release in dichotic conditions was larger for younger than for older participants. Across all participants, BMLDs were larger for GN than LNN and for 500-Hz than for 4000-Hz conditions, where envelope and fine-structure cues were most salient, respectively. Specific age-related differences were demonstrated for 500-Hz dichotic conditions in GN and LNN, reflecting reduced binaural temporal fine-structure coding. No significant age effects were observed for 4000-Hz dichotic conditions, consistent with similar use of binaural temporal envelope cues across age in these conditions. For all groups, thresholds and derived BMLD values obtained using the behavioral and CAEP methods were strongly correlated, supporting the notion that CAEP measures may be useful as an objective index of age-related changes in binaural temporal processing.ConclusionsThese results demonstrate an age-related decline in the processing of binaural temporal fine-structure cues with preserved temporal envelope coding that was similar with and without mild-to-moderate peripheral hearing loss. Such age-related changes can be reliably indexed by both behavioral and CAEP measures in young and older adults.
| S-EPMC5920708 | biostudies-literature
Project description:Quantifying eye movement is important for diagnosing various neurological and ocular diseases as well as AR/VR displays. We developed a simple setup for real-time dynamic gaze tracking and accommodation measurements based on Purkinje reflections, which are the reflections from front and back surfaces of the cornea and the eye lens. We used an accurate eye model in ZEMAX to simulate the Purkinje reflection positions at different focus distances of the eye, which matched the experimental data. A neural network was trained to simultaneously predict vergence and accommodation using data collected from 9 subjects. We demonstrated that the use of Purkinje reflection coordinates in machine learning resulted in precise estimation. The proposed system accurately predicted the accommodation with an accuracy better than 0.22 D using subject's own data and 0.40 D using other subjects' data with two-point calibration in tests performed with 9 subjects in our setup.
| S-EPMC10703819 | biostudies-literature