Browse
Submit Data
Databases
API
Help

Dataset Information

18 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Cross-modal noise compensation in audiovisual words.

ABSTRACT: Perceiving linguistic input is vital for human functioning, but the process is complicated by the fact that the incoming signal is often degraded. However, humans can compensate for unimodal noise by relying on simultaneous sensory input from another modality. Here, we investigated noise-compensation for spoken and printed words in two experiments. In the first behavioral experiment, we observed that accuracy was modulated by reaction time, bias and sensitivity, but noise compensation could nevertheless be explained via accuracy differences when controlling for RT, bias and sensitivity. In the second experiment, we also measured Event Related Potentials (ERPs) and observed robust electrophysiological correlates of noise compensation starting at around 350?ms after stimulus onset, indicating that noise compensation is most prominent at lexical/semantic processing levels.

SUBMITTER: Baart M

PROVIDER: S-EPMC5294401 | biostudies-literature | 2017 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Cross-modal noise compensation in audiovisual words.

Baart Martijn M Armstrong Blair C BC Martin Clara D CD Frost Ram R Carreiras Manuel M

Scientific reports 20170207

Perceiving linguistic input is vital for human functioning, but the process is complicated by the fact that the incoming signal is often degraded. However, humans can compensate for unimodal noise by relying on simultaneous sensory input from another modality. Here, we investigated noise-compensation for spoken and printed words in two experiments. In the first behavioral experiment, we observed that accuracy was modulated by reaction time, bias and sensitivity, but noise compensation could neve ...[more]

PMID: 28169316

Similar Datasets

Cross-modal metaphorical mapping of spoken emotion words onto vertical space.

Project description:From the field of embodied cognition, previous studies have reported evidence of metaphorical mapping of emotion concepts onto a vertical spatial axis. Most of the work on this topic has used visual words as the typical experimental stimuli. However, to our knowledge, no previous study has examined the association between affect and vertical space using a cross-modal procedure. The current research is a first step toward the study of the metaphorical mapping of emotions onto vertical space by means of an auditory to visual cross-modal paradigm. In the present study, we examined whether auditory words with an emotional valence can interact with the vertical visual space according to a 'positive-up/negative-down' embodied metaphor. The general method consisted in the presentation of a spoken word denoting a positive/negative emotion prior to the spatial localization of a visual target in an upper or lower position. In Experiment 1, the spoken words were passively heard by the participants and no reliable interaction between emotion concepts and bodily simulated space was found. In contrast, Experiment 2 required more active listening of the auditory stimuli. A metaphorical mapping of affect and space was evident but limited to the participants engaged in an emotion-focused task. Our results suggest that the association of affective valence and vertical space is not activated automatically during speech processing since an explicit semantic and/or emotional evaluation of the emotionally valenced stimuli was necessary to obtain an embodied effect. The results are discussed within the framework of the embodiment hypothesis.

| S-EPMC4531208 | biostudies-literature

The impact of spoken action words on performance in a cross-modal oddball task.

Project description:In this study a cross-modal oddball task was employed to study the effect that words spoken either non-urgently or urgently would have on a digit categorization task and if women would exhibit greater behavioral inhibitory control. The words were unrelated to the task itself, but related to the action required to complete the task. Forty participants (21 women) conducted a computerized categorization task while exposed to a sinewave tone as a standard stimulus (75% of the trials) or a to-be ignored word (press, stop) spoken either non-urgently or urgently as unexpected auditory deviant stimulus (6.25% trials for each category). Urgent words had sharp intonation and an average fundamental frequency (F0) ranging from 191.9 (stop) to 204.6 (press) Hz. Non-urgent words had low intonation with average F0 ranging from 103.9.9 (stop) to 120.3 (press) Hz. As expected, deviant distraction and longer response times were found by exposure to the word stop, but deviant distraction was not found to be significant with the word press or due to intonation. While the results showed that women had in general longer reaction times, there were no gender differences found related to the deviant distraction caused by word or intonation. The present results do not support the hypothesis that women have greater behavioral inhibitory control, but there was evidence that the meaning of the word could influence response times.

| S-EPMC6245795 | biostudies-literature

Arabic digits and spoken number words: Timing modulates the cross-modal numerical distance effect.

Project description:Moving seamlessly between spoken number words and Arabic digits is common in everyday life. In this study, we systematically investigated the correspondence between auditory number words and visual Arabic digits in adults. Auditory number words and visual Arabic digits were presented concurrently or sequentially and participants had to indicate whether they described the same quantity. We manipulated the stimulus onset asynchronies (SOAs) between the two stimuli (Experiment 1: -500 ms to +500 ms; Experiment 2: -200 ms to +200 ms). In both experiments, we found a significant cross-modal distance effect. This effect was strongest for simultaneous stimulus presentation and decreased with increasing SOAs. Numerical distance emerged as the most consistent significant predictor overall, in particular for simultaneous presentation. However, physical similarity between the stimuli was often a significant predictor of response times in addition to numerical distance, and at longer SOAs, physical similarity between the stimuli was the only significant predictor. This shows that SOA modulates the extent to which participants access quantity representations. Our results thus support the idea that a semantic quantity representation of auditory and visual numerical symbols is activated when participants perform a concurrent matching task, while at longer SOAs participants are more likely to rely on physical similarity between the stimuli. We also investigated whether individual differences in the efficiency of the cross-modal processing were related to differences in mathematical performance. Our results are inconclusive about whether the efficiency of cross-format numerical correspondence is related to mathematical competence in adults.

| S-EPMC6779017 | biostudies-other

Motor engagement relates to accurate perception of phonemes and audiovisual words, but not auditory words.

Project description:A longstanding debate has surrounded the role of the motor system in speech perception, but progress in this area has been limited by tasks that only examine isolated syllables and conflate decision-making with perception. Using an adaptive task that temporally isolates perception from decision-making, we examined an EEG signature of motor activity (sensorimotor μ/beta suppression) during the perception of auditory phonemes, auditory words, audiovisual words, and environmental sounds while holding difficulty constant at two levels (Easy/Hard). Results revealed left-lateralized sensorimotor μ/beta suppression that was related to perception of speech but not environmental sounds. Audiovisual word and phoneme stimuli showed enhanced left sensorimotor μ/beta suppression for correct relative to incorrect trials, while auditory word stimuli showed enhanced suppression for incorrect trials. Our results demonstrate that motor involvement in perception is left-lateralized, is specific to speech stimuli, and it not simply the result of domain-general processes. These results provide evidence for an interactive network for speech perception in which dorsal stream motor areas are dynamically engaged during the perception of speech depending on the characteristics of the speech signal. Crucially, this motor engagement has different effects on the perceptual outcome depending on the lexicality and modality of the speech stimulus.

| S-EPMC7835217 | biostudies-literature

Finding a home in the noise: cross-modal impact of anthropogenic vibration on animal search behaviour.

Project description:Chemical cues and signals enable animals to sense their surroundings over vast distances and find key resources, like food and shelter. However, the use of chemosensory information may be impaired in aquatic habitats by anthropogenic activities, which produce both water-borne sounds and substrate-borne vibrations, potentially affecting not only vibroacoustic sensing but other modalities as well. We attracted marine hermit crabs (Pagurus acadianus) in field experiments using a chemical cue indicative of a newly available shell home. We then quantified the number of crabs arriving in control versus impulsive noise conditions. Treatment (control or noise), time (before or after), and the interaction between the two significantly affected the numbers of crabs, with fewer crabs attracted to the chemical cue after noise exposure. The results indicate that noise can affect chemical information use in the marine environment, acting cross-modally to impact chemically-guided search behaviour in free-ranging animals. Broadly, anthropogenic noise and seabed vibration may have profound effects, even on behaviours mediated by other sensory modalities. Hence, the impact of noise should be investigated not only within, but also across sensory modalities.This article has an associated First Person interview with the first author of the paper.

| S-EPMC6679394 | biostudies-literature

The cross-modal double flash illusion depends on featural similarity between cross-modal inducers.

Project description:Despite extensive evidence of the possible interactions between multisensory signals, it remains unclear at what level of sensory processing these interactions take place. When two identical auditory beeps (inducers) are presented in quick succession accompanied by a single visual flash, observers often report seeing two visual flashes, rather than the physical one - the double flash illusion. This compelling illusion has often been considered to reflect direct interactions between neural activations in different primary sensory cortices. Against this simple account, here we show that by simply changing the inducer signals between featurally distinct signals (e.g. high- and low-pitch beeps) the illusory double flash is abolished. This result suggests that a critical component underlying the illusion is perceptual grouping of the inducer signals, consistent with the notion that multisensory combination is preceded by determination of whether the relevant signals share a common source of origin.

| S-EPMC3853685 | biostudies-literature

Noise-Induced Frequency Modifications of Tamarin Vocalizations: Implications for Noise Compensation in Nonhuman Primates.

Project description:Previous research suggests that nonhuman primates have limited flexibility in the frequency content of their vocalizations, particularly when compared to human speech. Consistent with this notion, several nonhuman primate species have demonstrated noise-induced changes in call amplitude and duration, with no evidence of changes to spectral content. This experiment used broad- and narrow-band noise playbacks to investigate the vocal control of two call types produced by cotton-top tamarins (Saguinus Oedipus). In 'combination long calls' (CLCs), peak fundamental frequency and the distribution of energy between low and high frequency harmonics (spectral tilt) changed in response to increased noise amplitude and bandwidth. In chirps, peak and maximum components of the fundamental frequency increased with increasing noise level, with no changes to spectral tilt. Other modifications included the Lombard effect and increases in chirp duration. These results provide the first evidence for noise-induced frequency changes in nonhuman primate vocalizations and suggest that future investigations of vocal plasticity in primates should include spectral parameters.

| S-EPMC4479599 | biostudies-literature

Noise reduction facilitated by dosage compensation in gene networks.

Project description:Genetic noise together with genome duplication and volume changes during cell cycle are significant contributors to cell-to-cell heterogeneity. How can cells buffer the effects of these unavoidable epigenetic and genetic variations on phenotypes that are sensitive to such variations? Here we show that a simple network motif that is essential for network-dosage compensation can reduce the effects of extrinsic noise on the network output. Using natural and synthetic gene networks with and without the network motif, we measure gene network activity in single yeast cells and find that the activity of the compensated network is significantly lower in noise compared with the non-compensated network. A mathematical analysis provides intuitive insights into these results and a novel stochastic model tracking cell-volume and cell-cycle predicts the experimental results. Our work implies that noise is a selectable trait tunable by evolution.

| S-EPMC5063963 | biostudies-literature

Multi-modal communication: song sparrows increase signal redundancy in noise.

Project description:Although the effects of anthropogenic noise on animal communication have been studied widely, most research on the effect of noise in communication has focused on signals in a single modality. Consequently, how multi-modal communication is affected by anthropogenic noise is relatively poorly understood. Here, we ask whether song sparrows (Melospiza melodia) show evidence of plasticity in response to noise in two aggressive signals in acoustic and visual modalities. We test two hypotheses: (i) that song sparrows will shift signalling effort to the visual modality (the multi-modal shift hypothesis) and (ii) that they will increase redundancy of their multi-modal signalling (the back-up signal hypothesis). We presented male song sparrows with song playback and a taxidermic mount with or without a low-frequency acoustic noise from a nearby speaker. We found that males did not switch their signalling effort to visual modality (i.e. wing waves) in response to the noise. However, the correlation between warbled soft songs and wing waves increased in the noise treatment, i.e. signals became more redundant. These results suggest that when faced with anthropogenic noise, song sparrows can increase the redundancy of their multi-modal signals, which may aid in the robustness of the communication system.

| S-EPMC6832173 | biostudies-literature

Cross-modal prediction in speech perception.

Project description:Speech perception often benefits from vision of the speaker's lip movements when they are available. One potential mechanism underlying this reported gain in perception arising from audio-visual integration is on-line prediction. In this study we address whether the preceding speech context in a single modality can improve audiovisual processing and whether this improvement is based on on-line information-transfer across sensory modalities. In the experiments presented here, during each trial, a speech fragment (context) presented in a single sensory modality (voice or lips) was immediately continued by an audiovisual target fragment. Participants made speeded judgments about whether voice and lips were in agreement in the target fragment. The leading single sensory context and the subsequent audiovisual target fragment could be continuous in either one modality only, both (context in one modality continues into both modalities in the target fragment) or neither modalities (i.e., discontinuous). The results showed quicker audiovisual matching responses when context was continuous with the target within either the visual or auditory channel (Experiment 1). Critically, prior visual context also provided an advantage when it was cross-modally continuous (with the auditory channel in the target), but auditory to visual cross-modal continuity resulted in no advantage (Experiment 2). This suggests that visual speech information can provide an on-line benefit for processing the upcoming auditory input through the use of predictive mechanisms. We hypothesize that this benefit is expressed at an early level of speech analysis.

| S-EPMC3187777 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data