The Communication of Timbral Intentions Between Pianists and Listeners and Its Dependence on Auditory-Visual Conditions.
Ontology highlight
ABSTRACT:
SUBMITTER: Li S
PROVIDER: S-EPMC8491637 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The ability to use temporal relationships between cross-modal cues facilitates perception and behavior. Previously we observed that temporally correlated changes in the size of a visual stimulus and the intensity in an auditory stimulus influenced the ability of listeners to perform an auditory selective attention task (Maddox, Atilgan, Bizley, & Lee, 2015). Participants detected timbral changes in a target sound while ignoring those in a simultaneously presented masker. When the visual stimulus was temporally coherent with the target sound, performance was significantly better than when the visual stimulus was temporally coherent with the masker, despite the visual stimulus conveying no task-relevant information. Here, we trained observers to detect audiovisual temporal coherence and asked whether this changed the way in which they were able to exploit visual information in the auditory selective attention task. We observed that after training, participants were able to benefit from temporal coherence between the visual stimulus and both the target and masker streams, relative to the condition in which the visual stimulus was coherent with neither sound. However, we did not observe such changes in a second group that were trained to discriminate modulation rate differences between temporally coherent audiovisual streams, although they did show an improvement in their overall performance. A control group did not change their performance between pretest and post-test and did not change how they exploited visual information. These results provide insights into how crossmodal experience may optimize multisensory integration.
| S-EPMC7868888 | biostudies-literature
Project description:When sight-reading music, pianists have to decode a large number of notes and immediately transform them into finger actions. How do they achieve such fast decoding? Pianists may use geometrical features contained in the musical score, such as the distance between notes, to improve their efficiency in reading them. The aim of this study is to investigate the visual information pianists rely on when reading music. We measured the accuracy of the musical score reading of 16 skilled pianists and investigated its relationship with the geometrical features. When a single note was presented, pianists easily read it when it was located within three ledger lines. When two notes with an octave interval were presented, interestingly, their readable range was extended compared to that of the single note. The pianists were also able to recognize the octave interval correctly even if they misread the height (or pitch) of the target notes. These results suggest that the pianists decoded two notes composing an octave interval as a single "two-tone geometric pattern." Analyzing the characteristics of incorrect responses, we also found that pianists used the geometrical features of the spatial relationship between the note head and the ledger line, and that the cause of the misreading could be categorized into four types: [Type I] Confusion to a neighboring note having the same ledger line configuration; [Type II] Interference from a commonly used height note having the same note name; [Type III] Misunderstanding based on the appearance probability; [Type IV] Combination of the above three. These results all indicate that the pianists' abilities in score reading rely greatly on the correlation between the geometric features and playing action, which the pianists acquired through long-time training.
| S-EPMC6261976 | biostudies-literature
Project description:Fine-tuned sensory functions typically characterize skilled individuals. Although numerous studies demonstrated enhanced unimodal sensory functions at both neural and behavioral levels in skilled individuals, little is known about their multisensory interaction function, especially multisensory integration and selective attention that involve volitional control of information derived from multiple sensory organs. In the current study, expert pianists and musically untrained individuals performed five sets of intensity discrimination tasks at the auditory and somatosensory modalities with different conditions: (1) auditory stimulus, (2) somatosensory stimulus, (3) congruent auditory and somatosensory stimuli (i.e., multisensory integration), (4) auditory and task-irrelevant somatosensory stimuli, and (5) somatosensory and task-irrelevant auditory stimuli. In the fourth and fifth conditions, participants were instructed to ignore a task-irrelevant stimulus and to pay attention to a task-relevant stimulus (i.e., selective attention), respectively. While the discrimination perception was superior in the condition (3) compared to the better one of the individual unimodal conditions only in the pianists, the task-irrelevant somatosensory stimulus worsened the auditory discrimination more in the pianists than the nonmusicians. These findings indicate unique multisensory interactions in expert pianists, which enables pianists to efficiently integrate the auditory and somatosensory information, but exacerbates top-down selective inhibition of somatosensory information during auditory processing.
| S-EPMC9307509 | biostudies-literature
Project description:In noisy settings, listening is aided by correlated dynamic visual cues gleaned from a talker's face-an improvement often attributed to visually reinforced linguistic information. In this study, we aimed to test the effect of audio-visual temporal coherence alone on selective listening, free of linguistic confounds. We presented listeners with competing auditory streams whose amplitude varied independently and a visual stimulus with varying radius, while manipulating the cross-modal temporal relationships. Performance improved when the auditory target's timecourse matched that of the visual stimulus. The fact that the coherence was between task-irrelevant stimulus features suggests that the observed improvement stemmed from the integration of auditory and visual streams into cross-modal objects, enabling listeners to better attend the target. These findings suggest that in everyday conditions, where listeners can often see the source of a sound, temporal cues provided by vision can help listeners to select one sound source from a mixture.
| S-EPMC4337603 | biostudies-literature
Project description:Cross-modal correspondences refer to associations between apparently unrelated stimulus features in different senses. For example, high and low auditory pitches are associated with high and low visual elevations, respectively. Here we examined how this crossmodal correspondence between visual elevation and auditory pitch relates to auditory elevation. We used audiovisual combinations of high- or low-frequency bursts of white noise and a visual stimulus comprising a white circle. Auditory and visual stimuli could each occur at high or low elevations. These multisensory stimuli could be congruent or incongruent for three correspondence types: cross-modal featural (auditory pitch/visual elevation), within-modal featural (auditory pitch/auditory elevation) and cross-modal spatial (auditory and visual elevation). Participants performed a 2AFC speeded classification (high or low) task while attending to auditory pitch, auditory elevation, or visual elevation. We tested for modulatory interactions between the three correspondence types. Modulatory interactions were absent when discriminating visual elevation. However, the within-modal featural correspondence affected the cross-modal featural correspondence during discrimination of auditory elevation and pitch, while the reverse modulation was observed only during discrimination of auditory pitch. The cross-modal spatial correspondence modulated the other two correspondences only when auditory elevation was being attended, was modulated by the cross-modal featural correspondence only during attention to auditory pitch, and was modulated by the within-modal featural correspondence while performing discrimination of either auditory elevation or pitch. We conclude that the cross-modal correspondence between auditory pitch and visual elevation interacts strongly with auditory elevation.
| S-EPMC7877490 | biostudies-literature
Project description:The effect of emotion on memory is powerful and complex. While there seems to be agreement that emotional arousal generally increases the likelihood that events are remembered, it is somewhat disputed whether also the valence of emotions influences memory. Specifically, several experiments by Kensinger and colleagues have provided evidence for the hypotheses that negative valanced emotions facilitate the encoding of perceptual details. On the other hand, Mather and colleagues have suggested that these results could be explained by confounding relationships of valence and arousal, i.e., that items that generate negative emotions are typically also more arousing. In this study, we provide a conceptual replication of Kensinger's findings. We employed a novel experimental design, in which the effects of standardized emotional arousing sounds on recognition accuracy for neutral visual scenes was measured. We indirectly manipulated the amount of visual detail that was encoded, by requiring participants to memorize either single exemplars (low interference) or multiple exemplars (high interference) of visual scene categories. With increasing visual overlap in the high interference condition, participants were required to encode a high degree of visual detail to successfully remember the exemplars. The results obtained from 60 healthy human participants confirmed Kensinger's hypothesis by showing that under conditions of high visual interference, negative valanced emotions led to higher levels of recognition accuracy compared to neutral and positive emotions. Furthermore, based on the normative arousal ratings of the stimulus set, our results suggest that the differential recognition effect cannot be explained by differing levels of arousal.
| S-EPMC6277761 | biostudies-other
Project description:Moving to a speech rhythm can enhance verbal processing in the listener by increasing temporal expectancies (Falk and Dalla Bella, 2016). Here we tested whether this hypothesis holds for prosodically diverse languages such as German (a lexical stress-language) and French (a non-stress language). Moreover, we examined the relation between motor performance and the benefits for verbal processing as a function of language. Sixty-four participants, 32 German and 32 French native speakers detected subtle word changes in accented positions in metrically structured sentences to which they previously tapped with their index finger. Before each sentence, they were cued by a metronome to tap either congruently (i.e., to accented syllables) or incongruently (i.e., to non-accented parts) to the following speech stimulus. Both French and German speakers detected words better when cued to tap congruently compared to incongruent tapping. Detection performance was predicted by participants' motor performance in the non-verbal cueing phase. Moreover, tapping rate while participants tapped to speech predicted detection differently for the two language groups, in particular in the incongruent tapping condition. We discuss our findings in light of the rhythmic differences of both languages and with respect to recent theories of expectancy-driven and multisensory speech processing.
| S-EPMC5387104 | biostudies-literature
Project description:How we perceive and learn about our environment is influenced by our prior experiences and existing representations of the world. Top-down cognitive processes, such as attention and expectations, can alter how we process sensory stimuli, both within a modality (e.g., effects of auditory experience on auditory perception), as well as across modalities (e.g., effects of visual feedback on sound localization). Here, we demonstrate that experience with different types of auditory input (spoken words vs. environmental sounds) modulates how humans remember concurrently-presented visual objects. Participants viewed a series of line drawings (e.g., picture of a cat) displayed in one of four quadrants while listening to a word or sound that was congruent (e.g., "cat" or <meow>), incongruent (e.g., "motorcycle" or <vroom-vroom>), or neutral (e.g., a meaningless pseudoword or a tonal beep) relative to the picture. Following the encoding phase, participants were presented with the original drawings plus new drawings and asked to indicate whether each one was "old" or "new." If a drawing was designated as "old," participants then reported where it had been displayed. We find that words and sounds both elicit more accurate memory for what objects were previously seen, but only congruent environmental sounds enhance memory for where objects were positioned - this, despite the fact that the auditory stimuli were not meaningful spatial cues of the objects' locations on the screen. Given that during real-world listening conditions, environmental sounds, but not words, reliably originate from the location of their referents, listening to sounds may attune the visual dorsal pathway to facilitate attention and memory for objects' locations. We propose that audio-visual associations in the environment and in our previous experience jointly contribute to visual memory, strengthening visual memory through exposure to auditory input.
| S-EPMC8350348 | biostudies-literature
Project description:Auditory stream segregation is a perceptual process by which the human auditory system groups sounds from different sources into perceptually meaningful elements (e.g., a voice or a melody). The perceptual segregation of sounds is important, for example, for the understanding of speech in noisy scenarios, a particularly challenging task for listeners with a cochlear implant (CI). It has been suggested that some aspects of stream segregation may be explained by relatively basic neural mechanisms at a cortical level. During the past decades, a variety of models have been proposed to account for the data from stream segregation experiments in normal-hearing (NH) listeners. However, little attention has been given to corresponding findings in CI listeners. The present study investigated whether a neural model of sequential stream segregation, proposed to describe the behavioral effects observed in NH listeners, can account for behavioral data from CI listeners. The model operates on the stimulus features at the cortical level and includes a competition stage between the neuronal units encoding the different percepts. The competition arises from a combination of mutual inhibition, adaptation, and additive noise. The model was found to capture the main trends in the behavioral data from CI listeners, such as the larger probability of a segregated percept with increasing the feature difference between the sounds as well as the build-up effect. Importantly, this was achieved without any modification to the model's competition stage, suggesting that stream segregation could be mediated by a similar mechanism in both groups of listeners.
| S-EPMC6616076 | biostudies-literature
Project description:When the brain tries to acquire an elaborate model of the world, multisensory integration should contribute to building predictions based on the various pieces of information, and deviance detection should repeatedly update these predictions by detecting "errors" from the actual sensory inputs. Accumulating evidence such as a hierarchical organization of the deviance-detection system indicates that the deviance-detection system can be interpreted in the predictive coding framework. Herein, we targeted mismatch negativity (MMN) as a type of prediction-error signal and investigated the relationship between multisensory integration and MMN. In particular, we studied whether and how cross-modal information processing affected MMN in rodents. We designed a new surface microelectrode array and simultaneously recorded visual and auditory evoked potentials from the visual and auditory cortices of rats under anesthesia. Then, we mapped MMNs for five types of deviant stimuli: single-modal deviants in (i) the visual oddball and (ii) auditory oddball paradigms, eliciting single-modal MMN; (iii) congruent audio-visual deviants, (iv) incongruent visual deviants, and (v) incongruent auditory deviants in the audio-visual oddball paradigm, eliciting cross-modal MMN. First, we demonstrated that visual MMN exhibited deviance detection properties and that the first-generation focus of visual MMN was localized in the visual cortex, as previously reported in human studies. Second, a comparison of MMN amplitudes revealed a non-linear relationship between single-modal and cross-modal MMNs. Moreover, congruent audio-visual MMN exhibited characteristics of both visual and auditory MMNs-its latency was similar to that of auditory MMN, whereas local blockage of N-methyl-D-aspartic acid receptors in the visual cortex diminished it as well as visual MMN. These results indicate that cross-modal information processing affects MMN without involving strong top-down effects, such as those of prior knowledge and attention. The present study is the first electrophysiological evidence of cross-modal MMN in animal models, and future studies on the neural mechanisms combining multisensory integration and deviance detection are expected to provide electrophysiological evidence to confirm the links between MMN and predictive coding theory.
| S-EPMC8484534 | biostudies-literature