Dataset Information

Sight over sound in the judgment of music performance.

ABSTRACT: Social judgments are made on the basis of both visual and auditory information, with consequential implications for our decisions. To examine the impact of visual information on expert judgment and its predictive validity for performance outcomes, this set of seven experiments in the domain of music offers a conservative test of the relative influence of vision versus audition. People consistently report that sound is the most important source of information in evaluating performance in music. However, the findings demonstrate that people actually depend primarily on visual information when making judgments about music performance. People reliably select the actual winners of live music competitions based on silent video recordings, but neither musical novices nor professional musicians were able to identify the winners based on sound recordings or recordings with both video and sound. The results highlight our natural, automatic, and nonconscious dependence on visual cues. The dominance of visual information emerges to the degree that it is overweighted relative to auditory information, even when sound is consciously valued as the core domain content.

SUBMITTER: Tsay CJ

PROVIDER: S-EPMC3767512 | biostudies-literature | 2013 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Sight over sound in the judgment of music performance.

Tsay Chia-Jung CJ

Proceedings of the National Academy of Sciences of the United States of America 20130819 36

Social judgments are made on the basis of both visual and auditory information, with consequential implications for our decisions. To examine the impact of visual information on expert judgment and its predictive validity for performance outcomes, this set of seven experiments in the domain of music offers a conservative test of the relative influence of vision versus audition. People consistently report that sound is the most important source of information in evaluating performance in music. H ...[more]

PMID: 23959902

Similar Datasets

Project description:Post-tonal music often poses perceptual and cognitive challenges for listeners, potentially related to the use of relatively uncommon and unfamiliar musical material and compositional processes. As a basic compositional device, repetition affects memory for music and is structured by composers in very different ways across tonal and post-tonal musical repertoires. Of particular concern is whether post-tonal music exhibits mnemonic affordances that allow listeners to experience a sense of global coherence, and whether repetition correlates strongly with aesthetic judgment. Although previous research suggests that repetition impacts aesthetic preference, empirical research has not mapped out the relationship between repetition and aesthetic judgments across a broad set of post-tonal music. Presenting 14 excerpts, grouped into three categories: tonal, modernist, and post-1970, we observed that indications of repetitions in the music for a group of 60 listeners, with and without musical training, showed significant periods of interindividual synchronization. Aesthetic judgments were assessed by means of ratings for the following parameters: familiarity with the piece, confidence of repetition responses, judgments of affordances for easy listening, coherence, similarity of moments, and recognition, as well as listeners' liking and interest. A principal component analysis (PCA) on the joint question data and the repetition responses suggested that two factors account for 92% of variance in the data. These factors were interpreted as dominated by aesthetic judgment and repetition strength. Linear mixed-effects regression indicated that repetition strength generally differed across excerpt category, with modernist excerpts featuring lower repetition strength compared to both post-1970 and tonal excerpts. Aesthetic preference, on the other hand, was lower for excerpts in both the modernist and post-1970 categories when compared to tonal excerpts. The analysis did not reveal difference in response behavior for repetition responses as a function of musical training, though it indicated higher preference of modernist excerpts with increasing levels of musical training. Overall, the results suggest that the two factors, aesthetic judgment and repetition strength, act as independent determinants in the experience of post-tonal music.

Project description:The sight and sound of a person speaking or a ball bouncing may seem simultaneous, but their corresponding neural signals are spread out over time as they arrive at different multisensory brain sites. How subjective timing relates to such neural timing remains a fundamental neuroscientific and philosophical puzzle. A dominant assumption is that temporal coherence is achieved by sensory resynchronisation or recalibration across asynchronous brain events. This assumption is easily confirmed by estimating subjective audiovisual timing for groups of subjects, which is on average similar across different measures and stimuli, and approximately veridical. But few studies have examined normal and pathological individual differences in such measures. Case PH, with lesions in pons and basal ganglia, hears people speak before seeing their lips move. Temporal order judgements (TOJs) confirmed this: voices had to lag lip-movements (by ?200 msec) to seem synchronous to PH. Curiously, voices had to lead lips (also by ?200 msec) to maximise the McGurk illusion (a measure of audiovisual speech integration). On average across these measures, PH's timing was therefore still veridical. Age-matched control participants showed similar discrepancies. Indeed, normal individual differences in TOJ and McGurk timing correlated negatively: subjects needing an auditory lag for subjective simultaneity needed an auditory lead for maximal McGurk, and vice versa. This generalised to the Stream-Bounce illusion. Such surprising antagonism seems opposed to good sensory resynchronisation, yet average timing across tasks was still near-veridical. Our findings reveal remarkable disunity of audiovisual timing within and between subjects. To explain this we propose that the timing of audiovisual signals within different brain mechanisms is perceived relative to the average timing across mechanisms. Such renormalisation fully explains the curious antagonistic relationship between disparate timing estimates in PH and healthy participants, and how they can still perceive the timing of external events correctly, on average.

Dataset Information

Sight over sound in the judgment of music performance.

Publications

Sight over sound in the judgment of music performance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure