The Effects of Audiovisual Inputs on Solving the Cocktail Party Problem in the Human Brain: An fMRI Study.
Ontology highlight
ABSTRACT:
SUBMITTER: Li Y
PROVIDER: S-EPMC6676955 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The cocktail party problem requires listeners to infer individual sound sources from mixtures of sound. The problem can be solved only by leveraging regularities in natural sound sources, but little is known about how such regularities are internalized. We explored whether listeners learn source "schemas"-the abstract structure shared by different occurrences of the same type of sound source-and use them to infer sources from mixtures. We measured the ability of listeners to segregate mixtures of time-varying sources. In each experiment a subset of trials contained schema-based sources generated from a common template by transformations (transposition and time dilation) that introduced acoustic variation but preserved abstract structure. Across several tasks and classes of sound sources, schema-based sources consistently aided source separation, in some cases producing rapid improvements in performance over the first few exposures to a schema. Learning persisted across blocks that did not contain the learned schema, and listeners were able to learn and use multiple schemas simultaneously. No learning was evident when schema were presented in the task-irrelevant (i.e., distractor) source. However, learning from task-relevant stimuli showed signs of being implicit, in that listeners were no more likely to report that sources recurred in experiments containing schema-based sources than in control experiments containing no schema-based sources. The results implicate a mechanism for rapidly internalizing abstract sound structure, facilitating accurate perceptual organization of sound sources that recur in the environment.
| S-EPMC5889675 | biostudies-literature
Project description:Are musicians better able to understand speech in noise than non-musicians? Recent findings have produced contradictory results. Here we addressed this question by asking musicians and non-musicians to understand target sentences masked by other sentences presented from different spatial locations, the classical 'cocktail party problem' in speech science. We found that musicians obtained a substantial benefit in this situation, with thresholds ~6?dB better than non-musicians. Large individual differences in performance were noted particularly for the non-musically trained group. Furthermore, in different conditions we manipulated the spatial location and intelligibility of the masking sentences, thus changing the amount of 'informational masking' (IM) while keeping the amount of 'energetic masking' (EM) relatively constant. When the maskers were unintelligible and spatially separated from the target (low in IM), musicians and non-musicians performed comparably. These results suggest that the characteristics of speech maskers and the amount of IM can influence the magnitude of the differences found between musicians and non-musicians in multiple-talker "cocktail party" environments. Furthermore, considering the task in terms of the EM-IM distinction provides a conceptual framework for future behavioral and neuroscientific studies which explore the underlying sensory and cognitive mechanisms contributing to enhanced "speech-in-noise" perception by musicians.
| S-EPMC4481518 | biostudies-other
Project description:Echolocating bats need to solve the problem of signal jamming by conspecifics when they are in a group. However, while several mechanisms have been suggested, it remains unclear how bats avoid confusion between their own echoes and interfering sounds in a complex acoustic environment. Here, we fixed on-board microphones onto individual frequency-modulating bats flying in groups. We found that group members broaden the inter-individual differences in the terminal frequencies of pulses, thereby decreasing the similarity of pulses among individuals. To understand what features most affect similarity between pulses, we calculated the similarity of signals mimicking pulses. We found that the similarity between those artificial signals was decreased most by manipulation of terminal frequency. These results demonstrate that the signal jamming problem is solved by this simple strategy, which may be universally used by animals that use active sensing, such as echolocating bats and electric fish, thereby transcending species and sensory modalities.
| S-EPMC6123623 | biostudies-literature
Project description:The "cocktail party problem" requires us to discern individual sound sources from mixtures of sources. The brain must use knowledge of natural sound regularities for this purpose. One much-discussed regularity is the tendency for frequencies to be harmonically related (integer multiples of a fundamental frequency). To test the role of harmonicity in real-world sound segregation, we developed speech analysis/synthesis tools to perturb the carrier frequencies of speech, disrupting harmonic frequency relations while maintaining the spectrotemporal envelope that determines phonemic content. We find that violations of harmonicity cause individual frequencies of speech to segregate from each other, impair the intelligibility of concurrent utterances despite leaving intelligibility of single utterances intact, and cause listeners to lose track of target talkers. However, additional segregation deficits result from replacing harmonic frequencies with noise (simulating whispering), suggesting additional grouping cues enabled by voiced speech excitation. Our results demonstrate acoustic grouping cues in real-world sound segregation.
| S-EPMC5974276 | biostudies-literature
Project description:The goal of this study was to investigate how cognitive factors influence performance in a multi-talker, "cocktail-party" like environment in musicians and non-musicians. This was achieved by relating performance in a spatial hearing task to cognitive processing abilities assessed using measures of executive function (EF) and visual attention in musicians and non-musicians. For the spatial hearing task, a speech target was presented simultaneously with two intelligible speech maskers that were either colocated with the target (0° azimuth) or were symmetrically separated from the target in azimuth (at ±15°). EF assessment included measures of cognitive flexibility, inhibition control and auditory working memory. Selective attention was assessed in the visual domain using a multiple object tracking task (MOT). For the MOT task, the observers were required to track target dots (n = 1,2,3,4,5) in the presence of interfering distractor dots. Musicians performed significantly better than non-musicians in the spatial hearing task. For the EF measures, musicians showed better performance on measures of auditory working memory compared to non-musicians. Furthermore, across all individuals, a significant correlation was observed between performance on the spatial hearing task and measures of auditory working memory. This result suggests that individual differences in performance in a cocktail party-like environment may depend in part on cognitive factors such as auditory working memory. Performance in the MOT task did not differ between groups. However, across all individuals, a significant correlation was found between performance in the MOT and spatial hearing tasks. A stepwise multiple regression analysis revealed that musicianship and performance on the MOT task significantly predicted performance on the spatial hearing task. Overall, these findings confirm the relationship between musicianship and cognitive factors including domain-general selective attention and working memory in solving the "cocktail party problem".
| S-EPMC4934907 | biostudies-literature
Project description:Group problem solving is a prototypical complex collective intellectual activity. Psychological research provides compelling evidence that problem solving in groups is both qualitatively and quantitatively different from doing so alone. However, the question of whether individual and collective problem solving involve the same neural substrate has not yet been addressed, mainly due to methodological limitations. In the current study, functional magnetic resonance imaging was performed to compare brain activation when participants solved Raven-like matrix problems in a small group and individually. In the group condition, the participant in the scanner was able to discuss the problem with other team members using a special communication device. In the individual condition, the participant was required to think aloud while solving the problem in the silent presence of the other team members. Greater activation was found in several brain regions during group problem solving, including the medial prefrontal cortex; lateral parietal, cingulate, precuneus and retrosplenial cortices; frontal and temporal poles. These areas have been identified as potential components of the so-called "social brain" on the basis of research using offline judgments of material related to socializing. Therefore, this study demonstrated the actual involvement of these regions in real-time social interactions, such as group problem solving. However, further connectivity analysis revealed that the social brain components are co-activated, but do not increase their coupling during cooperation as would be suggested for a holistic network. We suggest that the social mode of the brain may be described instead as a re-configuration of connectivity between basic networks, and we found decreased connectivity between the language and salience networks in the group compared to the individual condition. A control experiment showed that the findings from the main experiment cannot be entirely accounted for by discourse comprehension. Thus, the study demonstrates affordances provided by the presented new technique for neuroimaging the "group mind," implementing the single-brain version of the second-person neuroscience approach.
| S-EPMC7483667 | biostudies-literature
Project description:The human auditory system has the ability to segregate complex auditory scenes into a foreground component and a background, allowing us to listen to specific speech sounds from a mixture of sounds. Selective attention plays a crucial role in this process, colloquially known as the "cocktail party effect." It has not been possible to build a machine that can emulate this human ability in real-time. Here, we have developed a framework for the implementation of a neuromorphic sound segregation algorithm in a Field Programmable Gate Array (FPGA). This algorithm is based on the principles of temporal coherence and uses an attention signal to separate a target sound stream from background noise. Temporal coherence implies that auditory features belonging to the same sound source are coherently modulated and evoke highly correlated neural response patterns. The basis for this form of sound segregation is that responses from pairs of channels that are strongly positively correlated belong to the same stream, while channels that are uncorrelated or anti-correlated belong to different streams. In our framework, we have used a neuromorphic cochlea as a frontend sound analyser to extract spatial information of the sound input, which then passes through band pass filters that extract the sound envelope at various modulation rates. Further stages include feature extraction and mask generation, which is finally used to reconstruct the targeted sound. Using sample tonal and speech mixtures, we show that our FPGA architecture is able to segregate sound sources in real-time. The accuracy of segregation is indicated by the high signal-to-noise ratio (SNR) of the segregated stream (90, 77, and 55 dB for simple tone, complex tone, and speech, respectively) as compared to the SNR of the mixture waveform (0 dB). This system may be easily extended for the segregation of complex speech signals, and may thus find various applications in electronic devices such as for sound segregation and speech recognition.
| S-EPMC4557082 | biostudies-literature
Project description:The perceptual analysis of acoustic scenes involves binding together sounds from the same source and separating them from other sounds in the environment. In large social groups, listeners experience increased difficulty performing these tasks due to high noise levels and interference from the concurrent signals of multiple individuals. While a substantial body of literature on these issues pertains to human hearing and speech communication, few studies have investigated how nonhuman animals may be evolutionarily adapted to solve biologically analogous communication problems. Here, I review recent and ongoing work aimed at testing hypotheses about perceptual mechanisms that enable treefrogs in the genus Hyla to communicate vocally in noisy, multi-source social environments. After briefly introducing the genus and the methods used to study hearing in frogs, I outline several functional constraints on communication posed by the acoustic environment of breeding "choruses". Then, I review studies of sound source perception aimed at uncovering how treefrog listeners may be adapted to cope with these constraints. Specifically, this review covers research on the acoustic cues used in sequential and simultaneous auditory grouping, spatial release from masking, and dip listening. Throughout the paper, I attempt to illustrate how broad-scale, comparative studies of carefully considered animal models may ultimately reveal an evolutionary diversity of underlying mechanisms for solving cocktail-party-like problems in communication.
| S-EPMC4096220 | biostudies-literature
Project description:Dip listening refers to our ability to catch brief "acoustic glimpses" of speech and other sounds when fluctuating background noise levels momentarily decrease. Exploiting dips in natural fluctuations of noise contributes to our ability to overcome the "cocktail party problem" of understanding speech in multi-talker social environments. We presently know little about how nonhuman animals solve analogous communication problems. Here, we asked whether female grey treefrogs (Hyla chrysoscelis) might benefit from dip listening in selecting a mate in the noisy social setting of a breeding chorus. Consistent with a dip listening hypothesis, subjects recognized conspecific calls at lower thresholds when the dips in a chorus-like noise masker were long enough to allow glimpses of nine or more consecutive pulses. No benefits of dip listening were observed when dips were shorter and included five or fewer pulses. Recognition thresholds were higher when the noise fluctuated at a rate similar to the pulse rate of the call. In a second experiment, advertisement calls comprising six to nine pulses were necessary to elicit responses under quiet conditions. Together, these results suggest that in frogs, the benefits of dip listening are constrained by neural mechanisms underlying temporal pattern recognition. These constraints have important implications for the evolution of male signalling strategies in noisy social environments.
| S-EPMC3290330 | biostudies-literature
Project description:Despite considerable interest in the forces shaping the relationship between brain size and cognitive abilities, it remains controversial whether larger-brained animals are, indeed, better problem-solvers. Recently, several comparative studies have revealed correlations between brain size and traits thought to require advanced cognitive abilities, such as innovation, behavioral flexibility, invasion success, and self-control. However, the general assumption that animals with larger brains have superior cognitive abilities has been heavily criticized, primarily because of the lack of experimental support for it. Here, we designed an experiment to inquire whether specific neuroanatomical or socioecological measures predict success at solving a novel technical problem among species in the mammalian order Carnivora. We presented puzzle boxes, baited with food and scaled to accommodate body size, to members of 39 carnivore species from nine families housed in multiple North American zoos. We found that species with larger brains relative to their body mass were more successful at opening the boxes. In a subset of species, we also used virtual brain endocasts to measure volumes of four gross brain regions and show that some of these regions improve model prediction of success at opening the boxes when included with total brain size and body mass. Socioecological variables, including measures of social complexity and manual dexterity, failed to predict success at opening the boxes. Our results, thus, fail to support the social brain hypothesis but provide important empirical support for the relationship between relative brain size and the ability to solve this novel technical problem.
| S-EPMC4780594 | biostudies-literature