Deep neural network models of sound localization reveal how perception is adapted to real-world environments.
Ontology highlight
ABSTRACT:
SUBMITTER: Francl A
PROVIDER: S-EPMC8830739 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Tinnitus therapies have been combined with the use of varieties of sound/noise. For masking external sounds, location of the masker in space is important; however, effects of the spatial location of the masker on tinnitus are less understood. We aimed to test whether a masking sound location would affect the perception level of simulated tinnitus. The 4 kHz simulated tinnitus was induced in the right ear of healthy volunteers through an open-type earphone. White noise was presented to the right ear using a single-sided headphone or a speaker positioned on the right side at a distance of 1.8 m for masking the simulated tinnitus. In other sessions, monaurally recorded noise localized within the head (inside-head noise) or binaurally recorded noise localized outside the head (outside-head noise) was separately presented from a dual-sided headphone. The noise presented from a distant speaker and the outside-head noise masked the simulated tinnitus in 71.1% and 77.1% of measurements at a lower intensity compared to the noise beside the ear and the inside-head noise, respectively. In conclusion, spatial information regarding the masking noise may play a role in reducing the perception level of simulated tinnitus. Binaurally recorded sounds may be beneficial for an acoustic therapy of tinnitus.
| S-EPMC8795453 | biostudies-literature
Project description:In this paper, we propose joint optimization of deep neural network (DNN)-supported dereverberation and beamforming for the convolutional recurrent neural network (CRNN)-based sound event detection (SED) in multi-channel environments. First, the short-time Fourier transform (STFT) coefficients are calculated from multi-channel audio signals under the noisy and reverberant environments, which are then enhanced by the DNN-supported weighted prediction error (WPE) dereverberation with the estimated masks. Next, the STFT coefficients of the dereverberated multi-channel audio signals are conveyed to the DNN-supported minimum variance distortionless response (MVDR) beamformer in which DNN-supported MVDR beamforming is carried out with the source and noise masks estimated by the DNN. As a result, the single-channel enhanced STFT coefficients are shown at the output and tossed to the CRNN-based SED system, and then, the three modules are jointly trained by the single loss function designed for SED. Furthermore, to ease the difficulty of training a deep learning model for SED caused by the imbalance in the amount of data for each class, the focal loss is used as a loss function. Experimental results show that joint training of DNN-supported dereverberation and beamforming with the SED model under the supervision of focal loss significantly improves the performance under the noisy and reverberant environments.
| S-EPMC7180550 | biostudies-literature
Project description:The aim of this work is to predict the complexity perception of real world images. We propose a new complexity measure where different image features, based on spatial, frequency and color properties are linearly combined. In order to find the optimal set of weighting coefficients we have applied a Particle Swarm Optimization. The optimal linear combination is the one that best fits the subjective data obtained in an experiment where observers evaluate the complexity of real world scenes on a web-based interface. To test the proposed complexity measure we have performed a second experiment on a different database of real world scenes, where the linear combination previously obtained is correlated with the new subjective data. Our complexity measure outperforms not only each single visual feature but also two visual clutter measures frequently used in the literature to predict image complexity. To analyze the usefulness of our proposal, we have also considered two different sets of stimuli composed of real texture images. Tuning the parameters of our measure for this kind of stimuli, we have obtained a linear combination that still outperforms the single measures. In conclusion our measure, properly tuned, can predict complexity perception of different kind of images.
| S-EPMC4919018 | biostudies-literature
Project description:Recently, visual number sense has been identified from deep neural networks (DNNs). However, whether DNNs have the same capacity for real-world scenes, rather than the simple geometric figures that are often tested, is unclear. In this study, we explore the number perception of scenes using AlexNet and find that numerosity can be represented by the pattern of group activation of the category layer units. The global activation of these units increases with the number of objects in the scene, and the variations in their activation decrease accordingly. By decoding the numerosity from this pattern, we reveal that the embedding coefficient of a scene determines the likelihood of potential objects to contribute to numerical perception. This was demonstrated by the more optimized performance for pictures with relatively high embedding coefficients in both DNNs and humans. This study for the first time shows that a distinct feature in visual environments, revealed by DNNs, can modulate human perception, supported by a group-coding mechanism.
| S-EPMC10407052 | biostudies-literature
Project description:Perception is thought to be shaped by the environments for which organisms are optimized. These influences are difficult to test in biological organisms but may be revealed by machine perceptual systems optimized under different conditions. We investigated environmental and physiological influences on pitch perception, whose properties are commonly linked to peripheral neural coding limits. We first trained artificial neural networks to estimate fundamental frequency from biologically faithful cochlear representations of natural sounds. The best-performing networks replicated many characteristics of human pitch judgments. To probe the origins of these characteristics, we then optimized networks given altered cochleae or sound statistics. Human-like behavior emerged only when cochleae had high temporal fidelity and when models were optimized for naturalistic sounds. The results suggest pitch perception is critically shaped by the constraints of natural environments in addition to those of the cochlea, illustrating the use of artificial neural networks to reveal underpinnings of behavior.
| S-EPMC8671597 | biostudies-literature
Project description:How do we construct a sense of place in a real-world environment? Real-world environments are actively explored via saccades, head turns, and body movements. Yet, little is known about how humans process real-world scene information during active viewing conditions. Here, we exploited recent developments in virtual reality (VR) and in-headset eye-tracking to test the impact of active vs. passive viewing conditions on gaze behavior while participants explored novel, real-world, 360° scenes. In one condition, participants actively explored 360° photospheres from a first-person perspective via self-directed motion (saccades and head turns). In another condition, photospheres were passively displayed to participants while they were head-restricted. We found that, relative to passive viewers, active viewers displayed increased attention to semantically meaningful scene regions, suggesting more exploratory, information-seeking gaze behavior. We also observed signatures of exploratory behavior in eye movements, such as quicker, more entropic fixations during active as compared with passive viewing conditions. These results show that active viewing influences every aspect of gaze behavior, from the way we move our eyes to what we choose to attend to. Moreover, these results offer key benchmark measurements of gaze behavior in 360°, naturalistic environments.
| S-EPMC7459302 | biostudies-literature
Project description:Exposure to even a single episode of loud noise can damage synapses between cochlear hair cells and auditory nerve fibres, causing hidden hearing loss (HHL) that is not detected by audiometry. Here we investigate the effects of noise-induced HHL on functional hearing by measuring the ability of neurons in the auditory midbrain of mice to adapt to sound environments containing quiet and loud periods. Neurons from noise-exposed mice show less capacity for adaptation to loud environments, convey less information about sound intensity in those environments, and adaptation to the longer-term statistical structure of fluctuating sound environments is impaired. Adaptation comprises a cascade of both threshold and gain adaptation. Although noise exposure only impairs threshold adaptation directly, the preserved function of gain adaptation surprisingly aggravates coding deficits for loud environments. These deficits might help to understand why many individuals with seemingly normal hearing struggle to follow a conversation in background noise.
| S-EPMC6191434 | biostudies-literature
Project description:Perceptual load is a well-established determinant of attentional engagement in a task. So far, perceptual load has typically been manipulated by increasing either the number of task-relevant items or the perceptual processing demand (e.g. conjunction versus feature tasks). The tasks used often involved rather simple visual displays (e.g. letters or single objects). How can perceptual load be operationalized for richer, real-world images? A promising proxy is the visual complexity of an image. However, current predictive models for visual complexity have limited applicability to diverse real-world images. Here we modelled visual complexity using a deep convolutional neural network (CNN) trained to learn perceived ratings of visual complexity. We presented 53 observers with 4000 images from the PASCAL VOC dataset, obtaining 75 020 2-alternative forced choice paired comparisons across observers. Image visual complexity scores were obtained using the TrueSkill algorithm. A CNN with weights pre-trained on an object recognition task predicted complexity ratings with r = 0.83. By contrast, feature-based models used in the literature, working on image statistics such as entropy, edge density and JPEG compression ratio, only achieved r = 0.70. Thus, our model offers a promising method to quantify the perceptual load of real-world scenes through visual complexity.
| S-EPMC7277246 | biostudies-literature
Project description:There is a growing body of evidence that important aspects of human cognition have been marginalized, or overlooked, by traditional cognitive science. In particular, the use of laboratory-based experiments in which stimuli are artificial, and response options are fixed, inevitably results in findings that are less ecologically valid in relation to real-world behavior. In the present review we highlight the opportunities provided by a range of new mobile technologies that allow traditionally lab-bound measurements to now be collected during natural interactions with the world. We begin by outlining the theoretical support that mobile approaches receive from the development of embodied accounts of cognition, and we review the widening evidence that illustrates the importance of examining cognitive processes in their context. As we acknowledge, in practice, the development of mobile approaches brings with it fresh challenges, and will undoubtedly require innovation in paradigm design and analysis. If successful, however, the mobile cognition approach will offer novel insights in a range of areas, including understanding the cognitive processes underlying navigation through space and the role of attention during natural behavior. We argue that the development of real-world mobile cognition offers both increased ecological validity, and the opportunity to examine the interactions between perception, cognition and action-rather than examining each in isolation.
| S-EPMC5226959 | biostudies-other
Project description:Understanding movies and stories requires maintaining a high-level situation model that abstracts away from perceptual details to describe the location, characters, actions, and causal relationships of the currently unfolding event. These models are built not only from information present in the current narrative, but also from prior knowledge about schematic event scripts, which describe typical event sequences encountered throughout a lifetime. We analyzed fMRI data from 44 human subjects (male and female) presented with 16 three-minute stories, consisting of four schematic events drawn from two different scripts (eating at a restaurant or going through the airport). Aside from this shared script structure, the stories varied widely in terms of their characters and storylines, and were presented in two highly dissimilar formats (audiovisual clips or spoken narration). One group was presented with the stories in an intact temporal sequence, while a separate control group was presented with the same events in scrambled order. Regions including the posterior medial cortex, medial prefrontal cortex (mPFC), and superior frontal gyrus exhibited schematic event patterns that generalized across stories, subjects, and modalities. Patterns in mPFC were also sensitive to overall script structure, with temporally scrambled events evoking weaker schematic representations. Using a Hidden Markov Model, patterns in these regions predicted the script (restaurant vs airport) of unlabeled data with high accuracy and were used to temporally align multiple stories with a shared script. These results extend work on the perception of controlled, artificial schemas in human and animal experiments to naturalistic perception of complex narratives.SIGNIFICANCE STATEMENT In almost all situations we encounter in our daily lives, we are able to draw on our schematic knowledge about what typically happens in the world to better perceive and mentally represent our ongoing experiences. In contrast to previous studies that investigated schematic cognition using simple, artificial associations, we measured brain activity from subjects watching movies and listening to stories depicting restaurant or airport experiences. Our results reveal a network of brain regions that is sensitive to the shared temporal structure of these naturalistic situations. These regions abstract away from the particular details of each story, activating a representation of the general type of situation being perceived.
| S-EPMC6222059 | biostudies-literature