Deep neural network models of sound localization reveal how perception is adapted to real-world environments.
Ontology highlight
ABSTRACT:
SUBMITTER: Francl A
PROVIDER: S-EPMC8830739 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Sound delivery is a key aspect of immersivity in virtual and augmented reality (VR/AR), with studies hinting at a correlation between users' ability to locate sounds around them and the 'feeling of being there'. This is particularly true for WebVR, a method of delivering immersive experiences through a local web browser that has recently captured attention in multiple industries. In WebVR, audio is the main spatial cue. Designers need to select the correct number of sound sources so that users perceive the location of incoming sound correctly. Information on how users localize sound is essential. Sound localization experiments, so far, have been run only in empty spaces or closed rooms, without clear indications for designers in WebVR. Thus, in this study, we investigate sound localization directly through WebVR. To do so, we designed a traditional empty room for training and a city-like virtual environment for testing purposes. In our paper, we also discuss key design parameters, differences in perception for vertical and horizontal directions, the impact of training, and the role of changing virtual environments. In addition, we introduce and test a new sound cue along with the traditional pink noise sound to measure and explore the impact of different sound cues in different environments. The results demonstrate the potential of exploring sound localization using WebVR, and our study will support the development of virtual experiences in human-computer interaction that may be able to reach a large number of participants using a local web browser.
| S-EPMC9287443 | biostudies-literature
Project description:Tinnitus therapies have been combined with the use of varieties of sound/noise. For masking external sounds, location of the masker in space is important; however, effects of the spatial location of the masker on tinnitus are less understood. We aimed to test whether a masking sound location would affect the perception level of simulated tinnitus. The 4 kHz simulated tinnitus was induced in the right ear of healthy volunteers through an open-type earphone. White noise was presented to the right ear using a single-sided headphone or a speaker positioned on the right side at a distance of 1.8 m for masking the simulated tinnitus. In other sessions, monaurally recorded noise localized within the head (inside-head noise) or binaurally recorded noise localized outside the head (outside-head noise) was separately presented from a dual-sided headphone. The noise presented from a distant speaker and the outside-head noise masked the simulated tinnitus in 71.1% and 77.1% of measurements at a lower intensity compared to the noise beside the ear and the inside-head noise, respectively. In conclusion, spatial information regarding the masking noise may play a role in reducing the perception level of simulated tinnitus. Binaurally recorded sounds may be beneficial for an acoustic therapy of tinnitus.
| S-EPMC8795453 | biostudies-literature
Project description:In this paper, we propose joint optimization of deep neural network (DNN)-supported dereverberation and beamforming for the convolutional recurrent neural network (CRNN)-based sound event detection (SED) in multi-channel environments. First, the short-time Fourier transform (STFT) coefficients are calculated from multi-channel audio signals under the noisy and reverberant environments, which are then enhanced by the DNN-supported weighted prediction error (WPE) dereverberation with the estimated masks. Next, the STFT coefficients of the dereverberated multi-channel audio signals are conveyed to the DNN-supported minimum variance distortionless response (MVDR) beamformer in which DNN-supported MVDR beamforming is carried out with the source and noise masks estimated by the DNN. As a result, the single-channel enhanced STFT coefficients are shown at the output and tossed to the CRNN-based SED system, and then, the three modules are jointly trained by the single loss function designed for SED. Furthermore, to ease the difficulty of training a deep learning model for SED caused by the imbalance in the amount of data for each class, the focal loss is used as a loss function. Experimental results show that joint training of DNN-supported dereverberation and beamforming with the SED model under the supervision of focal loss significantly improves the performance under the noisy and reverberant environments.
| S-EPMC7180550 | biostudies-literature
Project description:Background & motivationHousehold chaos is an established risk factor for child development. However, current methods for measuring household chaos rely on parent surveys, meaning existing research efforts cannot disentangle potentially dynamic bidirectional relations between high chaos environments and child behavior problems.Proposed approachWe train and make publicly available a classifier to provide objective, high-resolution predictions of household chaos from real-world child-worn audio recordings. To do so, we collect and annotate a novel dataset of ground-truth auditory chaos labels compiled from over 411 h of daylong recordings collected via audio recorders worn by N=22 infants in their homes. We leverage an existing sound event classifier to identify candidate high chaos segments, increasing annotation efficiency 8.32× relative to random sampling.ResultOur best-performing model successfully classifies four levels of real-world household auditory chaos with a macro F1 score of 0.701 (Precision: 0.705, Recall: 0.702) and a weighted F1 score of 0.679 (Precision: 0.685, Recall: 0.680).SignificanceIn future work, high-resolution objective chaos predictions from our model can be leveraged for basic science and intervention, including testing theorized mechanisms by which chaos affects children's cognition and behavior. Additionally, to facilitate further model development we make publicly available the first and largest balanced annotated audio dataset of real-world household chaos.
| S-EPMC10764466 | biostudies-literature
Project description:The aim of this work is to predict the complexity perception of real world images. We propose a new complexity measure where different image features, based on spatial, frequency and color properties are linearly combined. In order to find the optimal set of weighting coefficients we have applied a Particle Swarm Optimization. The optimal linear combination is the one that best fits the subjective data obtained in an experiment where observers evaluate the complexity of real world scenes on a web-based interface. To test the proposed complexity measure we have performed a second experiment on a different database of real world scenes, where the linear combination previously obtained is correlated with the new subjective data. Our complexity measure outperforms not only each single visual feature but also two visual clutter measures frequently used in the literature to predict image complexity. To analyze the usefulness of our proposal, we have also considered two different sets of stimuli composed of real texture images. Tuning the parameters of our measure for this kind of stimuli, we have obtained a linear combination that still outperforms the single measures. In conclusion our measure, properly tuned, can predict complexity perception of different kind of images.
| S-EPMC4919018 | biostudies-literature
Project description:Perception is thought to be shaped by the environments for which organisms are optimized. These influences are difficult to test in biological organisms but may be revealed by machine perceptual systems optimized under different conditions. We investigated environmental and physiological influences on pitch perception, whose properties are commonly linked to peripheral neural coding limits. We first trained artificial neural networks to estimate fundamental frequency from biologically faithful cochlear representations of natural sounds. The best-performing networks replicated many characteristics of human pitch judgments. To probe the origins of these characteristics, we then optimized networks given altered cochleae or sound statistics. Human-like behavior emerged only when cochleae had high temporal fidelity and when models were optimized for naturalistic sounds. The results suggest pitch perception is critically shaped by the constraints of natural environments in addition to those of the cochlea, illustrating the use of artificial neural networks to reveal underpinnings of behavior.
| S-EPMC8671597 | biostudies-literature
Project description:Recently, visual number sense has been identified from deep neural networks (DNNs). However, whether DNNs have the same capacity for real-world scenes, rather than the simple geometric figures that are often tested, is unclear. In this study, we explore the number perception of scenes using AlexNet and find that numerosity can be represented by the pattern of group activation of the category layer units. The global activation of these units increases with the number of objects in the scene, and the variations in their activation decrease accordingly. By decoding the numerosity from this pattern, we reveal that the embedding coefficient of a scene determines the likelihood of potential objects to contribute to numerical perception. This was demonstrated by the more optimized performance for pictures with relatively high embedding coefficients in both DNNs and humans. This study for the first time shows that a distinct feature in visual environments, revealed by DNNs, can modulate human perception, supported by a group-coding mechanism.
| S-EPMC10407052 | biostudies-literature
Project description:Exposure to even a single episode of loud noise can damage synapses between cochlear hair cells and auditory nerve fibres, causing hidden hearing loss (HHL) that is not detected by audiometry. Here we investigate the effects of noise-induced HHL on functional hearing by measuring the ability of neurons in the auditory midbrain of mice to adapt to sound environments containing quiet and loud periods. Neurons from noise-exposed mice show less capacity for adaptation to loud environments, convey less information about sound intensity in those environments, and adaptation to the longer-term statistical structure of fluctuating sound environments is impaired. Adaptation comprises a cascade of both threshold and gain adaptation. Although noise exposure only impairs threshold adaptation directly, the preserved function of gain adaptation surprisingly aggravates coding deficits for loud environments. These deficits might help to understand why many individuals with seemingly normal hearing struggle to follow a conversation in background noise.
| S-EPMC6191434 | biostudies-literature
Project description:How do we construct a sense of place in a real-world environment? Real-world environments are actively explored via saccades, head turns, and body movements. Yet, little is known about how humans process real-world scene information during active viewing conditions. Here, we exploited recent developments in virtual reality (VR) and in-headset eye-tracking to test the impact of active vs. passive viewing conditions on gaze behavior while participants explored novel, real-world, 360° scenes. In one condition, participants actively explored 360° photospheres from a first-person perspective via self-directed motion (saccades and head turns). In another condition, photospheres were passively displayed to participants while they were head-restricted. We found that, relative to passive viewers, active viewers displayed increased attention to semantically meaningful scene regions, suggesting more exploratory, information-seeking gaze behavior. We also observed signatures of exploratory behavior in eye movements, such as quicker, more entropic fixations during active as compared with passive viewing conditions. These results show that active viewing influences every aspect of gaze behavior, from the way we move our eyes to what we choose to attend to. Moreover, these results offer key benchmark measurements of gaze behavior in 360°, naturalistic environments.
| S-EPMC7459302 | biostudies-literature
Project description:Perceptual load is a well-established determinant of attentional engagement in a task. So far, perceptual load has typically been manipulated by increasing either the number of task-relevant items or the perceptual processing demand (e.g. conjunction versus feature tasks). The tasks used often involved rather simple visual displays (e.g. letters or single objects). How can perceptual load be operationalized for richer, real-world images? A promising proxy is the visual complexity of an image. However, current predictive models for visual complexity have limited applicability to diverse real-world images. Here we modelled visual complexity using a deep convolutional neural network (CNN) trained to learn perceived ratings of visual complexity. We presented 53 observers with 4000 images from the PASCAL VOC dataset, obtaining 75 020 2-alternative forced choice paired comparisons across observers. Image visual complexity scores were obtained using the TrueSkill algorithm. A CNN with weights pre-trained on an object recognition task predicted complexity ratings with r = 0.83. By contrast, feature-based models used in the literature, working on image statistics such as entropy, edge density and JPEG compression ratio, only achieved r = 0.70. Thus, our model offers a promising method to quantify the perceptual load of real-world scenes through visual complexity.
| S-EPMC7277246 | biostudies-literature