Hyperrealistic neural decoding for reconstructing faces from fMRI activations via the GAN latent space.
Ontology highlight
ABSTRACT:
SUBMITTER: Dado T
PROVIDER: S-EPMC8741893 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Although distinct categories are reliably decoded from fMRI brain responses, it has proved more difficult to distinguish visually similar inputs, such as different faces. Here, we apply a recently developed deep learning system to reconstruct face images from human fMRI. We trained a variational auto-encoder (VAE) neural network using a GAN (Generative Adversarial Network) unsupervised procedure over a large data set of celebrity faces. The auto-encoder latent space provides a meaningful, topologically organized 1024-dimensional description of each image. We then presented several thousand faces to human subjects, and learned a simple linear mapping between the multi-voxel fMRI activation patterns and the 1024 latent dimensions. Finally, we applied this mapping to novel test images, translating fMRI patterns into VAE latent codes, and codes into face reconstructions. The system not only performed robust pairwise decoding (>95% correct), but also accurate gender classification, and even decoded which face was imagined, rather than seen.
| S-EPMC6529435 | biostudies-literature
Project description:The way spatial memory retrieval is represented in the brain remains unclear to date. Previous studies have displayed a hippocampus-centered navigation network using functional magnetic resonance imaging (fMRI) analysis. There have been some studies on the representation of navigation behavior by signal distribution patterns, but only in the hippocampus and adjacent structures. In this study, we aimed to determine (1) the brain regions that represent information in both intensity and distribution patterns during spatial memory retrieval and (2) whether the patterns of neural responses represent spatial memory retrieval behavior performance. Both univariate analysis [general linear model (GLM)] and multivariate pattern analysis (MVPA) were employed to reveal the spatial distributions of brain responses elicited by spatial memory retrieval. Correlation analyses were performed to detect the correspondences between brain responses and behavior performance. We found that spatial memory retrieval occurred in widespread brain regions, including the bilateral hippocampi, bilateral superior frontal gyrus, bilateral superior parietal lobules, bilateral occipital lobes, and cerebellum. The amplitude of activation in the left hippocampus showed a significant negative correlation (r = -0.46, p = 0.039) with the number of task completions. Additionally, within-subject classification accuracies based on the blood oxygenation level-dependent (BOLD) signal patterns of the right middle temporal gyrus (rMTG) rostral areas in the Brainnetome Atlas showed a significant positive correlation (r = 0.78, p < 0.0001) with retrieval accuracy. In summary, our findings have implications for understanding the separation between navigational and non-navigational states and emphasizing the utility of MVPA in the whole brain.
| S-EPMC8168467 | biostudies-literature
Project description:Social anxiety disorder (SAD), which is characterized by the fear of being rejected and negatively evaluated, involves altered brain activation during the processing of negative emotions in a social context. Although associated temperament traits, such as shyness or behavioral inhibition, have been studied, there is still insufficient knowledge to support the dimensional approach, which assumes a continuum from subclinical to clinical levels of social anxiety symptoms. This study used functional magnetic resonance imaging (fMRI) to examine the neural bases of individual differences in social anxiety. Our sample included participants with both healthy/subclinical as well as clinical levels of social anxiety. Forty-six participants with a wide range of social anxiety levels performed a gender decision task with emotional facial expressions during fMRI scanning. Activation in the left anterior insula and right lateral prefrontal cortex in response to angry faces was positively correlated with the level of social anxiety in a regression analysis. The results substantiate, with a dimensional approach, those obtained in previous studies that involved SAD patients or healthy and subclinical participants. It may help to refine further therapeutic strategies based on markers of social anxiety.
| S-EPMC4040104 | biostudies-literature
Project description:We demonstrate that multivoxel pattern analysis can be used to decode place-related information in fMRI. Subjects performed a working memory version of the Morris water maze task in a virtual environment with a single wall cue. The voxel data that corresponds to when subjects were located at the goal was extracted for seven regions implicated in spatial navigation, and then used to train a pattern classifier based on partial least squares. Using a leave-one-out (LOO) test procedure, goal locations at E, W, N positions (relative to the cue as S) were predicted significantly better than a naïve classifier for voxels in medial prefrontal cortex, hippocampus, and inferior parietal cortex. Prediction with voxels from other regions involved in navigation was also better than a naïve classifier, which raises the possibility that goal-location information is widely disseminated among the navigation network. It turns out that predictive capability of all regions combined significantly decreases, relative to no change, only when voxel data from the hippocampus is left out. This implies that the hippocampus contains some unique information that identifies goal locations, whereas other regions contain information that also identifies goal locations but is more redundant. Classification of goal locations is an important step toward decoding a variety of place-related information in spatial navigation with fMRI.
| S-EPMC2826536 | biostudies-literature
Project description:Previous studies have demonstrated that migraine is associated with enhanced perception and altered cerebral processing of sensory stimuli. More recently, it has been suggested that this sensory hypersensitivity might reflect a more general enhanced response to aversive emotional stimuli. Using functional magnetic resonance imaging and emotional face stimuli (fearful, happy and sad faces), we compared whole-brain activation between 41 migraine patients without aura in interictal period and 49 healthy controls. Migraine patients showed increased neural activation to fearful faces compared to neutral faces in the right middle frontal gyrus and frontal pole relative to healthy controls. We also found that higher attack frequency in migraine patients was related to increased activation mainly in the right primary somatosensory cortex (corresponding to the face area) to fearful expressions and in the right dorsal striatal regions to happy faces. In both analyses, activation differences remained significant after controlling for anxiety and depressive symptoms. These findings indicate that enhanced response to emotional stimuli might explain the migraine trigger effect of psychosocial stressors that gradually leads to increased somatosensory response to emotional clues and thus contributes to the progression or chronification of migraine.
| S-EPMC6462777 | biostudies-literature
Project description:IntroductionOxytocin is a key mediator of emotional and social behavior that seems to be of relevance for the development and maintenance of addictive behaviors. We thus investigated the effect of oxytocin on neural response and behavior during a face-matching task in a sample of social drinkers.MethodsThirteen social drinkers underwent a randomized double-blind placebo-controlled cross-over functional magnetic resonance imaging face-matching task with and without prior intranasal application of 24 international units oxytocin. Effects of oxytocin and task condition (faces, shapes) on brain activation and individual task performance were assessed.ResultsFace-matching compared to shape-matching trials resulted in higher brain activation in the bilateral amygdala, hippocampus and parts of the occipital gyri. Oxytocin application vs. placebo reduced activation in bilateral amygdala, parts of the frontal gyri, and the parietal lobe. Region of interest analyses indicated that the oxytocin-induced attenuation of amygdala response was specific to face-stimuli and associated with lower subjective alcohol craving, and a lower percentage of heavy-drinking days (defined as ≥ 5 standard drinks/day).ConclusionFor the first time, we could show that a larger oxytocin-induced attenuation of amygdala response to fearful faces is associated with lower subjective craving for alcohol and percentage of heavy drinking days in social drinkers. Modulation of amygdala activation, induced by emotional stimuli, might represent a neurobiological substrate of oxytocin's protective effects on drug seeking behavior.
| S-EPMC8236029 | biostudies-literature
Project description:Blind source separation (BSS) is commonly used in functional magnetic resonance imaging (fMRI) data analysis. Recently, BSS models based on restricted Boltzmann machine (RBM), one of the building blocks of deep learning models, have been shown to improve brain network identification compared to conventional single matrix factorization models such as independent component analysis (ICA). These models, however, trained RBM on fMRI volumes, and are hence challenged by model complexity and limited training set. In this article, we propose to apply RBM to fMRI time courses instead of volumes for BSS. The proposed method not only interprets fMRI time courses explicitly to take advantages of deep learning models in latent feature learning but also substantially reduces model complexity and increases the scale of training set to improve training efficiency. Our experimental results based on Human Connectome Project (HCP) datasets demonstrated the superiority of the proposed method over ICA and the one that applied RBM to fMRI volumes in identifying task-related components, resulted in more accurate and specific representations of task-related activations. Moreover, our method separated out components representing intermixed effects between task events, which could reflect inherent interactions among functionally connected brain regions. Our study demonstrates the value of RBM in mining complex structures embedded in large-scale fMRI data and its potential as a building block for deeper models in fMRI data analysis.
| S-EPMC6866484 | biostudies-literature
Project description:Multivariate neuroimaging studies indicate that the brain represents word and object concepts in a format that readily generalises across stimuli. Here we investigated whether this was true for neural representations of simple events described using sentences. Participants viewed sentences describing four events in different ways. Multivariate classifiers were trained to discriminate the four events using a subset of sentences, allowing us to test generalisation to novel sentences. We found that neural patterns in a left-lateralised network of frontal, temporal and parietal regions discriminated events in a way that generalised successfully over changes in the syntactic and lexical properties of the sentences used to describe them. In contrast, decoding in visual areas was sentence-specific and failed to generalise to novel sentences. In the reverse analysis, we tested for decoding of syntactic and lexical structure, independent of the event being described. Regions displaying this coding were limited and largely fell outside the canonical semantic network. Our results indicate that a distributed neural network represents the meaning of event sentences in a way that is robust to changes in their structure and form. They suggest that the semantic system disregards the surface properties of stimuli in order to represent their underlying conceptual significance.
| S-EPMC8270886 | biostudies-literature
Project description:Brain decoding, aiming to identify the brain states using neural activity, is important for cognitive neuroscience and neural engineering. However, existing machine learning methods for fMRI-based brain decoding either suffer from low classification performance or poor explainability. Here, we address this issue by proposing a biologically inspired architecture, Spatial Temporal-pyramid Graph Convolutional Network (STpGCN), to capture the spatial-temporal graph representation of functional brain activities. By designing multi-scale spatial-temporal pathways and bottom-up pathways that mimic the information process and temporal integration in the brain, STpGCN is capable of explicitly utilizing the multi-scale temporal dependency of brain activities via graph, thereby achieving high brain decoding performance. Additionally, we propose a sensitivity analysis method called BrainNetX to better explain the decoding results by automatically annotating task-related brain regions from the brain-network standpoint. We conduct extensive experiments on fMRI data under 23 cognitive tasks from Human Connectome Project (HCP) S1200. The results show that STpGCN significantly improves brain-decoding performance compared to competing baseline models; BrainNetX successfully annotates task-relevant brain regions. Post hoc analysis based on these regions further validates that the hierarchical structure in STpGCN significantly contributes to the explainability, robustness and generalization of the model. Our methods not only provide insights into information representation in the brain under multiple cognitive tasks but also indicate a bright future for fMRI-based brain decoding.
| S-EPMC10089104 | biostudies-literature
Project description:Although it is well established that prior experience with faces determines their subsequent social-emotional evaluation, recent work shows that top-down inhibitory mechanisms, including response inhibition, can lead to social devaluation after even a single, brief exposure. These rapidly induced effects indicate interplay among perceptual, attentional, response-selection and social-emotional networks; yet, the brain mechanisms underlying this are not well understood. This study used functional magnetic resonance imaging (fMRI) to investigate the neural mechanism mediating the relationship between inhibitory control and emotional devaluation. Participants performed two tasks: (i) a Go/No-Go task in response to faces and (ii) a trustworthiness rating task involving the previously seen faces. No-Go faces were rated as significantly less trustworthy than Go faces. By examining brain activations during Task 1, behavioral measures and brain activations obtained in Task 2 could be predicted. Specifically, activity in brain areas during Task 1 associated with (i) executive control and response suppression (i.e. lateral prefrontal cortex) and (ii) affective responses and value representation (i.e. orbitofrontal cortex), systematically covaried with behavioral ratings and amygdala activity obtained during Task 2. The present findings offer insights into the neural mechanisms linking inhibitory processes to affective responses.
| S-EPMC3427860 | biostudies-literature