Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Multi-Modality Emotion Recognition Model with GAT-Based Multi-Head Inter-Modality Attention.

ABSTRACT: Emotion recognition has been gaining attention in recent years due to its applications on artificial agents. To achieve a good performance with this task, much research has been conducted on the multi-modality emotion recognition model for leveraging the different strengths of each modality. However, a research question remains: what exactly is the most appropriate way to fuse the information from different modalities? In this paper, we proposed audio sample augmentation and an emotion-oriented encoder-decoder to improve the performance of emotion recognition and discussed an inter-modality, decision-level fusion method based on a graph attention network (GAT). Compared to the baseline, our model improved the weighted average F1-scores from 64.18 to 68.31% and the weighted average accuracy from 65.25 to 69.88%.

SUBMITTER: Fu C

PROVIDER: S-EPMC7506856 | biostudies-literature | 2020 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Multi-Modality Emotion Recognition Model with GAT-Based Multi-Head Inter-Modality Attention.

Fu Changzeng C Liu Chaoran C Ishi Carlos Toshinori CT Ishiguro Hiroshi H

Sensors (Basel, Switzerland) 20200829 17

Emotion recognition has been gaining attention in recent years due to its applications on artificial agents. To achieve a good performance with this task, much research has been conducted on the multi-modality emotion recognition model for leveraging the different strengths of each modality. However, a research question remains: what exactly is the most appropriate way to fuse the information from different modalities? In this paper, we proposed audio sample augmentation and an emotion-oriented ...[more]

PMID: 32872511

Similar Datasets

Personality first in emotion: a deep neural network based on electroencephalogram channel attention for cross-subject emotion recognition.

Project description:In recent years, more and more researchers have focused on emotion recognition methods based on electroencephalogram (EEG) signals. However, most studies only consider the spatio-temporal characteristics of EEG and the modelling based on this feature, without considering personality factors, let alone studying the potential correlation between different subjects. Considering the particularity of emotions, different individuals may have different subjective responses to the same physical stimulus. Therefore, emotion recognition methods based on EEG signals should tend to be personalized. This paper models the personalized EEG emotion recognition from the macro and micro levels. At the macro level, we use personality characteristics to classify the individuals' personalities from the perspective of 'birds of a feather flock together'. At the micro level, we employ deep learning models to extract the spatio-temporal feature information of EEG. To evaluate the effectiveness of our method, we conduct an EEG emotion recognition experiment on the ASCERTAIN dataset. Our experimental results demonstrate that the recognition accuracy of our proposed method is 72.4% and 75.9% on valence and arousal, respectively, which is 10.2% and 9.1% higher than that of no consideration of personalization.

| S-EPMC8371362 | biostudies-literature

Ventromedial prefrontal cortex mediates visual attention during facial emotion recognition.

Project description:The ventromedial prefrontal cortex is known to play a crucial role in regulating human social and emotional behaviour, yet the precise mechanisms by which it subserves this broad function remain unclear. Whereas previous neuropsychological studies have largely focused on the role of the ventromedial prefrontal cortex in higher-order deliberative processes related to valuation and decision-making, here we test whether ventromedial prefrontal cortex may also be critical for more basic aspects of orienting attention to socially and emotionally meaningful stimuli. Using eye tracking during a test of facial emotion recognition in a sample of lesion patients, we show that bilateral ventromedial prefrontal cortex damage impairs visual attention to the eye regions of faces, particularly for fearful faces. This finding demonstrates a heretofore unrecognized function of the ventromedial prefrontal cortex-the basic attentional process of controlling eye movements to faces expressing emotion.

| S-EPMC4032099 | biostudies-literature

DM3Loc: multi-label mRNA subcellular localization prediction and analysis based on multi-head self-attention mechanism.

Project description:Subcellular localization of messenger RNAs (mRNAs), as a prevalent mechanism, gives precise and efficient control for the translation process. There is mounting evidence for the important roles of this process in a variety of cellular events. Computational methods for mRNA subcellular localization prediction provide a useful approach for studying mRNA functions. However, few computational methods were designed for mRNA subcellular localization prediction and their performance have room for improvement. Especially, there is still no available tool to predict for mRNAs that have multiple localization annotations. In this paper, we propose a multi-head self-attention method, DM3Loc, for multi-label mRNA subcellular localization prediction. Evaluation results show that DM3Loc outperforms existing methods and tools in general. Furthermore, DM3Loc has the interpretation ability to analyze RNA-binding protein motifs and key signals on mRNAs for subcellular localization. Our analyses found hundreds of instances of mRNA isoform-specific subcellular localizations and many significantly enriched gene functions for mRNAs in different subcellular localizations.

| S-EPMC8096227 | biostudies-literature

ATM-TCR: TCR-Epitope Binding Affinity Prediction Using a Multi-Head Self-Attention Model.

Project description:TCR-epitope pair binding is the key component for T cell regulation. The ability to predict whether a given pair binds is fundamental to understanding the underlying biology of the binding mechanism as well as developing T-cell mediated immunotherapy approaches. The advent of large-scale public databases containing TCR-epitope binding pairs enabled the recent development of computational prediction methods for TCR-epitope binding. However, the number of epitopes reported along with binding TCRs is far too small, resulting in poor out-of-sample performance for unseen epitopes. In order to address this issue, we present our model ATM-TCR which uses a multi-head self-attention mechanism to capture biological contextual information and improve generalization performance. Additionally, we present a novel application of the attention map from our model to improve out-of-sample performance by demonstrating on recent SARS-CoV-2 data.

| S-EPMC9299376 | biostudies-literature

Face-Based Attention Recognition Model for Children with Autism Spectrum Disorder.

Project description:Attention recognition plays a vital role in providing learning support for children with autism spectrum disorders (ASD). The unobtrusiveness of face-tracking techniques makes it possible to build automatic systems to detect and classify attentional behaviors. However, constructing such systems is a challenging task due to the complexity of attentional behavior in ASD. This paper proposes a face-based attention recognition model using two methods. The first is based on geometric feature transformation using a support vector machine (SVM) classifier, and the second is based on the transformation of time-domain spatial features to 2D spatial images using a convolutional neural network (CNN) approach. We conducted an experimental study on different attentional tasks for 46 children (ASD n=20, typically developing children n=26) and explored the limits of the face-based attention recognition model for participant and task differences. Our results show that the geometric feature transformation using an SVM classifier outperforms the CNN approach. Also, attention detection is more generalizable within typically developing children than within ASD groups and within low-attention tasks than within high-attention tasks. This paper highlights the basis for future face-based attentional recognition for real-time learning and clinical attention interventions.Supplementary informationThe online version contains supplementary material available at 10.1007/s41666-021-00101-y.

| S-EPMC8982782 | biostudies-literature

An Innovative Multi-Model Neural Network Approach for Feature Selection in Emotion Recognition Using Deep Feature Clustering.

Project description:Emotional awareness perception is a largely growing field that allows for more natural interactions between people and machines. Electroencephalography (EEG) has emerged as a convenient way to measure and track a user's emotional state. The non-linear characteristic of the EEG signal produces a high-dimensional feature vector resulting in high computational cost. In this paper, characteristics of multiple neural networks are combined using Deep Feature Clustering (DFC) to select high-quality attributes as opposed to traditional feature selection methods. The DFC method shortens the training time on the network by omitting unusable attributes. First, Empirical Mode Decomposition (EMD) is applied as a series of frequencies to decompose the raw EEG signal. The spatiotemporal component of the decomposed EEG signal is expressed as a two-dimensional spectrogram before the feature extraction process using Analytic Wavelet Transform (AWT). Four pre-trained Deep Neural Networks (DNN) are used to extract deep features. Dimensional reduction and feature selection are achieved utilising the differential entropy-based EEG channel selection and the DFC technique, which calculates a range of vocabularies using k-means clustering. The histogram characteristic is then determined from a series of visual vocabulary items. The classification performance of the SEED, DEAP and MAHNOB datasets combined with the capabilities of DFC show that the proposed method improves the performance of emotion recognition in short processing time and is more competitive than the latest emotion recognition methods.

| S-EPMC7374326 | biostudies-literature

Sparse Granger Causality Analysis Model Based on Sensors Correlation for Emotion Recognition Classification in Electroencephalography.

Project description:In recent years, affective computing based on electroencephalogram (EEG) data has attracted increased attention. As a classic EEG feature extraction model, Granger causality analysis has been widely used in emotion classification models, which construct a brain network by calculating the causal relationships between EEG sensors and select the key EEG features. Traditional EEG Granger causality analysis uses the L 2 norm to extract features from the data, and so the results are susceptible to EEG artifacts. Recently, several researchers have proposed Granger causality analysis models based on the least absolute shrinkage and selection operator (LASSO) and the L 1/2 norm to solve this problem. However, the conventional sparse Granger causality analysis model assumes that the connections between each sensor have the same prior probability. This paper shows that if the correlation between the EEG data from each sensor can be added to the Granger causality network as prior knowledge, the EEG feature selection ability and emotional classification ability of the sparse Granger causality model can be enhanced. Based on this idea, we propose a new emotional computing model, named the sparse Granger causality analysis model based on sensor correlation (SC-SGA). SC-SGA integrates the correlation between sensors as prior knowledge into the Granger causality analysis based on the L 1/2 norm framework for feature extraction, and uses L 2 norm logistic regression as the emotional classification algorithm. We report the results of experiments using two real EEG emotion datasets. These results demonstrate that the emotion classification accuracy of the SC-SGA model is better than that of existing models by 2.46-21.81%.

| S-EPMC8358835 | biostudies-literature

LncReader: identification of dual functional long non-coding RNAs using multi-head self-attention mechanism

Project description:This SuperSeries is composed of the SubSeries listed below.

2022-04-26 | GSE201485 | GEO

Modality-specific attention in foraging bumblebees.

Project description:Attentional demands can prevent humans and other animals from performing multiple tasks simultaneously. Some studies, however, show that tasks presented in different sensory modalities (e.g. visual and auditory) can be processed simultaneously. This suggests that, at least in these cases, attention might be modality-specific and divided differently between tasks when present in the same modality compared with different modalities. We investigated this possibility in bumblebees (Bombus terrestris) using a biologically relevant experimental set-up where they had to simultaneously choose more rewarding flowers and avoid simulated predatory attacks by robotic 'spiders'. We found that when the tasks had to be performed using visual cues alone, bees failed to perform both tasks simultaneously. However, when highly rewarding flowers were indicated by olfactory cues and predators were indicated by visual cues, bees managed to perform both tasks successfully. Our results thus provide evidence for modality-specific attention in foraging bees and establish a novel framework for future studies of crossmodal attention in ecologically realistic settings.

| S-EPMC4632517 | biostudies-literature

How facial masks alter the interaction of gaze direction, head orientation, and emotion recognition.

Project description:The COVID-19 pandemic has altered the way we interact with each other: mandatory mask-wearing obscures facial information that is crucial for emotion recognition. Whereas the influence of wearing a mask on emotion recognition has been repeatedly investigated, little is known about the impact on interaction effects among emotional signals and other social signals. Therefore, the current study sought to explore how gaze direction, head orientation, and emotional expression interact with respect to emotion perception, and how these interactions are altered by wearing a face mask. In two online experiments, we presented face stimuli from the Radboud Faces Database displaying different facial expressions (anger, fear, happiness, neutral, and sadness), gaze directions (-13°, 0°, and 13°), and head orientations (-45°, 0°, and 45°) - either without (Experiment 1) or with mask (Experiment 2). Participants categorized the displayed emotional expressions. Not surprisingly, masks impaired emotion recognition. Surprisingly, without the mask, emotion recognition was unaffected by averted head orientations and only slightly affected by gaze direction. The mask strongly interfered with this ability. The mask increased the influence of head orientation and gaze direction, in particular for the emotions that were poorly recognized with mask. The results suggest that in case of uncertainty due to ambiguity or absence of signals, we seem to unconsciously factor in extraneous information.

| S-EPMC9533556 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data