Personality first in emotion: a deep neural network based on electroencephalogram channel attention for cross-subject emotion recognition.
Ontology highlight
ABSTRACT:
SUBMITTER: Tian Z
PROVIDER: S-EPMC8371362 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND:Facial emotion recognition (FER) is essential to guide social functioning and behaviour for interpersonal communication. FER may be altered in severe mental illness such as in psychosis and in borderline personality disorder patients. However, it is unclear if these FER alterations are specifically related to psychosis. Awareness of FER alterations may be useful in clinical settings to improve treatment strategies. The aim of our study was to examine FER in patients with severe mental disorder and their relation with psychotic symptomatology. MATERIALS AND METHODS:Socio-demographic and clinical variables were collected. Alterations on emotion recognition were assessed in 3 groups: patients with first episode psychosis (FEP) (n = 64), borderline personality patients (BPD) (n = 37) and healthy controls (n = 137), using the Degraded Facial Affect Recognition Task. The Positive and Negative Syndrome Scale, Structured Interview for Schizotypy Revised and Community Assessment of Psychic Experiences scales were used to assess positive psychotic symptoms. WAIS III subtests were used to assess IQ. RESULTS:Kruskal-Wallis analysis showed a significant difference between groups on the FER of neutral faces score between FEP, BPD patients and controls and between FEP patients and controls in angry face recognition. No significant differences were found between groups in the fear or happy conditions. There was a significant difference between groups in the attribution of negative emotion to happy faces. BPD and FEP groups had a much higher tendency to recognize happy faces as negatives. There was no association with the different symptom domains in either group. CONCLUSIONS:FEP and BPD patients have problems in recognizing neutral faces more frequently than controls. Moreover, patients tend to over-report negative emotions in recognition of happy faces. Although no relation between psychotic symptoms and FER alterations was found, these deficits could contribute to a patient's misinterpretations in daily life.
| S-EPMC4965014 | biostudies-other
Project description:The electroencephalogram (EEG) has great attraction in emotion recognition studies due to its resistance to deceptive actions of humans. This is one of the most significant advantages of brain signals in comparison to visual or speech signals in the emotion recognition context. A major challenge in EEG-based emotion recognition is that EEG recordings exhibit varying distributions for different people as well as for the same person at different time instances. This nonstationary nature of EEG limits the accuracy of it when subject independency is the priority. The aim of this study is to increase the subject-independent recognition accuracy by exploiting pretrained state-of-the-art Convolutional Neural Network (CNN) architectures. Unlike similar studies that extract spectral band power features from the EEG readings, raw EEG data is used in our study after applying windowing, pre-adjustments and normalization. Removing manual feature extraction from the training system overcomes the risk of eliminating hidden features in the raw data and helps leverage the deep neural network's power in uncovering unknown features. To improve the classification accuracy further, a median filter is used to eliminate the false detections along a prediction interval of emotions. This method yields a mean cross-subject accuracy of 86.56% and 78.34% on the Shanghai Jiao Tong University Emotion EEG Dataset (SEED) for two and three emotion classes, respectively. It also yields a mean cross-subject accuracy of 72.81% on the Database for Emotion Analysis using Physiological Signals (DEAP) and 81.8% on the Loughborough University Multimodal Emotion Dataset (LUMED) for two emotion classes. Furthermore, the recognition model that has been trained using the SEED dataset was tested with the DEAP dataset, which yields a mean prediction accuracy of 58.1% across all subjects and emotion classes. Results show that in terms of classification accuracy, the proposed approach is superior to, or on par with, the reference subject-independent EEG emotion recognition studies identified in literature and has limited complexity due to the elimination of the need for feature extraction.
| S-EPMC7181114 | biostudies-literature
Project description:IntroductionRecently, the microstate analysis method has been widely used to investigate the temporal and spatial dynamics of electroencephalogram (EEG) signals. However, most studies have focused on EEG at resting state, and few use microstate analysis to study emotional EEG. This paper aims to investigate the temporal and spatial patterns of EEG in emotional states, and the specific neurophysiological significance of microstates during the emotion cognitive process, and further explore the feasibility and effectiveness of applying the microstate analysis to emotion recognition.MethodsWe proposed a KLGEV-criterion-based microstate analysis method, which can automatically and adaptively identify the optimal number of microstates in emotional EEG. The extracted temporal and spatial microstate features then served as novel feature sets to improve the performance of EEG emotion recognition. We evaluated the proposed method on two publicly available emotional EEG datasets: the SJTU Emotion EEG Dataset (SEED) and the Database for Emotion Analysis using Physiological Signals (DEAP).ResultsFor the SEED dataset, 10 microstates were identified using the proposed method. These temporal and spatial features were fed into AutoGluon, an open-source automatic machine learning model, yielding an average three-class accuracy of 70.38% (±8.03%) in subject-dependent emotion recognition. For the DEAP dataset, the method identified 9 microstates. The average accuracy in the arousal dimension was 74.33% (±5.17%) and 75.49% (±5.70%) in the valence dimension, which were competitive performance compared to some previous machine-learning-based studies. Based on these results, we further discussed the neurophysiological relationship between specific microstates and emotions, which broaden our knowledge of the interpretability of EEG microstates. In particular, we found that arousal ratings were positively correlated with the activity of microstate C (anterior regions of default mode network) and negatively correlated with the activity of microstate D (dorsal attention network), while valence ratings were positively correlated with the activity of microstate B (visual network) and negatively correlated with the activity of microstate D (dorsal attention network).DiscussionIn summary, the findings in this paper indicate that the proposed KLGEV-criterion-based method can be employed to research emotional EEG signals effectively, and the microstate features are promising feature sets for EEG-based emotion recognition.
| S-EPMC10972890 | biostudies-literature
Project description:Emotion recognition based on electroencephalograms has become an active research area. Yet, identifying emotions using only brainwaves is still very challenging, especially the subject-independent task. Numerous studies have tried to propose methods to recognize emotions, including machine learning techniques like convolutional neural network (CNN). Since CNN has shown its potential in generalization to unseen subjects, manipulating CNN hyperparameters like the window size and electrode order might be beneficial. To our knowledge, this is the first work that extensively observed the parameter selection effect on the CNN. The temporal information in distinct window sizes was found to significantly affect the recognition performance, and CNN was found to be more responsive to changing window sizes than the support vector machine. Classifying the arousal achieved the best performance with a window size of ten seconds, obtaining 56.85% accuracy and a Matthews correlation coefficient (MCC) of 0.1369. Valence recognition had the best performance with a window length of eight seconds at 73.34% accuracy and an MCC value of 0.4669. Spatial information from varying the electrode orders had a small effect on the classification. Overall, valence results had a much more superior performance than arousal results, which were, perhaps, influenced by features related to brain activity asymmetry between the left and right hemispheres.
| S-EPMC7957771 | biostudies-literature
Project description:Emotion states recognition using wireless signals is an emerging area of research that has an impact on neuroscientific studies of human behaviour and well-being monitoring. Currently, standoff emotion detection is mostly reliant on the analysis of facial expressions and/or eye movements acquired from optical or video cameras. Meanwhile, although they have been widely accepted for recognizing human emotions from the multimodal data, machine learning approaches have been mostly restricted to subject dependent analyses which lack of generality. In this paper, we report an experimental study which collects heartbeat and breathing signals of 15 participants from radio frequency (RF) reflections off the body followed by novel noise filtering techniques. We propose a novel deep neural network (DNN) architecture based on the fusion of raw RF data and the processed RF signal for classifying and visualising various emotion states. The proposed model achieves high classification accuracy of 71.67% for independent subjects with 0.71, 0.72 and 0.71 precision, recall and F1-score values respectively. We have compared our results with those obtained from five different classical ML algorithms and it is established that deep learning offers a superior performance even with limited amount of raw RF and post processed time-sequence data. The deep learning model has also been validated by comparing our results with those from ECG signals. Our results indicate that using wireless signals for stand-by emotion state detection is a better alternative to other technologies with high accuracy and have much wider applications in future studies of behavioural sciences.
| S-EPMC7857608 | biostudies-literature
Project description:The ventromedial prefrontal cortex is known to play a crucial role in regulating human social and emotional behaviour, yet the precise mechanisms by which it subserves this broad function remain unclear. Whereas previous neuropsychological studies have largely focused on the role of the ventromedial prefrontal cortex in higher-order deliberative processes related to valuation and decision-making, here we test whether ventromedial prefrontal cortex may also be critical for more basic aspects of orienting attention to socially and emotionally meaningful stimuli. Using eye tracking during a test of facial emotion recognition in a sample of lesion patients, we show that bilateral ventromedial prefrontal cortex damage impairs visual attention to the eye regions of faces, particularly for fearful faces. This finding demonstrates a heretofore unrecognized function of the ventromedial prefrontal cortex-the basic attentional process of controlling eye movements to faces expressing emotion.
| S-EPMC4032099 | biostudies-literature
Project description:Background: Facial emotion recognition is impaired in addiction and personality disorders. Dysfunctional personality beliefs reflect negative interpersonal schemas that may underpin emotion recognition deficits. We aimed to examine the association between personality beliefs and emotion recognition among participants with cocaine use disorder including those with comorbid personality disorders. Methods: We recruited 70 participants with cocaine use disorder aged between 19 and 52 who had used 14 g of cocaine over 4.8 years on average. Thirty-eight participants had an additional personality disorder (11 Borderline, 7 Histrionic, 5 Antisocial, 10 Avoidant, and 5 Obsessive-Compulsive). Dysfunctional beliefs were indicated with the Personality Belief Questionnaire, and facial emotion recognition was indicated with the Ekman's Test. We applied correlations/multiple regressions to test the relationship between beliefs and emotion recognition. Results: Personality beliefs reflecting paranoid, borderline, and antisocial schemas were negatively associated with emotion recognition. Antisocial beliefs were associated with poorer recognition of fear, and paranoid beliefs with poorer recognition of disgust. Antisocial beliefs were significantly associated with emotion recognition after adjusting for cocaine use. Conclusion: Dysfunctional personality beliefs are associated with poorer emotion recognition in cocaine addiction. Personality-related negative schemas about the self and others can impact social cognition and interaction during cocaine treatment.
| S-EPMC6591705 | biostudies-literature
Project description:This randomized controlled study examined neurological changes in socioemotional processing skills through parent training in caregivers of children with attention-deficit/hyperactivity disorder. Thirty mothers of children with attention-deficit/hyperactivity disorder were stratified into parent training and non-parent training groups. Functional magnetic resonance imaging was performed during the "Reading the Mind in the Eyes" test, and parenting difficulties were evaluated using the Parenting Stress Index and the Parenting Scale, twice (before and after parent training). Only mothers in the parent training group showed a significant decrease in Parenting Stress Index and Parenting Scale scores. They also demonstrated increased activity in the left occipital fusiform gyrus during the task of estimating emotions from facial pictures. We presumed that these changes might reflect the potential impact of enrollment in parent training in reducing stress, which might have increased activation of the fusiform gyrus.
| S-EPMC10435396 | biostudies-literature
Project description:Emotional awareness perception is a largely growing field that allows for more natural interactions between people and machines. Electroencephalography (EEG) has emerged as a convenient way to measure and track a user's emotional state. The non-linear characteristic of the EEG signal produces a high-dimensional feature vector resulting in high computational cost. In this paper, characteristics of multiple neural networks are combined using Deep Feature Clustering (DFC) to select high-quality attributes as opposed to traditional feature selection methods. The DFC method shortens the training time on the network by omitting unusable attributes. First, Empirical Mode Decomposition (EMD) is applied as a series of frequencies to decompose the raw EEG signal. The spatiotemporal component of the decomposed EEG signal is expressed as a two-dimensional spectrogram before the feature extraction process using Analytic Wavelet Transform (AWT). Four pre-trained Deep Neural Networks (DNN) are used to extract deep features. Dimensional reduction and feature selection are achieved utilising the differential entropy-based EEG channel selection and the DFC technique, which calculates a range of vocabularies using k-means clustering. The histogram characteristic is then determined from a series of visual vocabulary items. The classification performance of the SEED, DEAP and MAHNOB datasets combined with the capabilities of DFC show that the proposed method improves the performance of emotion recognition in short processing time and is more competitive than the latest emotion recognition methods.
| S-EPMC7374326 | biostudies-literature
Project description:Emotion recognition from electroencephalogram (EEG) signals requires accurate and efficient signal processing and feature extraction. Deep learning technology has enabled the automatic extraction of raw EEG signal features that contribute to classifying emotions more accurately. Despite such advances, classification of emotions from EEG signals, especially recorded during recalling specific memories or imagining emotional situations has not yet been investigated. In addition, high-density EEG signal classification using deep neural networks faces challenges, such as high computational complexity, redundant channels, and low accuracy. To address these problems, we evaluate the effects of using a simple channel selection method for classifying self-induced emotions based on deep learning. The experiments demonstrate that selecting key channels based on signal statistics can reduce the computational complexity by 89% without decreasing the classification accuracy. The channel selection method with the highest accuracy was the kurtosis-based method, which achieved accuracies of 79.03% and 79.36% for the valence and arousal scales, respectively. The experimental results show that the proposed framework outperforms conventional methods, even though it uses fewer channels. Our proposed method can be beneficial for the effective use of EEG signals in practical applications.
| S-EPMC9523358 | biostudies-literature