Emotion recognition based on customized smart bracelet with built-in accelerometer.
Ontology highlight
ABSTRACT: Background
Recently, emotion recognition has become a hot topic in human-computer interaction. If computers could understand human emotions, they could interact better with their users. This paper proposes a novel method to recognize human emotions (neutral, happy, and angry) using a smart bracelet with built-in accelerometer.Methods
In this study, a total of 123 participants were instructed to wear a customized smart bracelet with built-in accelerometer that can track and record their movements. Firstly, participants walked two minutes as normal, which served as walking behaviors in a neutral emotion condition. Participants then watched emotional film clips to elicit emotions (happy and angry). The time interval between watching two clips was more than four hours. After watching film clips, they walked for one minute, which served as walking behaviors in a happy or angry emotion condition. We collected raw data from the bracelet and extracted a few features from raw data. Based on these features, we built classification models for classifying three types of emotions (neutral, happy, and angry).Results and discussion
For two-category classification, the classification accuracy can reach 91.3% (neutral vs. angry), 88.5% (neutral vs. happy), and 88.5% (happy vs. angry), respectively; while, for the differentiation among three types of emotions (neutral, happy, and angry), the accuracy can reach 81.2%.Conclusions
Using wearable devices, we found it is possible to recognize human emotions (neutral, happy, and angry) with fair accuracy. Results of this study may be useful to improve the performance of human-computer interaction.
SUBMITTER: Zhang Z
PROVIDER: S-EPMC4974923 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Research in psychology has shown that the way a person walks reflects that person's current mood (or emotional state). Recent studies have used mobile phones to detect emotional states from movement data.The objective of our study was to investigate the use of movement sensor data from a smart watch to infer an individual's emotional state. We present our findings of a user study with 50 participants.The experimental design is a mixed-design study: within-subjects (emotions: happy, sad, and neutral) and between-subjects (stimulus type: audiovisual "movie clips" and audio "music clips"). Each participant experienced both emotions in a single stimulus type. All participants walked 250 m while wearing a smart watch on one wrist and a heart rate monitor strap on the chest. They also had to answer a short questionnaire (20 items; Positive Affect and Negative Affect Schedule, PANAS) before and after experiencing each emotion. The data obtained from the heart rate monitor served as supplementary information to our data. We performed time series analysis on data from the smart watch and a t test on questionnaire items to measure the change in emotional state. Heart rate data was analyzed using one-way analysis of variance. We extracted features from the time series using sliding windows and used features to train and validate classifiers that determined an individual's emotion.Overall, 50 young adults participated in our study; of them, 49 were included for the affective PANAS questionnaire and 44 for the feature extraction and building of personal models. Participants reported feeling less negative affect after watching sad videos or after listening to sad music, P<.006. For the task of emotion recognition using classifiers, our results showed that personal models outperformed personal baselines and achieved median accuracies higher than 78% for all conditions of the design study for binary classification of happiness versus sadness.Our findings show that we are able to detect changes in the emotional state as well as in behavioral responses with data obtained from the smartwatch. Together with high accuracies achieved across all users for classification of happy versus sad emotional states, this is further evidence for the hypothesis that movement sensor data can be used for emotion recognition.
| S-EPMC6105867 | biostudies-other
Project description:Objectives: Little is known about the physical activity of patients after cardiac surgery. This study was designed to assess this activity using a connected bracelet.Methods: In this prospective, monocentric study, patients scheduled for cardiac surgery were offered to wear an electronic bracelet. The main objective was to measure the physical activity recovery. Secondary objectives were the predictors of the correct use of the monitoring system, of the physical recovery and, if any, the relationship between physical activity and out-of-hospital morbidity.Results: One hundred patients were included. Most patients (86%) were interested in participating in the study. The compliance to the device and to the study protocol was good (94%). At discharge, the mean number of daily steps was 1454 ± 145 steps, increasing quite homogeneously, reaching 5801±1151 steps at Day 60. The best fit regression curve gave a maximum number of steps at 5897±119 (r2 = 0.97). The 85% level of activity was achieved at Day 30±3. No predictor of noncompliance was found. At discharge, age was independently associated with a lower number of daily steps (p <0.001). At Day 60, age, peripheral arterial disease and cardio-pulmonary bypass duration were independently associated with a lower number of daily steps (p = 0.039, p = 0.041 and p = 0.033, respectively).Conclusions: After cardiac surgery, wearing a smart bracelet recording daily steps is simple, well tolerated and suitable for measuring physical activity. Standard patients achieved around 6000 daily steps 2 months after discharge. 85% of this activity is reached in the first month.Clinical trial registry number: NCT03113565.
| S-EPMC7707519 | biostudies-literature
Project description:Emotion recognition is key to interpersonal communication and to human-machine interaction. Body expression may contribute to emotion recognition, but most past studies focused on a few motions, limiting accurate recognition. Moreover, emotions in most previous research were acted out, resulting in non-natural motion, which is unapplicable in reality. We present an approach for emotion recognition based on body motion in naturalistic settings, examining authentic emotions, natural movement, and a broad collection of motion parameters. A lab experiment using 24 participants manipulated participants' emotions using pretested movies into five conditions: happiness, relaxation, fear, sadness, and emotionally-neutral. Emotion was manipulated within subjects, with fillers in between and a counterbalanced order. A motion capture system measured posture and motion during standing and walking; a force plate measured center of pressure location. Traditional statistics revealed nonsignificant effects of emotions on most motion parameters; only 7 of 229 parameters demonstrate significant effects. Most significant effects are in parameters representing postural control during standing, which is consistent with past studies. Yet, the few significant effects suggest that it is impossible to recognize emotions based on a single motion parameter. We therefore developed machine learning models to classify emotions using a collection of parameters, and examined six models: k-nearest neighbors, decision tree, logistic regression, and the support vector machine with radial base function and linear and polynomial functions. The decision tree using 25 parameters provided the highest average accuracy (45.8%), more than twice the random guess for five conditions, which advances past studies demonstrating comparable accuracies, due to our naturalistic setting. This research suggests that machine learning models are valuable for emotion recognition in reality and lays the foundation for further progress in emotion recognition models, informing the development of recognition devices (e.g., depth camera), to be used in home-setting human-machine interactions.
| S-EPMC10499259 | biostudies-literature
Project description:Hand gesture recognition (HGR) systems using electromyography (EMG) bracelet-type sensors are currently largely used over other HGR technologies. However, bracelets are susceptible to electrode rotation, causing a decrease in HGR performance. In this work, HGR systems with an algorithm for orientation correction are proposed. The proposed orientation correction method is based on the computation of the maximum energy channel using a synchronization gesture. Then, the channels of the EMG are rearranged in a new sequence which starts with the maximum energy channel. This new sequence of channels is used for both training and testing. After the EMG channels are rearranged, this signal passes through the following stages: pre-processing, feature extraction, classification, and post-processing. We implemented user-specific and user-general HGR models based on a common architecture which is robust to rotations of the EMG bracelet. Four experiments were performed, taking into account two different metrics which are the classification and recognition accuracy for both models implemented in this work, where each model was evaluated with and without rotation of the bracelet. The classification accuracy measures how well a model predicted which gesture is contained somewhere in a given EMG, whereas recognition accuracy measures how well a model predicted when it occurred, how long it lasted, and which gesture is contained in a given EMG. The results of the experiments (without and with orientation correction) executed show an increase in performance from 44.5% to 81.2% for classification and from 43.3% to 81.3% for recognition in user-general models, while in user-specific models, the results show an increase in performance from 39.8% to 94.9% for classification and from 38.8% to 94.2% for recognition. The results obtained in this work evidence that the proposed method for orientation correction makes the performance of an HGR robust to rotations of the EMG bracelet.
| S-EPMC7665113 | biostudies-literature
Project description:Accurate emotion recognition from speech is important for applications like smart health care, smart entertainment, and other smart services. High accuracy emotion recognition from Chinese speech is challenging due to the complexities of the Chinese language. In this paper, we explore how to improve the accuracy of speech emotion recognition, including speech signal feature extraction and emotion classification methods. Five types of features are extracted from a speech sample: mel frequency cepstrum coefficient (MFCC), pitch, formant, short-term zero-crossing rate and short-term energy. By comparing statistical features with deep features extracted by a Deep Belief Network (DBN), we attempt to find the best features to identify the emotion status for speech. We propose a novel classification method that combines DBN and SVM (support vector machine) instead of using only one of them. In addition, a conjugate gradient method is applied to train DBN in order to speed up the training process. Gender-dependent experiments are conducted using an emotional speech database created by the Chinese Academy of Sciences. The results show that DBN features can reflect emotion status better than artificial features, and our new classification approach achieves an accuracy of 95.8%, which is higher than using either DBN or SVM separately. Results also show that DBN can work very well for small training databases if it is properly designed.
| S-EPMC5539696 | biostudies-other
Project description:Purpose: Most built environment studies have quantified characteristics of the areas around participants' homes. However, the environmental exposures for physical activity (PA) are spatially dynamic rather than static. Thus, merged accelerometer and global positioning system (GPS) data were utilized to estimate associations between the built environment and PA among adults. Methods: Participants (N = 142) were recruited on trails in Massachusetts and wore an accelerometer and GPS unit for 1-4 days. Two binary outcomes were created: moderate-to-vigorous PA (MVPA vs. light PA-to-sedentary); and light-to-vigorous PA (LVPA vs. sedentary). Five built environment variables were created within 50-meter buffers around GPS points: population density, street density, land use mix (LUM), greenness, and walkability index. Generalized linear mixed models were fit to examine associations between environmental variables and both outcomes, adjusting for demographic covariates. Results: Overall, in the fully adjusted models, greenness was positively associated with MVPA and LVPA (odds ratios [ORs] = 1.15, 95% confidence interval [CI] = 1.03, 1.30 and 1.25, 95% CI = 1.12, 1.41, respectively). In contrast, street density and LUM were negatively associated with MVPA (ORs = 0.69, 95% CI = 0.67, 0.71 and 0.87, 95% CI = 0.78, 0.97, respectively) and LVPA (ORs = 0.79, 95% CI = 0.77, 0.81 and 0.81, 95% CI = 0.74, 0.90, respectively). Negative associations of population density and walkability with both outcomes reached statistical significance, yet the effect sizes were small. Conclusions: Concurrent monitoring of activity with accelerometers and GPS units allowed us to investigate relationships between objectively measured built environment around GPS points and minute-by-minute PA. Negative relationships between street density and LUM and PA contrast evidence from most built environment studies in adults. However, direct comparisons should be made with caution since most previous studies have focused on spatially fixed buffers around home locations, rather than the precise locations where PA occurs.
| S-EPMC6701185 | biostudies-literature
Project description:Visual contents such as movies and animation evoke various human emotions. We examine an argument that the emotion from the visual contents may vary according to the contrast control of the scenes contained in the contents. We sample three emotions including positive, neutral and negative to prove our argument. We also sample several scenes of these emotions from visual contents and control the contrast of the scenes. We manipulate the contrast of the scenes and measure the change of valence and arousal from human participants who watch the contents using a deep emotion recognition module based on electroencephalography (EEG) signals. As a result, we conclude that the enhancement of contrast induces the increase of valence, while the reduction of contrast induces the decrease. Meanwhile, the contrast control affects arousal on a very minute scale.
| S-EPMC7472638 | biostudies-literature
Project description:Accurate recognition and understating of human emotions is an essential skill that can improve the collaboration between humans and machines. In this vein, electroencephalogram (EEG)-based emotion recognition is considered an active research field with challenging issues regarding the analyses of the nonstationary EEG signals and the extraction of salient features that can be used to achieve accurate emotion recognition. In this paper, an EEG-based emotion recognition approach with a novel time-frequency feature extraction technique is presented. In particular, a quadratic time-frequency distribution (QTFD) is employed to construct a high resolution time-frequency representation of the EEG signals and capture the spectral variations of the EEG signals over time. To reduce the dimensionality of the constructed QTFD-based representation, a set of 13 time- and frequency-domain features is extended to the joint time-frequency-domain and employed to quantify the QTFD-based time-frequency representation of the EEG signals. Moreover, to describe different emotion classes, we have utilized the 2D arousal-valence plane to develop four emotion labeling schemes of the EEG signals, such that each emotion labeling scheme defines a set of emotion classes. The extracted time-frequency features are used to construct a set of subject-specific support vector machine classifiers to classify the EEG signals of each subject into the different emotion classes that are defined using each of the four emotion labeling schemes. The performance of the proposed approach is evaluated using a publicly available EEG dataset, namely the DEAPdataset. Moreover, we design three performance evaluation analyses, namely the channel-based analysis, feature-based analysis and neutral class exclusion analysis, to quantify the effects of utilizing different groups of EEG channels that cover various regions in the brain, reducing the dimensionality of the extracted time-frequency features and excluding the EEG signals that correspond to the neutral class, on the capability of the proposed approach to discriminate between different emotion classes. The results reported in the current study demonstrate the efficacy of the proposed QTFD-based approach in recognizing different emotion classes. In particular, the average classification accuracies obtained in differentiating between the various emotion classes defined using each of the four emotion labeling schemes are within the range of 73.8 % ? 86.2 % . Moreover, the emotion classification accuracies achieved by our proposed approach are higher than the results reported in several existing state-of-the-art EEG-based emotion recognition studies.
| S-EPMC6111567 | biostudies-literature
Project description:Emotion recognition from electroencephalogram (EEG) signals requires accurate and efficient signal processing and feature extraction. Deep learning technology has enabled the automatic extraction of raw EEG signal features that contribute to classifying emotions more accurately. Despite such advances, classification of emotions from EEG signals, especially recorded during recalling specific memories or imagining emotional situations has not yet been investigated. In addition, high-density EEG signal classification using deep neural networks faces challenges, such as high computational complexity, redundant channels, and low accuracy. To address these problems, we evaluate the effects of using a simple channel selection method for classifying self-induced emotions based on deep learning. The experiments demonstrate that selecting key channels based on signal statistics can reduce the computational complexity by 89% without decreasing the classification accuracy. The channel selection method with the highest accuracy was the kurtosis-based method, which achieved accuracies of 79.03% and 79.36% for the valence and arousal scales, respectively. The experimental results show that the proposed framework outperforms conventional methods, even though it uses fewer channels. Our proposed method can be beneficial for the effective use of EEG signals in practical applications.
| S-EPMC9523358 | biostudies-literature
Project description:Physical activity patterns can reveal information about one's health status. Built-in sensors in a smartphone, in comparison to a patient's self-report, can collect activity recognition data more objectively, unobtrusively, and continuously. A variety of data analysis approaches have been proposed in the literature. In this study, we applied the movelet method to classify the activities performed using smartphone accelerometer and gyroscope data, which measure a phone's acceleration and angular velocity, respectively. The movelet method constructs a personalized dictionary for each participant using training data and classifies activities in new data with the dictionary. Our results show that this method has the advantages of being interpretable and transparent. A unique aspect of our movelet application involves extracting unique information, optimally, from multiple sensors. In comparison to single-sensor applications, our approach jointly incorporates the accelerometer and gyroscope sensors with the movelet method. Our findings show that combining data from the two sensors can result in more accurate activity recognition than using each sensor alone. In particular, the joint-sensor method reduces errors of the gyroscope-only method in differentiating between standing and sitting. It also reduces errors in the accelerometer-only method when classifying vigorous activities.
| S-EPMC9002497 | biostudies-literature