Emotion recognition based on customized smart bracelet with built-in accelerometer.
Ontology highlight
ABSTRACT: Background
Recently, emotion recognition has become a hot topic in human-computer interaction. If computers could understand human emotions, they could interact better with their users. This paper proposes a novel method to recognize human emotions (neutral, happy, and angry) using a smart bracelet with built-in accelerometer.Methods
In this study, a total of 123 participants were instructed to wear a customized smart bracelet with built-in accelerometer that can track and record their movements. Firstly, participants walked two minutes as normal, which served as walking behaviors in a neutral emotion condition. Participants then watched emotional film clips to elicit emotions (happy and angry). The time interval between watching two clips was more than four hours. After watching film clips, they walked for one minute, which served as walking behaviors in a happy or angry emotion condition. We collected raw data from the bracelet and extracted a few features from raw data. Based on these features, we built classification models for classifying three types of emotions (neutral, happy, and angry).Results and discussion
For two-category classification, the classification accuracy can reach 91.3% (neutral vs. angry), 88.5% (neutral vs. happy), and 88.5% (happy vs. angry), respectively; while, for the differentiation among three types of emotions (neutral, happy, and angry), the accuracy can reach 81.2%.Conclusions
Using wearable devices, we found it is possible to recognize human emotions (neutral, happy, and angry) with fair accuracy. Results of this study may be useful to improve the performance of human-computer interaction.
SUBMITTER: Zhang Z
PROVIDER: S-EPMC4974923 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Research in psychology has shown that the way a person walks reflects that person's current mood (or emotional state). Recent studies have used mobile phones to detect emotional states from movement data.The objective of our study was to investigate the use of movement sensor data from a smart watch to infer an individual's emotional state. We present our findings of a user study with 50 participants.The experimental design is a mixed-design study: within-subjects (emotions: happy, sad, and neutral) and between-subjects (stimulus type: audiovisual "movie clips" and audio "music clips"). Each participant experienced both emotions in a single stimulus type. All participants walked 250 m while wearing a smart watch on one wrist and a heart rate monitor strap on the chest. They also had to answer a short questionnaire (20 items; Positive Affect and Negative Affect Schedule, PANAS) before and after experiencing each emotion. The data obtained from the heart rate monitor served as supplementary information to our data. We performed time series analysis on data from the smart watch and a t test on questionnaire items to measure the change in emotional state. Heart rate data was analyzed using one-way analysis of variance. We extracted features from the time series using sliding windows and used features to train and validate classifiers that determined an individual's emotion.Overall, 50 young adults participated in our study; of them, 49 were included for the affective PANAS questionnaire and 44 for the feature extraction and building of personal models. Participants reported feeling less negative affect after watching sad videos or after listening to sad music, P<.006. For the task of emotion recognition using classifiers, our results showed that personal models outperformed personal baselines and achieved median accuracies higher than 78% for all conditions of the design study for binary classification of happiness versus sadness.Our findings show that we are able to detect changes in the emotional state as well as in behavioral responses with data obtained from the smartwatch. Together with high accuracies achieved across all users for classification of happy versus sad emotional states, this is further evidence for the hypothesis that movement sensor data can be used for emotion recognition.
| S-EPMC6105867 | biostudies-other
Project description:Objectives: Little is known about the physical activity of patients after cardiac surgery. This study was designed to assess this activity using a connected bracelet.Methods: In this prospective, monocentric study, patients scheduled for cardiac surgery were offered to wear an electronic bracelet. The main objective was to measure the physical activity recovery. Secondary objectives were the predictors of the correct use of the monitoring system, of the physical recovery and, if any, the relationship between physical activity and out-of-hospital morbidity.Results: One hundred patients were included. Most patients (86%) were interested in participating in the study. The compliance to the device and to the study protocol was good (94%). At discharge, the mean number of daily steps was 1454 ± 145 steps, increasing quite homogeneously, reaching 5801±1151 steps at Day 60. The best fit regression curve gave a maximum number of steps at 5897±119 (r2 = 0.97). The 85% level of activity was achieved at Day 30±3. No predictor of noncompliance was found. At discharge, age was independently associated with a lower number of daily steps (p <0.001). At Day 60, age, peripheral arterial disease and cardio-pulmonary bypass duration were independently associated with a lower number of daily steps (p = 0.039, p = 0.041 and p = 0.033, respectively).Conclusions: After cardiac surgery, wearing a smart bracelet recording daily steps is simple, well tolerated and suitable for measuring physical activity. Standard patients achieved around 6000 daily steps 2 months after discharge. 85% of this activity is reached in the first month.Clinical trial registry number: NCT03113565.
| S-EPMC7707519 | biostudies-literature
Project description:Hand gesture recognition (HGR) systems using electromyography (EMG) bracelet-type sensors are currently largely used over other HGR technologies. However, bracelets are susceptible to electrode rotation, causing a decrease in HGR performance. In this work, HGR systems with an algorithm for orientation correction are proposed. The proposed orientation correction method is based on the computation of the maximum energy channel using a synchronization gesture. Then, the channels of the EMG are rearranged in a new sequence which starts with the maximum energy channel. This new sequence of channels is used for both training and testing. After the EMG channels are rearranged, this signal passes through the following stages: pre-processing, feature extraction, classification, and post-processing. We implemented user-specific and user-general HGR models based on a common architecture which is robust to rotations of the EMG bracelet. Four experiments were performed, taking into account two different metrics which are the classification and recognition accuracy for both models implemented in this work, where each model was evaluated with and without rotation of the bracelet. The classification accuracy measures how well a model predicted which gesture is contained somewhere in a given EMG, whereas recognition accuracy measures how well a model predicted when it occurred, how long it lasted, and which gesture is contained in a given EMG. The results of the experiments (without and with orientation correction) executed show an increase in performance from 44.5% to 81.2% for classification and from 43.3% to 81.3% for recognition in user-general models, while in user-specific models, the results show an increase in performance from 39.8% to 94.9% for classification and from 38.8% to 94.2% for recognition. The results obtained in this work evidence that the proposed method for orientation correction makes the performance of an HGR robust to rotations of the EMG bracelet.
| S-EPMC7665113 | biostudies-literature
Project description:Accurate emotion recognition from speech is important for applications like smart health care, smart entertainment, and other smart services. High accuracy emotion recognition from Chinese speech is challenging due to the complexities of the Chinese language. In this paper, we explore how to improve the accuracy of speech emotion recognition, including speech signal feature extraction and emotion classification methods. Five types of features are extracted from a speech sample: mel frequency cepstrum coefficient (MFCC), pitch, formant, short-term zero-crossing rate and short-term energy. By comparing statistical features with deep features extracted by a Deep Belief Network (DBN), we attempt to find the best features to identify the emotion status for speech. We propose a novel classification method that combines DBN and SVM (support vector machine) instead of using only one of them. In addition, a conjugate gradient method is applied to train DBN in order to speed up the training process. Gender-dependent experiments are conducted using an emotional speech database created by the Chinese Academy of Sciences. The results show that DBN features can reflect emotion status better than artificial features, and our new classification approach achieves an accuracy of 95.8%, which is higher than using either DBN or SVM separately. Results also show that DBN can work very well for small training databases if it is properly designed.
| S-EPMC5539696 | biostudies-other
Project description:Purpose: Most built environment studies have quantified characteristics of the areas around participants' homes. However, the environmental exposures for physical activity (PA) are spatially dynamic rather than static. Thus, merged accelerometer and global positioning system (GPS) data were utilized to estimate associations between the built environment and PA among adults. Methods: Participants (N = 142) were recruited on trails in Massachusetts and wore an accelerometer and GPS unit for 1-4 days. Two binary outcomes were created: moderate-to-vigorous PA (MVPA vs. light PA-to-sedentary); and light-to-vigorous PA (LVPA vs. sedentary). Five built environment variables were created within 50-meter buffers around GPS points: population density, street density, land use mix (LUM), greenness, and walkability index. Generalized linear mixed models were fit to examine associations between environmental variables and both outcomes, adjusting for demographic covariates. Results: Overall, in the fully adjusted models, greenness was positively associated with MVPA and LVPA (odds ratios [ORs] = 1.15, 95% confidence interval [CI] = 1.03, 1.30 and 1.25, 95% CI = 1.12, 1.41, respectively). In contrast, street density and LUM were negatively associated with MVPA (ORs = 0.69, 95% CI = 0.67, 0.71 and 0.87, 95% CI = 0.78, 0.97, respectively) and LVPA (ORs = 0.79, 95% CI = 0.77, 0.81 and 0.81, 95% CI = 0.74, 0.90, respectively). Negative associations of population density and walkability with both outcomes reached statistical significance, yet the effect sizes were small. Conclusions: Concurrent monitoring of activity with accelerometers and GPS units allowed us to investigate relationships between objectively measured built environment around GPS points and minute-by-minute PA. Negative relationships between street density and LUM and PA contrast evidence from most built environment studies in adults. However, direct comparisons should be made with caution since most previous studies have focused on spatially fixed buffers around home locations, rather than the precise locations where PA occurs.
| S-EPMC6701185 | biostudies-literature
Project description:Visual contents such as movies and animation evoke various human emotions. We examine an argument that the emotion from the visual contents may vary according to the contrast control of the scenes contained in the contents. We sample three emotions including positive, neutral and negative to prove our argument. We also sample several scenes of these emotions from visual contents and control the contrast of the scenes. We manipulate the contrast of the scenes and measure the change of valence and arousal from human participants who watch the contents using a deep emotion recognition module based on electroencephalography (EEG) signals. As a result, we conclude that the enhancement of contrast induces the increase of valence, while the reduction of contrast induces the decrease. Meanwhile, the contrast control affects arousal on a very minute scale.
| S-EPMC7472638 | biostudies-literature
Project description:Accurate recognition and understating of human emotions is an essential skill that can improve the collaboration between humans and machines. In this vein, electroencephalogram (EEG)-based emotion recognition is considered an active research field with challenging issues regarding the analyses of the nonstationary EEG signals and the extraction of salient features that can be used to achieve accurate emotion recognition. In this paper, an EEG-based emotion recognition approach with a novel time-frequency feature extraction technique is presented. In particular, a quadratic time-frequency distribution (QTFD) is employed to construct a high resolution time-frequency representation of the EEG signals and capture the spectral variations of the EEG signals over time. To reduce the dimensionality of the constructed QTFD-based representation, a set of 13 time- and frequency-domain features is extended to the joint time-frequency-domain and employed to quantify the QTFD-based time-frequency representation of the EEG signals. Moreover, to describe different emotion classes, we have utilized the 2D arousal-valence plane to develop four emotion labeling schemes of the EEG signals, such that each emotion labeling scheme defines a set of emotion classes. The extracted time-frequency features are used to construct a set of subject-specific support vector machine classifiers to classify the EEG signals of each subject into the different emotion classes that are defined using each of the four emotion labeling schemes. The performance of the proposed approach is evaluated using a publicly available EEG dataset, namely the DEAPdataset. Moreover, we design three performance evaluation analyses, namely the channel-based analysis, feature-based analysis and neutral class exclusion analysis, to quantify the effects of utilizing different groups of EEG channels that cover various regions in the brain, reducing the dimensionality of the extracted time-frequency features and excluding the EEG signals that correspond to the neutral class, on the capability of the proposed approach to discriminate between different emotion classes. The results reported in the current study demonstrate the efficacy of the proposed QTFD-based approach in recognizing different emotion classes. In particular, the average classification accuracies obtained in differentiating between the various emotion classes defined using each of the four emotion labeling schemes are within the range of 73.8 % ? 86.2 % . Moreover, the emotion classification accuracies achieved by our proposed approach are higher than the results reported in several existing state-of-the-art EEG-based emotion recognition studies.
| S-EPMC6111567 | biostudies-literature
Project description:Music is often discussed to be emotional because it reflects expressive movements in audible form. Thus, a valid approach to measure musical emotion could be to assess movement stimulated by music. In two experiments we evaluated the discriminative power of mobile-device generated acceleration data produced by free movement during music listening for the prediction of ratings on the Geneva Emotion Music Scales (GEMS-9). The quality of prediction for different dimensions of GEMS varied between experiments for tenderness (R12(first experiment) = 0.50, R22(second experiment) = 0.39), nostalgia (R12 = 0.42, R22 = 0.30), wonder (R12 = 0.25, R22 = 0.34), sadness (R12 = 0.24, R22 = 0.35), peacefulness (R12 = 0.20, R22 = 0.35) and joy (R12 = 0.19, R22 = 0.33) and transcendence (R12 = 0.14, R22 = 0.00). For others like power (R12 = 0.42, R22 = 0.49) and tension (R12 = 0.28, R22 = 0.27) results could be almost reproduced. Furthermore, we extracted two principle components from GEMS ratings, one representing arousal and the other one valence of the experienced feeling. Both qualities, arousal and valence, could be predicted by acceleration data, indicating, that they provide information on the quantity and quality of experience. On the one hand, these findings show how music-evoked movement patterns relate to music-evoked feelings. On the other hand, they contribute to integrate findings from the field of embodied music cognition into music recommender systems.
| S-EPMC4945052 | biostudies-other
Project description:A thorough analysis of sports is becoming increasingly important during the training process of badminton players at both the recreational and professional level. Nowadays, game situations are usually filmed and reviewed afterwards in order to analyze the game situation, but these video set-ups tend to be difficult to analyze, expensive, and intrusive to set up. In contrast, we classified badminton movements using off-the-shelf accelerometer and gyroscope data. To this end, we organized a data capturing campaign and designed a novel neural network using different frame sizes as input. This paper shows that with only accelerometer data, our novel convolutional neural network is able to distinguish nine activities with 86% precision when using a sampling frequency of 50 Hz. Adding the gyroscope data causes an increase of up to 99% precision, as compared to, respectively, 79% and 88% when using a traditional convolutional neural network. In addition, our paper analyses the impact of different sensor placement options and discusses the impact of different sampling frequenciess of the sensors. As such, our approach provides a low cost solution that is easy to use and can collect useful information for the analysis of a badminton game.
| S-EPMC7506561 | biostudies-literature
Project description:Accelerometers have been widely embedded in most current mobile devices, enabling easy and intuitive operations. This paper proposes a Motion Gesture Recognition system (MGRA) based on accelerometer data only, which is entirely implemented on mobile devices and can provide users with real-time interactions. A robust and unique feature set is enumerated through the time domain, the frequency domain and singular value decomposition analysis using our motion gesture set containing 11,110 traces. The best feature vector for classification is selected, taking both static and mobile scenarios into consideration. MGRA exploits support vector machine as the classifier with the best feature vector. Evaluations confirm that MGRA can accommodate a broad set of gesture variations within each class, including execution time, amplitude and non-gestural movement. Extensive evaluations confirm that MGRA achieves higher accuracy under both static and mobile scenarios and costs less computation time and energy on an LG Nexus 5 than previous methods.
| S-EPMC4851044 | biostudies-literature