Dataset Information

Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement.

ABSTRACT: In a social world, observing the actions of others is fundamental to understanding what they are doing, as well as their intentions and feelings. Studies of the neural basis and decoding of action observation are important for understanding action-related processes and have implications for cognitive, social neuroscience, and human-machine interaction (HMI). In the current study, we first investigated temporal-spatial dynamics during action observation using a combined 64-channel electroencephalography (EEG) and 48-channel functional near-infrared spectroscopy (fNIRS) system. We measured brain activation while 16 healthy participants observed three action tasks: (1) grasping a cup with the intention of drinking; (2) grasping a cup with the intention of moving it; and (3) touching a cup with an unclear intention. The EEG and fNIRS source analysis results revealed the dynamic involvement of both the mirror neuron system (MNS) and the theory of mind (ToM)/mentalizing network during action observation. The source analysis results suggested that the extent to which these two systems were engaged was determined by the clarity of the intention of the observed action. Based on the difference in neural activity observed among different action-observation tasks in the first experiment, we conducted a second experiment to classify the neural processes underlying action observation using a feature classification method. We constructed complex brain networks based on the EEG and fNIRS data. Fusing features from both EEG and fNIRS complex brain networks resulted in a classification accuracy of 72.7% for the three action observation tasks. This study provides a theoretical and empirical basis for elucidating the neural mechanisms of action observation and intention understanding, and a feasible method for decoding the underlying neural processes.

SUBMITTER: Ge S

PROVIDER: S-EPMC6803538 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement.

Ge Sheng S Wang Peng P Liu Hui H Lin Pan P Gao Junfeng J Wang Ruimin R Iramina Keiji K Zhang Quan Q Zheng Wenming W

Frontiers in human neuroscience 20191015

In a social world, observing the actions of others is fundamental to understanding what they are doing, as well as their intentions and feelings. Studies of the neural basis and decoding of action observation are important for understanding action-related processes and have implications for cognitive, social neuroscience, and human-machine interaction (HMI). In the current study, we first investigated temporal-spatial dynamics during action observation using a combined 64-channel electroencephal ...[more]

PMID: 31680910

Similar Datasets

Project description:Motor imagery (MI) and action observation have proven to be efficacious adjuncts to traditional physiotherapy for enhancing motor recovery following stroke. Recently, researchers have used a combined approach called imagined imitation (II), where an individual watches a motor task being performed, while simultaneously imagining they are performing the movement. While neurofeedback (NFB) has been used extensively with MI to improve patients' ability to modulate sensorimotor activity and enhance motor recovery, the effectiveness of using NFB with II to modulate brain activity is unknown. This project tested the ability of participants to modulate sensorimotor activity during electroencephalography-based II-NFB of a complex, multi-part unilateral handshake, and whether this ability transferred to a subsequent bout of MI. Moreover, given the goal of translating findings from NFB research into practical applications, such as rehabilitation, the II-NFB system was designed with several user interface and user experience features, in an attempt to both drive user engagement and match the level of challenge to the abilities of the subjects. In particular, at easy difficulty levels the II-NFB system incentivized contralateral sensorimotor up-regulation (via event related desynchronization of the mu rhythm), while at higher difficulty levels the II-NFB system incentivized sensorimotor lateralization (i.e., both contralateral up-regulation and ipsilateral down-regulation). Thirty-two subjects, receiving real or sham NFB attended four sessions where they engaged in II-NFB training and subsequent MI. Results showed the NFB group demonstrated more bilateral sensorimotor activity during sessions 2-4 during II-NFB and subsequent MI, indicating mixed success for the implementation of this particular II-NFB system. Here we discuss our findings in the context of the design features included in the II-NFB system, highlighting limitations that should be considered in future designs.

Project description:Material properties, such as softness or stickiness, determine how an object can be used. Based on our real-life experience, we form strong expectations about how objects should behave under force, given their typical material properties. Such expectations have been shown to modulate perceptual processes, but we currently do not know how expectation influences the temporal dynamics of the cortical visual analysis for objects and their materials. Here, we tracked the neural representations of expected and unexpected material behaviors using time-resolved EEG decoding in a violation-of-expectation paradigm, where objects fell to the ground and deformed in expected or unexpected ways. Participants were 25 men and women. Our study yielded three key results: First, both objects and materials were represented rapidly and in a temporally sustained fashion. Second, objects exhibiting unexpected material behaviors were more successfully decoded than objects exhibiting expected behaviors within 190 ms after the impact, which might indicate additional processing demands when expectations are unmet. Third, general signals of expectation fulfillment that generalize across specific objects and materials were found within the first 150 ms after the impact. Together, our results provide new insights into the temporal neural processing cascade that underlies the analysis of real-world material behaviors. They reveal a sequence of predictions, with cortical signals progressing from a general signature of expectation fulfillment toward increased processing of unexpected material behaviors.SIGNIFICANCE STATEMENT In the real world, we can make accurate predictions about how an object's material shapes its behavior: For instance, we know that cups are typically made of porcelain and shatter when we accidentally drop them. Here, we use EEG to experimentally test how expectations about material behaviors impact neural processing. We showed our participants videos of objects that exhibited expected material behaviors (e.g., a glass shattering when falling to the ground) or unexpected material behaviors (e.g., a glass melting on impact). Our results reveal a hierarchy of predictions in cortex: The visual system rapidly generates signals that index whether expectations about material behaviors are met. These signals are followed by increased processing of objects displaying unexpected material behaviors.

Project description:Although efforts to characterize human movement through electroencephalography (EEG) have revealed neural activities unique to limb control that can be used to infer movement kinematics, it is still unknown the extent to which EEG can be used to discern the expressive qualities that influence such movements. In this study we used EEG and inertial sensors to record brain activity and movement of five skilled and certified Laban Movement Analysis (LMA) dancers. Each dancer performed whole body movements of three Action types: movements devoid of expressive qualities ("Neutral"), non-expressive movements while thinking about specific expressive qualities ("Think"), and enacted expressive movements ("Do"). The expressive movement qualities that were used in the "Think" and "Do" actions consisted of a sequence of eight Laban Effort qualities as defined by LMA-a notation system and language for describing, visualizing, interpreting and documenting all varieties of human movement. We used delta band (0.2-4 Hz) EEG as input to a machine learning algorithm that computed locality-preserving Fisher's discriminant analysis (LFDA) for dimensionality reduction followed by Gaussian mixture models (GMMs) to decode the type of Action. We also trained our LFDA-GMM models to classify all the possible combinations of Action Type and Laban Effort quality (giving a total of 17 classes). Classification accuracy rates were 59.4 ± 0.6% for Action Type and 88.2 ± 0.7% for Laban Effort quality Type. Ancillary analyses of the potential relations between the EEG and movement kinematics of the dancer's body, indicated that motion-related artifacts did not significantly influence our classification results. In summary, this research demonstrates that EEG has valuable information about the expressive qualities of movement. These results may have applications for advancing the understanding of the neural basis of expressive movements and for the development of neuroprosthetics to restore movements.

Dataset Information

Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement.

Publications

Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets