Dataset Information

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features.

ABSTRACT: The functional independence of individuals with upper limb impairment could be enhanced by teleoperated robots that can assist with activities of daily living. However, robot control is not always intuitive for the operator. In this work, eye gaze was leveraged as a natural way to infer human intent and advance action recognition for shared autonomy control schemes. We introduced a classifier structure for recognizing low-level action primitives that incorporates novel three-dimensional gaze-related features. We defined an action primitive as a triplet comprised of a verb, target object, and hand object. A recurrent neural network was trained to recognize a verb and target object, and was tested on three different activities. For a representative activity (making a powdered drink), the average recognition accuracy was 77% for the verb and 83% for the target object. Using a non-specific approach to classifying and indexing objects in the workspace, we observed a modest level of generalizability of the action primitive classifier across activities, including those for which the classifier was not trained. The novel input features of gaze object angle and its rate of change were especially useful for accurately recognizing action primitives and reducing the observational latency of the classifier.

SUBMITTER: Wang X

PROVIDER: S-EPMC7593660 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features.

Wang Xiaoyu X Haji Fathaliyan Alireza A Santos Veronica J VJ

Frontiers in neurorobotics 20201015

The functional independence of individuals with upper limb impairment could be enhanced by teleoperated robots that can assist with activities of daily living. However, robot control is not always intuitive for the operator. In this work, eye gaze was leveraged as a natural way to infer human intent and advance action recognition for shared autonomy control schemes. We introduced a classifier structure for recognizing low-level action primitives that incorporates novel three-dimensional gaze-rel ...[more]

PMID: 33178006

Similar Datasets

Project description:Human-robot collaboration could be advanced by facilitating the intuitive, gaze-based control of robots, and enabling robots to recognize human actions, infer human intent, and plan actions that support human goals. Traditionally, gaze tracking approaches to action recognition have relied upon computer vision-based analyses of two-dimensional egocentric camera videos. The objective of this study was to identify useful features that can be extracted from three-dimensional (3D) gaze behavior and used as inputs to machine learning algorithms for human action recognition. We investigated human gaze behavior and gaze-object interactions in 3D during the performance of a bimanual, instrumental activity of daily living: the preparation of a powdered drink. A marker-based motion capture system and binocular eye tracker were used to reconstruct 3D gaze vectors and their intersection with 3D point clouds of objects being manipulated. Statistical analyses of gaze fixation duration and saccade size suggested that some actions (pouring and stirring) may require more visual attention than other actions (reach, pick up, set down, and move). 3D gaze saliency maps, generated with high spatial resolution for six subtasks, appeared to encode action-relevant information. The "gaze object sequence" was used to capture information about the identity of objects in concert with the temporal sequence in which the objects were visually regarded. Dynamic time warping barycentric averaging was used to create a population-based set of characteristic gaze object sequences that accounted for intra- and inter-subject variability. The gaze object sequence was used to demonstrate the feasibility of a simple action recognition algorithm that utilized a dynamic time warping Euclidean distance metric. Averaged over the six subtasks, the action recognition algorithm yielded an accuracy of 96.4%, precision of 89.5%, and recall of 89.2%. This level of performance suggests that the gaze object sequence is a promising feature for action recognition whose impact could be enhanced through the use of sophisticated machine learning classifiers and algorithmic improvements for real-time implementation. Robots capable of robust, real-time recognition of human actions during manipulation tasks could be used to improve quality of life in the home and quality of work in industrial environments.

Project description:Based on evolutionary psychological theories, numerous eye-tracking studies have demonstrated how people visually perceive a potential mate in order to efficiently estimate the person's mate value. Companies are currently working on sexualized robots that provide numerous human-like visual cues which foster the visual resemblance to humans. To gain more elaborated knowledge on how people react to sexualized robots compared with humans, the present study empirically investigated whether heterosexual males transfer deep-rooted evolutionary psychological processes of mate perception to human-like and machine-like sexualized robots. Moreover, we aimed to learn more about the processes of orienting responses toward human and non-human stimuli and about potential predictors of visual attention to robots. Therefore, we conducted an eye-tracking study in which 15 heterosexual men, 12 homosexual men, and 18 heterosexual women were confronted with stimuli showing women, human-like gynoid robots and machine-like gynoid robots. For the sample as a whole, there was no difference in the amount of time spent looking at the human and non-human breasts. However, the results for the heterosexual males supported the assumption that human breasts attract more visual attention than do the breast areas of human-like and machine-like robots. The pelvic region yielded an unexpected gaze pattern, as all participants spent more time looking at the robotic pelvic area than at the human one, with more visual attention paid to the machine-like robots than to the human-like robots. The results of the viewing times toward the head revealed that all participants had a stronger need to gain visual information about the human head in comparison to the robotic heads, underlining the importance of authenticity in terms of emotions and motivations that can only be decoded in humans. Moreover, the study showed that individuals more frequently switched their visual attention toward different body parts of the robots in comparison to the female stimuli, implying that non-human sexualized representations evoked a higher need for visual exploration.

Dataset Information

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features.

Publications

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets