Dataset Information

Post-processing integration and semi-automated analysis of eye-tracking and motion-capture data obtained in immersive virtual reality environments to measure visuomotor integration.

ABSTRACT: Mobile eye-tracking and motion-capture techniques yield rich, precisely quantifiable data that can inform our understanding of the relationship between visual and motor processes during task performance. However, these systems are rarely used in combination, in part because of the significant time and human resources required for post-processing and analysis. Recent advances in computer vision have opened the door for more efficient processing and analysis solutions. We developed a post-processing pipeline to integrate mobile eye-tracking and full-body motion-capture data. These systems were used simultaneously to measure visuomotor integration in an immersive virtual environment. Our approach enables calculation of a 3D gaze vector that can be mapped to the participant's body position and objects in the virtual environment using a uniform coordinate system. This approach is generalizable to other configurations, and enables more efficient analysis of eye, head, and body movements together during visuomotor tasks administered in controlled, repeatable environments.

SUBMITTER: Miller HL

PROVIDER: S-EPMC8276594 | biostudies-literature | 2021 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Post-processing integration and semi-automated analysis of eye-tracking and motion-capture data obtained in immersive virtual reality environments to measure visuomotor integration.

Miller Haylie L HL Zurutuza Ian R IR Fears Nicholas E NE Polat Suleyman O SO Nielsen Rodney D RD

Proceedings. Eye Tracking Research & Applications Symposium 20210501

Mobile eye-tracking and motion-capture techniques yield rich, precisely quantifiable data that can inform our understanding of the relationship between visual and motor processes during task performance. However, these systems are rarely used in combination, in part because of the significant time and human resources required for post-processing and analysis. Recent advances in computer vision have opened the door for more efficient processing and analysis solutions. We developed a post-processi ...[more]

PMID: 34263270

Similar Datasets

Project description:Aphasia is a highly disabling acquired language disorder generally caused by a left-lateralized brain damage. Even if traditional therapies have been shown to induce an adequate clinical improvement, a large percentage of patients are left with some degree of language impairments. Therefore, new approaches to common speech therapies are urgently needed in order to maximize the recovery from aphasia. The recent application of virtual reality (VR) to aphasia rehabilitation has already evidenced its usefulness in promoting a more pragmatically oriented treatment than conventional therapies (CT). In the present study, thirty-six chronic persons with aphasia (PWA) were randomly assigned to two groups. The VR group underwent conversational therapy during VR everyday life setting observation, while the control group was trained in a conventional setting without VR support. All patients were extensively tested through a neuropsychological battery which included not only measures for language skills and communication efficacy but also self-esteem and quality of life questionnairies. All patients were trained through a conversational approach by a speech therapist twice a week for six months (total 48 sessions). After the treatment, no significant differences among groups were found in the different measures. However, the amount of improvement in the different areas was distributed over far more cognitive and psychological aspects in the VR group than in the control group. Indeed, the within-group comparisons showed a significant enhancement in different language tasks (i.e., oral comprehension, repetition, and written language) only in the VR group. Significant gains, after the treatment, were also found, in the VR group, in different psychological dimensions (i.e., self-esteem and emotional and mood state). Given the importance of these aspects for aphasia recovery, we believe that our results add to previous evidence which points to the ecological validity and feasibility of VR treatment for language recovery and psychosocial well-being.

Project description:BackgroundComplex motor tasks in immersive virtual reality using a head-mounted display (HMD-VR) have been shown to increase cognitive load and decrease motor performance compared to conventional computer screens (CS). Separately, visuomotor adaptation in HMD-VR has been shown to recruit more explicit, cognitive strategies, resulting in decreased implicit mechanisms thought to contribute to motor memory formation. However, it is unclear whether visuomotor adaptation in HMD-VR increases cognitive load and whether cognitive load is related to explicit mechanisms and long-term motor memory formation.MethodsWe randomized 36 healthy participants into three equal groups. All groups completed an established visuomotor adaptation task measuring explicit and implicit mechanisms, combined with a dual-task probe measuring cognitive load. Then, all groups returned after 24-h to measure retention of the overall adaptation. One group completed both training and retention tasks in CS (measuring long-term retention in a CS environment), one group completed both training and retention tasks in HMD-VR (measuring long-term retention in an HMD-VR environment), and one group completed the training task in HMD-VR and the retention task in CS (measuring context transfer from an HMD-VR environment). A Generalized Linear Mixed-Effect Model (GLMM) was used to compare cognitive load between CS and HMD-VR during visuomotor adaptation, t-tests were used to compare overall adaptation and explicit and implicit mechanisms between CS and HMD-VR training environments, and ANOVAs were used to compare group differences in long-term retention and context transfer.ResultsCognitive load was found to be greater in HMD-VR than in CS. This increased cognitive load was related to decreased use of explicit, cognitive mechanisms early in adaptation. Moreover, increased cognitive load was also related to decreased long-term motor memory formation. Finally, training in HMD-VR resulted in decreased long-term retention and context transfer.ConclusionsOur findings show that cognitive load increases in HMD-VR and relates to explicit learning and long-term motor memory formation during motor learning. Future studies should examine what factors cause increased cognitive load in HMD-VR motor learning and whether this impacts HMD-VR training and long-term retention in clinical populations.

Project description:A mixed design was created using text and game-like multimedia to instruct in the content of physics. The study assessed which variables predicted learning gains after a 1-h lesson on the electric field. The three manipulated variables were: (1) level of embodiment; (2) level of active generativity; and (3) presence of story narrative. Two types of tests were administered: (1) a traditional text-based physics test answered with a keyboard; and (2) a more embodied, transfer test using the Wacom large tablet where learners could use gestures (long swipes) to create vectors and answers. The 166 participants were randomly assigned to four conditions: (1) symbols and text; (2) low embodied; (3) high embodied/active; or (4) high embodied/active with narrative. The last two conditions were active because the on-screen content could be manipulated with gross body gestures gathered via the Kinect sensor. Results demonstrated that the three groups that included embodiment learned significantly more than the symbols and text group on the traditional keyboard post-test. When knowledge was assessed with the Wacom tablet format that facilitated gestures, the two active gesture-based groups scored significantly higher. In addition, engagement scores were significantly higher for the two active embodied groups. The Wacom results suggest test sensitivity issues; the more embodied test revealed greater gains in learning for the more embodied conditions. We recommend that as more embodied learning comes to the fore, more sensitive tests that incorporate gesture be used to accurately assess learning. The predicted differences in engagement and learning for the condition with the graphically rich story narrative were not supported. We hypothesize that a narrative effect for motivation and learning may be difficult to uncover in a lab experiment where participants are primarily motivated by course credit. Several design principles for mediated and embodied science education are proposed.

Dataset Information

Post-processing integration and semi-automated analysis of eye-tracking and motion-capture data obtained in immersive virtual reality environments to measure visuomotor integration.

Publications

Post-processing integration and semi-automated analysis of eye-tracking and motion-capture data obtained in immersive virtual reality environments to measure visuomotor integration.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets