Dataset Information

Electrode-free visual prosthesis/exoskeleton control using augmented reality glasses in a first proof-of-technical-concept study.

ABSTRACT: In the field of neuroprosthetics, the current state-of-the-art method involves controlling the prosthesis with electromyography (EMG) or electrooculography/electroencephalography (EOG/EEG). However, these systems are both expensive and time consuming to calibrate, susceptible to interference, and require a lengthy learning phase by the patient. Therefore, it is an open challenge to design more robust systems that are suitable for everyday use and meet the needs of patients. In this paper, we present a new concept of complete visual control for a prosthesis, an exoskeleton or another end effector using augmented reality (AR) glasses presented for the first time in a proof-of-concept study. By using AR glasses equipped with a monocular camera, a marker attached to the prosthesis is tracked. Minimal relative movements of the head with respect to the prosthesis are registered by tracking and used for control. Two possible control mechanisms including visual feedback are presented and implemented for both a motorized hand orthosis and a motorized hand prosthesis. Since the grasping process is mainly controlled by vision, the proposed approach appears to be natural and intuitive.

SUBMITTER: Hazubski S

PROVIDER: S-EPMC7530745 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Electrode-free visual prosthesis/exoskeleton control using augmented reality glasses in a first proof-of-technical-concept study.

Hazubski Simon S Hoppe Harald H Otte Andreas A

Scientific reports 20201001 1

In the field of neuroprosthetics, the current state-of-the-art method involves controlling the prosthesis with electromyography (EMG) or electrooculography/electroencephalography (EOG/EEG). However, these systems are both expensive and time consuming to calibrate, susceptible to interference, and require a lengthy learning phase by the patient. Therefore, it is an open challenge to design more robust systems that are suitable for everyday use and meet the needs of patients. In this paper, we pre ...[more]

PMID: 33004950

Similar Datasets

Project description:Augmented reality (AR) enhances the user's environment by projecting virtual objects into the real world in real-time. Brain-computer interfaces (BCIs) are systems that enable users to control external devices with their brain signals. BCIs can exploit AR technology to interact with the physical and virtual world and to explore new ways of displaying feedback. This is important for users to perceive and regulate their brain activity or shape their communication intentions while operating in the physical world. In this study, twelve healthy participants were introduced to and asked to choose between two motor-imagery tasks: mental drawing and interacting with a virtual cube. Participants first performed a functional localizer run, which was used to select a single fNIRS channel for decoding their intentions in eight subsequent choice-encoding runs. In each run participants were asked to select one choice of a six-item list. A rotating AR cube was displayed on a computer screen as the main stimulus, where each face of the cube was presented for 6 s and represented one choice of the six-item list. For five consecutive trials, participants were instructed to perform the motor-imagery task when the face of the cube that represented their choice was facing them (therewith temporally encoding the selected choice). In the end of each run, participants were provided with the decoded choice based on a joint analysis of all five trials. If the decoded choice was incorrect, an active error-correction procedure was applied by the participant. The choice list provided in each run was based on the decoded choice of the previous run. The experimental design allowed participants to navigate twice through a virtual menu that consisted of four levels if all choices were correctly decoded. Here we demonstrate for the first time that by using AR feedback and flexible choice encoding in form of search trees, we can increase the degrees of freedom of a BCI system. We also show that participants can successfully navigate through a nested menu and achieve a mean accuracy of 74% using a single motor-imagery task and a single fNIRS channel.

Project description:BackgroundHand amputation can have a truly debilitating impact on the life of the affected person. A multifunctional myoelectric prosthesis controlled using pattern classification can be used to restore some of the lost motor abilities. However, learning to control an advanced prosthesis can be a challenging task, but virtual and augmented reality (AR) provide means to create an engaging and motivating training.MethodsIn this study, we present a novel training framework that integrates virtual elements within a real scene (AR) while allowing the view from the first-person perspective. The framework was evaluated in 13 able-bodied subjects and a limb-deficient person divided into intervention (IG) and control (CG) groups. The IG received training by performing simulated clothespin task and both groups conducted a pre- and posttest with a real prosthesis. When training with the AR, the subjects received visual feedback on the generated grasping force. The main outcome measure was the number of pins that were successfully transferred within 20 min (task duration), while the number of dropped and broken pins were also registered. The participants were asked to score the difficulty of the real task (posttest), fun-factor and motivation, as well as the utility of the feedback.ResultsThe performance (median/interquartile range) consistently increased during the training sessions (4/3 to 22/4). While the results were similar for the two groups in the pretest, the performance improved in the posttest only in IG. In addition, the subjects in IG transferred significantly more pins (28/10.5 versus 14.5/11), and dropped (1/2.5 versus 3.5/2) and broke (5/3.8 versus 14.5/9) significantly fewer pins in the posttest compared to CG. The participants in IG assigned (mean ± std) significantly lower scores to the difficulty compared to CG (5.2 ± 1.9 versus 7.1 ± 0.9), and they highly rated the fun factor (8.7 ± 1.3) and usefulness of feedback (8.5 ± 1.7).ConclusionThe results demonstrated that the proposed AR system allows for the transfer of skills from the simulated to the real task while providing a positive user experience. The present study demonstrates the effectiveness and flexibility of the proposed AR framework. Importantly, the developed system is open source and available for download and further development.

Project description:BackgroundAugmented reality (AR) glasses with GPS navigation represent the rapidly evolving technology which spares (and externalizes) navigational capacities. Regarding the expected everyday usage of this device, its impact on neuroplastic brain changes and navigation abilities should be evaluated.AimsThis study aimed to assess possible changes in functional connectivity (FC) of hippocampus and other brain regions involved in spatial navigation.MethodsThirty-three healthy participants completed two resting state functional magnetic resonance imaging (rsfMRI) measurements at the baseline and after 3 months. For this period, the experimental group (n = 17) has had used AR device (Vuzix M100) with incorporated GPS guidance system during navigation in real world. Participants from the control group (n = 16) have not used any GPS device while navigating during walking. The rsfMRI FC of right and left hippocampi was analyzed using a seed-driven approach. Virtual city task was used to test navigational abilities both before and after the usage of AR device.ResultsWe identified strong functional coupling of right and left hippocampi at the baseline (p < 0.05, FDR corrected). Mild changes in bilateral hippocampal FC (p < 0.05, FDR uncorrected) were observed in both assessed groups mainly between the bilateral hippocampi and between each hippocampus and temporal regions and cerebellum. However, the experimental group showed FC decrease after three months of using GPS navigation implemented in AR glasses in contrast to FC increase in the control group without such intervention. Importantly, no effect of intervention on navigational abilities was observed.DiscussionOur observation supports the assumption that externalization of spatial navigation to technological device (GPS in AR glasses) can decrease the functional coupling between hippocampus and associated brain regions. Considering some limitations of the present study, further studies should elucidate the mechanism of the observed changes and their impact on cognitive abilities.

Dataset Information

Electrode-free visual prosthesis/exoskeleton control using augmented reality glasses in a first proof-of-technical-concept study.

Publications

Electrode-free visual prosthesis/exoskeleton control using augmented reality glasses in a first proof-of-technical-concept study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets