Dataset Information

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic.

ABSTRACT: To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocorticographic (ECoG) and depth electrodes within motor areas. The subjects performed the complex task in 71.4% (20/28) and 67.7% (21/31) of trials after minimal training. Balanced accuracy for detecting movements was 91.1% and 92.9%, significantly greater than chance accuracies (p < 0.05). After BMI-based initiation, the MPL completed the entire task 100% (one object) and 70% (three objects) of the time. The MPL took approximately 12.2 s for task completion after system improvements implemented for the second subject. Our hybrid-BMI design prevented all but one baseline false positive from initiating the system. The novel approach demonstrated in this proof-of-principle study, using hybrid input, supervisory control, and intelligent robotics, addresses limitations of current BMIs.

SUBMITTER: McMullen DP

PROVIDER: S-EPMC4057363 | biostudies-literature | 2014 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic.

McMullen David P DP Hotson Guy G Katyal Kapil D KD Wester Brock A BA Fifer Matthew S MS McGee Timothy G TG Harris Andrew A Johannes Matthew S MS Vogelstein R Jacob RJ Ravitz Alan D AD Anderson William S WS Thakor Nitish V NV Crone Nathan E NE

IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society 20131212 4

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the obje ...[more]

PMID: 24760914

Similar Datasets

Project description:BackgroundTo introduce a novel method of patient positioning for high precision intracranial radiotherapy.MethodsAn infrared(IR)-array, reproducibly attached to the patient via a vacuum-mouthpiece(vMP) and connected to the table via a 6 degree-of-freedom(DoF) mechanical arm serves as positioning and fixation system. After IR-based manual prepositioning to rough treatment position and fixation of the mechanical arm, a cone-beam CT(CBCT) is performed. A robotic 6 DoF treatment couch (HexaPOD) then automatically corrects all remaining translations and rotations. This absolute position of infrared markers at the first fraction acts as reference for the following fractions where patients are manually prepositioned to within +/- 2 mm and +/- 2 degrees of this IR reference position prior to final HexaPOD-based correction; consequently CBCT imaging is only required once at the first treatment fraction.The preclinical feasibility and attainable repositioning accuracy of this method was evaluated on a phantom and human volunteers as was the clinical efficacy on 7 pilot study patients.ResultsPhantom and volunteer manual IR-based prepositioning to within +/- 2 mm and +/- 2 degrees in 6 DoF was possible within a mean(+/- SD) of 90 +/- 31 and 56 +/- 22 seconds respectively. Mean phantom translational and rotational precision after 6 DoF corrections by the HexaPOD was 0.2 +/- 0.2 mm and 0.7 +/- 0.8 degrees respectively. For the actual patient collective, the mean 3D vector for inter-treatment repositioning accuracy (n = 102) was 1.6 +/- 0.8 mm while intra-fraction movement (n = 110) was 0.6 +/- 0.4 mm.ConclusionsThis novel semi-automatic 6DoF IR-based system has been shown to compare favourably with existing non-invasive intracranial repeat fixation systems with respect to handling, reproducibility and, more importantly, intra-fraction rigidity. Some advantages are full cranial positioning flexibility for single and fractionated IGRT treatments and possibly increased patient comfort.

Dataset Information

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic.

Publications

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets