Dataset Information

Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus.

ABSTRACT:

Objective

To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the 'hand knob' area of precentral gyrus, a step toward the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak.

Approach

Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the 'Brain-to-Speech' pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times.

Main results

A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while an RNN classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio.

Significance

The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.

SUBMITTER: Wilson GH

PROVIDER: S-EPMC8293867 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Objective.Microelectrode arrays are standard tools for conducting chronic electrophysiological experiments, allowing researchers to simultaneously record from large numbers of neurons. Specifically, Utah electrode arrays (UEAs) have been utilized by scientists in many species, including rodents, rhesus macaques, marmosets, and human participants. The field of clinical human brain-computer interfaces currently relies on the UEA as a number of research groups have clearance from the United States Federal Drug Administration (FDA) for this device through the investigational device exemption pathway. Despite its widespread usage in systems neuroscience, few studies have comprehensively evaluated the reliability and signal quality of the Utah array over long periods of time in a large dataset.Approach.We collected and analyzed over 6000 recorded datasets from various cortical areas spanning almost nine years of experiments, totaling 17 rhesus macaques (Macaca mulatta) and 2 human subjects, and 55 separate microelectrode Utah arrays. The scale of this dataset allowed us to evaluate the average life of these arrays, based primarily on the signal-to-noise ratio of each electrode over time.Main results.Using implants in primary motor, premotor, prefrontal, and somatosensory cortices, we found that the average lifespan of available recordings from UEAs was 622 days, although we provide several examples of these UEAs lasting over 1000 days and one up to 9 years; human implants were also shown to last longer than non-human primate implants. We also found that electrode length did not affect longevity and quality, but iridium oxide metallization on the electrode tip exhibited superior yield as compared to platinum metallization.Significance.Understanding longevity and reliability of microelectrode array recordings allows researchers to set expectations and plan experiments accordingly and maximize the amount of high-quality data gathered. Our results suggest that one can expect chronic unit recordings to last at least two years, with the possibility for arrays to last the better part of a decade.

Project description:Our goal was to identify spatial filtering methods that would improve decoding of continuous arm movements from epidural field potentials as well as demonstrate the use of the epidural signals in a closed-loop brain-machine interface (BMI) system in monkeys.Eleven spatial filtering options were compared offline using field potentials collected from 64-channel high-density epidural arrays in monkeys. Arrays were placed over arm/hand motor cortex in which intracortical microelectrodes had previously been implanted and removed leaving focal cortical damage but no lasting motor deficits. Spatial filters tested included: no filtering, common average referencing (CAR), principle component analysis, and eight novel modifications of the common spatial pattern (CSP) algorithm. The spatial filtering method and decoder combination that performed the best offline was then used online where monkeys controlled cursor velocity using continuous wrist position decoded from epidural field potentials in real time.Optimized CSP methods improved continuous wrist position decoding accuracy by 69% over CAR and by 80% compared to no filtering. Kalman decoders performed better than linear regression decoders and benefitted from including more spatially-filtered signals but not from pre-smoothing the calculated power spectra. Conversely, linear regression decoders required fewer spatially-filtered signals and were improved by pre-smoothing the power values. The 'position-to-velocity' transformation used during online control enabled the animals to generate smooth closed-loop movement trajectories using the somewhat limited position information available in the epidural signals. The monkeys' online performance significantly improved across days of closed-loop training.Most published BMI studies that use electrocorticographic signals to decode continuous limb movements either use no spatial filtering or CAR. This study suggests a substantial improvement in decoding accuracy could be attained by using our new version of the CSP algorithm that extends the traditional CSP method for use with continuous limb movement data.

Project description:We present an autopsied case with A8344G-mutated myoclonus epilepsy with ragged red fibers (MERRF)/mitochondrial encephalomyopathy with lactic acidosis and stroke-like episodes (MELAS) overlap syndrome accompanied by stroke-like episodes localized to the precentral gyrus. A 16-year-old Japanese woman suddenly experienced repetitive consciousness disturbances with increased serum lactate and creatine kinase levels. Magnetic resonance imaging showed abnormal intensity of bilateral precentral gyrus. She was clinically diagnosed as having a mitochondrial disorder and the A8344G mutation was detected in mitochondrial DNA. At 17 years of age, she died from congestive heart failure secondary to a third episode of lactic acidosis. Neuropatho-logically, multifocal laminar necrosis, which is responsible for stroke-like episodes in MELAS, was seen in the frontal cortex including the precentral gyrus, but there was no neuronal loss and gliosis in the basal ganglia, cerebellum, and brainstem, which were compatible with MERRF. Hypertrophy of the vascular smooth muscle and choroidal epithelium were seen, and were strongly visualized by an anti-mitochondrial antibody. Skeletal muscles showed uneven muscular diameters, increased central nuclei, and ragged red fibers (RRFs). Decreased cytochrome c oxidase (COX) activity and strongly succinate dehydrogenase (SDH)-reactive blood vessels were also noted. Stroke-like episodes in MERRF/MELAS overlap syndrome are thought to be rare in the frontal cortex including the precentral gyrus. Only two cases of MERRF/MELAS overlap syndrome with A8344G mutation, including this case, have shown stroke-like episodes in the frontal lobes. Other than the A8344G mutation and frontal lobe involvement, they had a high degree of similarity in terms of presence of RRFs, gastrointestinal dysfunction, and lack of typical MERRF neuropathology. In conclusion, this is an important case describing the clinical spectrum associated with A8344G-mutated MERRF/MELAS overlap syndrome.