Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans.
Ontology highlight
ABSTRACT:
SUBMITTER: Pei X
PROVIDER: S-EPMC3772685 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Brain signals can provide the basis for a non-muscular communication and control system, a brain-computer interface (BCI), for people with motor disabilities. A common approach to creating BCI devices is to decode kinematic parameters of movements using signals recorded by intracortical microelectrodes. Recent studies have shown that kinematic parameters of hand movements can also be accurately decoded from signals recorded by electrodes placed on the surface of the brain (electrocorticography (ECoG)). In the present study, we extend these results by demonstrating that it is also possible to decode the time course of the flexion of individual fingers using ECoG signals in humans, and by showing that these flexion time courses are highly specific to the moving finger. These results provide additional support for the hypothesis that ECoG could be the basis for powerful clinically practical BCI systems, and also indicate that ECoG is useful for studying cortical dynamics related to motor function.
| S-EPMC3664231 | biostudies-literature
Project description:Electrocorticography (ECoG) signals have emerged as a potential control signal for brain-computer interface (BCI) applications due to balancing signal quality and implant invasiveness. While there have been numerous demonstrations in which ECoG signals were used to decode motor movements and to develop BCI systems, the extent of information that can be decoded has been uncertain. Therefore, we sought to determine if ECoG signals could be used to decode kinematics (speed, velocity, and position) of arm movements in 3D space.To investigate this, we designed a 3D center-out reaching task that was performed by five epileptic patients undergoing temporary placement of ECoG arrays. We used the ECoG signals within a hierarchical partial-least squares (PLS) regression model to perform offline prediction of hand speed, velocity, and position.The hierarchical PLS regression model enabled us to predict hand speed, velocity, and position during 3D reaching movements from held-out test sets with accuracies above chance in each patient with mean correlation coefficients between 0.31 and 0.80 for speed, 0.27 and 0.54 for velocity, and 0.22 and 0.57 for position. While beta band power changes were the most significant features within the model used to classify movement and rest, the local motor potential and high gamma band power changes, were the most important features in the prediction of kinematic parameters.We believe that this study represents the first demonstration that truly three-dimensional movements can be predicted from ECoG recordings in human patients. Furthermore, this prediction underscores the potential to develop BCI systems with multiple degrees of freedom in human patients using ECoG.
| S-EPMC5535759 | biostudies-literature
Project description:While words are distinguished primarily by consonants and vowels in many languages, tones are also used in the majority of the world's languages to cue lexical contrasts. However, studies on novel word learning have largely concentrated on consonants and vowels. To shed more light on the use of tonal information in novel word learning and its relationship with the development of phonological categories, the present study explored how adults' ability to learn minimal pair pseudowords in a tone language is modulated by their native phonological knowledge. Twenty-four adult speakers of three languages were tested: Cantonese, Mandarin, and French. Eye-tracking was used to record eye movements of these learners, while they were watching animated cartoons in Cantonese. On each trial, adults had to learn two new label-object associations, while the labels differed minimally by a consonant, a vowel, or a tone. Learning would therefore attest to participants' ability to use phonological information to distinguish the paired words. Results first revealed that adult learners in each language group performed better than chance in all conditions. Moreover, compared to native Cantonese adults, both Mandarin- and French-speaking adults performed worse on all three contrasts. In addition, French adults were worse on tones when compared to Mandarin adults. Lastly, no advantage for consonantal information in native lexical processing was found for Cantonese-speaking adults as predicted by the "division of labor" proposal, thus confirming crosslinguistic differences in consonant/vowel weight between speakers of tonal vs. non-tonal languages. These findings establish rapid novel word learning in a non-native language (long-term learning will have to be further assessed), modulated by native phonological knowledge. The implications of the findings of this adult study for further infant word learning studies are discussed.
| S-EPMC6066720 | biostudies-literature
Project description:Pathological conditions such as amyotrophic lateral sclerosis or damage to the brainstem can leave patients severely paralyzed but fully aware, in a condition known as 'locked-in syndrome'. Communication in this state is often reduced to selecting individual letters or words by arduous residual movements. More intuitive and rapid communication may be restored by directly interfacing with language areas of the cerebral cortex. We used a grid of closely spaced, nonpenetrating micro-electrodes to record local field potentials (LFPs) from the surface of face motor cortex and Wernicke's area. From these LFPs we were successful in classifying a small set of words on a trial-by-trial basis at levels well above chance. We found that the pattern of electrodes with the highest accuracy changed for each word, which supports the idea that closely spaced micro-electrodes are capable of capturing neural signals from independent neural processing assemblies. These results further support using cortical surface potentials (electrocorticography) in brain-computer interfaces. These results also show that LFPs recorded from the cortical surface (micro-electrocorticography) of language areas can be used to classify speech-related cortical rhythms and potentially restore communication to locked-in patients.
| S-EPMC2970568 | biostudies-literature
Project description:Speech sounds are traditionally divided into consonants and vowels. When only vowels or only consonants are replaced by noise, listeners are more accurate understanding sentences in which consonants are replaced but vowels remain. From such data, vowels have been suggested to be more important for understanding sentences; however, such conclusions are mitigated by the fact that replaced consonant segments were roughly one-third shorter than vowels. We report two experiments that demonstrate listener performance to be better predicted by simple psychoacoustic measures of cochlea-scaled spectral change across time. First, listeners identified sentences in which portions of consonants (C), vowels (V), CV transitions, or VC transitions were replaced by noise. Relative intelligibility was not well accounted for on the basis of Cs, Vs, or their transitions. In a second experiment, distinctions between Cs and Vs were abandoned. Instead, portions of sentences were replaced on the basis of cochlea-scaled spectral entropy (CSE). Sentence segments having relatively high, medium, or low entropy were replaced with noise. Intelligibility decreased linearly as the amount of replaced CSE increased. Duration of signal replaced and proportion of consonants/vowels replaced fail to account for listener data. CSE corresponds closely with the linguistic construct of sonority (or vowel-likeness) that is useful for describing phonological systematicity, especially syllable composition. Results challenge traditional distinctions between consonants and vowels. Speech intelligibility is better predicted by nonlinguistic sensory measures of uncertainty (potential information) than by orthodox physical acoustic measures or linguistic constructs.
| S-EPMC2901476 | biostudies-literature
Project description:Functional mapping of eloquent cortex is often necessary prior to invasive brain surgery, but current techniques that derive this mapping have important limitations. In this article, we demonstrate the first comprehensive evaluation of a rapid, robust, and practical mapping system that uses passive recordings of electrocorticographic signals. This mapping procedure is based on the BCI2000 and SIGFRIED technologies that we have been developing over the past several years. In our study, we evaluated 10 patients with epilepsy from four different institutions and compared the results of our procedure with the results derived using electrical cortical stimulation (ECS) mapping. The results show that our procedure derives a functional motor cortical map in only a few minutes. They also show a substantial concurrence with the results derived using ECS mapping. Specifically, compared with ECS maps, a next-neighbor evaluation showed no false negatives, and only 0.46 and 1.10% false positives for hand and tongue maps, respectively. In summary, we demonstrate the first comprehensive evaluation of a practical and robust mapping procedure that could become a new tool for planning of invasive brain surgeries.
| S-EPMC2754703 | biostudies-literature
Project description:The synchronized activity of neuronal populations across multiple distant brain areas may reflect coordinated interactions of large-scale brain networks. Currently, there is no established method to investigate the temporal transitions between these large-scale networks that would allow, for example, to decode finger movements. Here we applied a matrix factorization method employing principal component and temporal independent component analyses to identify brain activity synchronizations. In accordance with previous studies investigating "muscle synergies", we refer to this activity as "brain activity synergy". Using electroencephalography (EEG), we first estimated cortical current sources (CSs) and then identified brain activity synergies within the estimated CS signals. A decoding analysis for finger movement in eight directions showed that such CS synergies provided more information for dissociating between movements than EEG sensor signals, EEG synergy, or CS signals, suggesting that temporal activation patterns of the synchronizing CSs may contain information related to motor control. A quantitative analysis of features selected by the decoders further revealed temporal transitions among the primary motor area, dorsal and ventral premotor areas, pre-supplementary motor area, and supplementary motor area, which may reflect transitions in motor planning and execution. These results provide a proof of concept for brain activity synergy estimation using CSs.
| S-EPMC5595824 | biostudies-other
Project description:People affected by severe neuro-degenerative diseases (e.g., late-stage amyotrophic lateral sclerosis (ALS) or locked-in syndrome) eventually lose all muscular control. Thus, they cannot use traditional assistive communication devices that depend on muscle control, or brain-computer interfaces (BCIs) that depend on the ability to control gaze. While auditory and tactile BCIs can provide communication to such individuals, their use typically entails an artificial mapping between the stimulus and the communication intent. This makes these BCIs difficult to learn and use. In this study, we investigated the use of selective auditory attention to natural speech as an avenue for BCI communication. In this approach, the user communicates by directing his/her attention to one of two simultaneously presented speakers. We used electrocorticographic (ECoG) signals in the gamma band (70-170 Hz) to infer the identity of attended speaker, thereby removing the need to learn such an artificial mapping. Our results from twelve human subjects show that a single cortical location over superior temporal gyrus or pre-motor cortex is typically sufficient to identify the attended speaker within 10 s and with 77% accuracy (50% accuracy due to chance). These results lay the groundwork for future studies that may determine the real-time performance of BCIs based on selective auditory attention to speech.
| S-EPMC4776341 | biostudies-literature
Project description:Object recognition is a key function in both human and machine vision. While brain decoding of seen and imagined objects has been achieved, the prediction is limited to training examples. We present a decoding approach for arbitrary objects using the machine vision principle that an object category is represented by a set of features rendered invariant through hierarchical processing. We show that visual features, including those derived from a deep convolutional neural network, can be predicted from fMRI patterns, and that greater accuracy is achieved for low-/high-level features with lower-/higher-level visual areas, respectively. Predicted features are used to identify seen/imagined object categories (extending beyond decoder training) from a set of computed features for numerous object images. Furthermore, decoding of imagined objects reveals progressive recruitment of higher-to-lower visual representations. Our results demonstrate a homology between human and machine vision and its utility for brain-based information retrieval.
| S-EPMC5458127 | biostudies-literature
Project description:ObjectiveTo validate the use of passive functional mapping using electrocorticographic (ECoG) broadband gamma signals for identifying receptive language cortex.MethodsWe mapped language function in 23 patients using ECoG and using electrical cortical stimulation (ECS) in a subset of 15 subjects.ResultsThe qualitative comparison between cortical sites identified by ECoG and ECS show a high concordance. A quantitative comparison indicates a high level of sensitivity (95%) and a lower level of specificity (59%). Detailed analysis reveals that 82% of all cortical sites identified by ECoG were within one contact of a site identified by ECS.ConclusionsThese results show that passive functional mapping reliably localizes receptive language areas, and that there is a substantial concordance between the ECoG- and ECS-based methods. They also point to a more refined understanding of the differences between ECoG- and ECS-based mappings. This refined understanding helps to clarify the instances in which the two methods disagree and can explain why neurosurgical practice has established the concept of a "safety margin."SignificancePassive functional mapping using ECoG signals provides a fast, robust, and reliable method for identifying receptive language areas without many of the risks and limitations associated with ECS.
| S-EPMC6414063 | biostudies-literature