Structured neuronal encoding and decoding of human speech features.
Ontology highlight
ABSTRACT:
SUBMITTER: Tankus A
PROVIDER: S-EPMC4445934 | biostudies-literature | 2012
REPOSITORIES: biostudies-literature
Publications
Structured neuronal encoding and decoding of human speech features.
Tankus Ariel A Fried Itzhak I Shoham Shy S
Nature communications 20120101
Human speech sounds are produced through a coordinated movement of structures along the vocal tract. Here we show highly structured neuronal encoding of vowel articulation. In medial-frontal neurons, we observe highly specific tuning to individual vowels, whereas superior temporal gyrus neurons have nonspecific, sinusoidally modulated tuning (analogous to motor cortical directional tuning). At the neuronal population level, a decoding analysis reveals that the underlying structure of vowel encod ...[more]
Similar Datasets
Project description:Cell signaling dynamics and transcriptional regulatory activities are variable within specific cell types responding to an identical stimulus. In addition to studying the network interactions, there is much interest in utilizing single cell scale data to elucidate the non-random aspects of the variability involved in cellular decision making. Previous studies have considered the information transfer between the signaling and transcriptional domains based on an instantaneous relationship between the molecular activities. These studies predict a limited binary on/off encoding mechanism which underestimates the complexity of biological information processing, and hence the utility of single cell resolution data. Here we pursue a novel strategy that reformulates the information transfer problem as involving dynamic features of signaling rather than molecular abundances. We pursue a computational approach to test if and how the transcriptional regulatory activity patterns can be informative of the temporal history of signaling. Our analysis reveals (1) the dynamic features of signaling that significantly alter transcriptional regulatory patterns (encoding), and (2) the temporal history of signaling that can be inferred from single cell scale snapshots of transcriptional activity (decoding). Immediate early gene expression patterns were informative of signaling peak retention kinetics, whereas transcription factor activity patterns were informative of activation and deactivation kinetics of signaling. Moreover, the information processing aspects varied across the network, with each component encoding a selective subset of the dynamic signaling features. We developed novel sensitivity and information transfer maps to unravel the dynamic multiplexing of signaling features at each of these network components. Unsupervised clustering of the maps revealed two groups that aligned with network motifs distinguished by transcriptional feedforward vs feedback interactions. Our new computational methodology impacts the single cell scale experiments by identifying downstream snapshot measures required for inferring specific dynamical features of upstream signals involved in the regulation of cellular responses.
| S-EPMC4619640 | biostudies-literature
Project description:BackgroundThe hippocampus underpins our ability to navigate, to form and recollect memories, and to imagine future experiences. How activity across millions of hippocampal neurons supports these functions is a fundamental question in neuroscience, wherein the size, sparseness, and organization of the hippocampal neural code are debated.ResultsHere, by using multivariate pattern classification and high spatial resolution functional MRI, we decoded activity across the population of neurons in the human medial temporal lobe while participants navigated in a virtual reality environment. Remarkably, we could accurately predict the position of an individual within this environment solely from the pattern of activity in his hippocampus even when visual input and task were held constant. Moreover, we observed a dissociation between responses in the hippocampus and parahippocampal gyrus, suggesting that they play differing roles in navigation.ConclusionsThese results show that highly abstracted representations of space are expressed in the human hippocampus. Furthermore, our findings have implications for understanding the hippocampal population code and suggest that, contrary to current consensus, neuronal ensembles representing place memories must be large and have an anisotropic structure.
| S-EPMC2670980 | biostudies-literature
Project description:Speakers of all human languages regularly use intonational pitch to convey linguistic meaning, such as to emphasize a particular word. Listeners extract pitch movements from speech and evaluate the shape of intonation contours independent of each speaker's pitch range. We used high-density electrocorticography to record neural population activity directly from the brain surface while participants listened to sentences that varied in intonational pitch contour, phonetic content, and speaker. Cortical activity at single electrodes over the human superior temporal gyrus selectively represented intonation contours. These electrodes were intermixed with, yet functionally distinct from, sites that encoded different information about phonetic features or speaker identity. Furthermore, the representation of intonation contours directly reflected the encoding of speaker-normalized relative pitch but not absolute pitch.
| S-EPMC9584035 | biostudies-literature
Project description:When speaking, we dynamically coordinate movements of our jaw, tongue, lips, and larynx. To investigate the neural mechanisms underlying articulation, we used direct cortical recordings from human sensorimotor cortex while participants spoke natural sentences that included sounds spanning the entire English phonetic inventory. We used deep neural networks to infer speakers' articulator movements from produced speech acoustics. Individual electrodes encoded a diversity of articulatory kinematic trajectories (AKTs), each revealing coordinated articulator movements toward specific vocal tract shapes. AKTs captured a wide range of movement types, yet they could be differentiated by the place of vocal tract constriction. Additionally, AKTs manifested out-and-back trajectories with harmonic oscillator dynamics. While AKTs were functionally stereotyped across different sentences, context-dependent encoding of preceding and following movements during production of the same phoneme demonstrated the cortical representation of coarticulation. Articulatory movements encoded in sensorimotor cortex give rise to the complex kinematics underlying continuous speech production. VIDEO ABSTRACT.
| S-EPMC5992088 | biostudies-literature
Project description:Speech perception is thought to rely on a cortical feedforward serial transformation of acoustic into linguistic representations. Using intracranial recordings across the entire human auditory cortex, electrocortical stimulation, and surgical ablation, we show that cortical processing across areas is not consistent with a serial hierarchical organization. Instead, response latency and receptive field analyses demonstrate parallel and distinct information processing in the primary and nonprimary auditory cortices. This functional dissociation was also observed where stimulation of the primary auditory cortex evokes auditory hallucination but does not distort or interfere with speech perception. Opposite effects were observed during stimulation of nonprimary cortex in superior temporal gyrus. Ablation of the primary auditory cortex does not affect speech perception. These results establish a distributed functional organization of parallel information processing throughout the human auditory cortex and demonstrate an essential independent role for nonprimary auditory cortex in speech processing.
| S-EPMC8456481 | biostudies-literature
Project description:The superior temporal gyrus (STG) and neighboring brain regions play a key role in human language processing. Previous studies have attempted to reconstruct speech information from brain activity in the STG, but few of them incorporate the probabilistic framework and engineering methodology used in modern speech recognition systems. In this work, we describe the initial efforts toward the design of a neural speech recognition (NSR) system that performs continuous phoneme recognition on English stimuli with arbitrary vocabulary sizes using the high gamma band power of local field potentials in the STG and neighboring cortical areas obtained via electrocorticography.The system implements a Viterbi decoder that incorporates phoneme likelihood estimates from a linear discriminant analysis model and transition probabilities from an n-gram phonemic language model. Grid searches were used in an attempt to determine optimal parameterizations of the feature vectors and Viterbi decoder.The performance of the system was significantly improved by using spatiotemporal representations of the neural activity (as opposed to purely spatial representations) and by including language modeling and Viterbi decoding in the NSR system.These results emphasize the importance of modeling the temporal dynamics of neural responses when analyzing their variations with respect to varying stimuli and demonstrate that speech recognition techniques can be successfully leveraged when decoding speech from neural signals. Guided by the results detailed in this work, further development of the NSR system could have applications in the fields of automatic speech recognition and neural prosthetics.
| S-EPMC5031534 | biostudies-literature
Project description:Restoration of speech communication for locked-in patients by means of brain computer interfaces (BCIs) is currently an important area of active research. Among the neural signals obtained from intracranial recordings, single/multi-unit activity (SUA/MUA), local field potential (LFP), and electrocorticography (ECoG) are good candidates for an input signal for BCIs. However, the question of which signal or which combination of the three signal modalities is best suited for decoding speech production remains unverified. In order to record SUA, LFP, and ECoG simultaneously from a highly localized area of human ventral sensorimotor cortex (vSMC), we fabricated an electrode the size of which was 7 by 13 mm containing sparsely arranged microneedle and conventional macro contacts. We determined which signal modality is the most capable of decoding speech production, and tested if the combination of these signals could improve the decoding accuracy of spoken phonemes. Feature vectors were constructed from spike frequency obtained from SUAs and event-related spectral perturbation derived from ECoG and LFP signals, then input to the decoder. The results showed that the decoding accuracy for five spoken vowels was highest when features from multiple signals were combined and optimized for each subject, and reached 59% when averaged across all six subjects. This result suggests that multi-scale signals convey complementary information for speech articulation. The current study demonstrated that simultaneous recording of multi-scale neuronal activities could raise decoding accuracy even though the recording area is limited to a small portion of cortex, which is advantageous for future implementation of speech-assisting BCIs.
| S-EPMC5895763 | biostudies-literature
Project description:Natural communication often occurs in dialogue, differentially engaging auditory and sensorimotor brain regions during listening and speaking. However, previous attempts to decode speech directly from the human brain typically consider listening or speaking tasks in isolation. Here, human participants listened to questions and responded aloud with answers while we used high-density electrocorticography (ECoG) recordings to detect when they heard or said an utterance and to then decode the utterance's identity. Because certain answers were only plausible responses to certain questions, we could dynamically update the prior probabilities of each answer using the decoded question likelihoods as context. We decode produced and perceived utterances with accuracy rates as high as 61% and 76%, respectively (chance is 7% and 20%). Contextual integration of decoded question likelihoods significantly improves answer decoding. These results demonstrate real-time decoding of speech in an interactive, conversational setting, which has important implications for patients who are unable to communicate.
| S-EPMC6667454 | biostudies-literature
Project description:Nitric oxide, produced by the neuronal nitric oxide synthase (nNOS) from L-arginine is an important second messenger molecule in the central nervous system: It influences the synthesis and release of neurotransmitters and plays an important role in long-term potentiation, long-term depression and neuroendocrine secretion. However, under certain pathological conditions such as Alzheimer's or Parkinson's disease, stroke and multiple sclerosis, excessive NO production can lead to tissue damage. It is thus desirable to control NO production in these situations. So far, little is known about the substrate supply to human nNOS as a determinant of its activity. Measuring bioactive NO via cGMP formation in reporter cells, we demonstrate here that nNOS in both, human A673 neuroepithelioma and TGW-nu-I neuroblastoma cells can be fast and efficiently nourished by extracellular arginine that enters the cells via membrane transporters (pool I that is freely exchangeable with the extracellular space). When this pool was depleted, NO synthesis was partially sustained by intracellular arginine sources not freely exchangeable with the extracellular space (pool II). Protein breakdown made up by far the largest part of pool II in both cell types. In contrast, citrulline to arginine conversion maintained NO synthesis only in TGW-nu-I neuroblastoma, but not A673 neuroepithelioma cells. Histidine mimicked the effect of protease inhibitors causing an almost complete nNOS inhibition in cells incubated additionally in lysine that depletes the exchangeable arginine pool. Our results identify new ways to modulate nNOS activity by modifying its substrate supply.
| S-EPMC3706577 | biostudies-literature
Project description:Extracting meaning from a dynamic and variable flow of incoming information is a major goal of both natural and artificial intelligence. Computer vision (CV) guided by deep learning (DL) has made significant strides in recognizing a specific identity despite highly variable attributes. This is the same challenge faced by the nervous system and partially addressed by the concept cells-neurons exhibiting selective firing in response to specific persons/places, described in the human medial temporal lobe (MTL) . Yet, access to neurons representing a particular concept is limited due to these neurons' sparse coding. It is conceivable, however, that the information required for such decoding is present in relatively small neuronal populations. To evaluate how well neuronal populations encode identity information in natural settings, we recorded neuronal activity from multiple brain regions of nine neurosurgical epilepsy patients implanted with depth electrodes, while the subjects watched an episode of the TV series "24". First, we devised a minimally supervised CV algorithm (with comparable performance against manually-labeled data) to detect the most prevalent characters (above 1% overall appearance) in each frame. Next, we implemented DL models that used the time-varying population neural data as inputs and decoded the visual presence of the four main characters throughout the episode. This methodology allowed us to compare "computer vision" with "neuronal vision"-footprints associated with each character present in the activity of a subset of neurons-and identify the brain regions that contributed to this decoding process. We then tested the DL models during a recognition memory task following movie viewing where subjects were asked to recognize clip segments from the presented episode. DL model activations were not only modulated by the presence of the corresponding characters but also by participants' subjective memory of whether they had seen the clip segment, and by the associative strengths of the characters in the narrative plot. The described approach can offer novel ways to probe the representation of concepts in time-evolving dynamic behavioral tasks. Further, the results suggest that the information required to robustly decode concepts is present in the population activity of only tens of neurons even in brain regions beyond MTL.
| S-EPMC9837190 | biostudies-literature