Natural speech reveals the semantic maps that tile human cerebral cortex.
Ontology highlight
ABSTRACT:
SUBMITTER: Huth AG
PROVIDER: S-EPMC4852309 | biostudies-literature | 2016 Apr
REPOSITORIES: biostudies-literature
Publications
Natural speech reveals the semantic maps that tile human cerebral cortex.
Huth Alexander G AG de Heer Wendy A WA Griffiths Thomas L TL Theunissen Frédéric E FE Gallant Jack L JL
Nature 20160401 7600
The meaning of language is represented in regions of the cerebral cortex collectively known as the 'semantic system'. However, little of the semantic system has been mapped comprehensively, and the semantic selectivity of most regions is unknown. Here we systematically map semantic selectivity across the cortex using voxel-wise modelling of functional MRI (fMRI) data collected while subjects listened to hours of narrative stories. We show that the semantic system is organized into intricate patt ...[more]
Similar Datasets
Project description:Speech processing requires extracting meaning from acoustic patterns using a set of intermediate representations based on a dynamic segmentation of the speech stream. Using whole brain mapping obtained in fMRI, we investigate the locus of cortical phonemic processing not only for single phonemes but also for short combinations made of diphones and triphones. We find that phonemic processing areas are much larger than previously described: they include not only the classical areas in the dorsal superior temporal gyrus but also a larger region in the lateral temporal cortex where diphone features are best represented. These identified phonemic regions overlap with the lexical retrieval region, but we show that short word retrieval is not sufficient to explain the observed responses to diphones. Behavioral studies have shown that phonemic processing and lexical retrieval are intertwined. Here, we also have identified candidate regions within the speech cortical network where this joint processing occurs.
| S-EPMC10354060 | biostudies-literature
Project description:Speech and language are considered uniquely human abilities: animals have communication systems, but they do not match human linguistic skills in terms of recursive structure and combinatorial power. Yet, in evolution, spoken language must have emerged from neural mechanisms at least partially available in animals. In this paper, we will demonstrate how our understanding of speech perception, one important facet of language, has profited from findings and theory in nonhuman primate studies. Chief among these are physiological and anatomical studies showing that primate auditory cortex, across species, shows patterns of hierarchical structure, topographic mapping and streams of functional processing. We will identify roles for different cortical areas in the perceptual processing of speech and review functional imaging work in humans that bears on our understanding of how the brain decodes and monitors speech. A new model connects structures in the temporal, frontal and parietal lobes linking speech perception and production.
| S-EPMC2846110 | biostudies-literature
Project description:Speech perception involves the integration of sensory input with expectations based on the context of that speech. Much debate surrounds the issue of whether or not prior knowledge feeds back to affect early auditory encoding in the lower levels of the speech processing hierarchy, or whether perception can be best explained as a purely feedforward process. Although there has been compelling evidence on both sides of this debate, experiments involving naturalistic speech stimuli to address these questions have been lacking. Here, we use a recently introduced method for quantifying the semantic context of speech and relate it to a commonly used method for indexing low-level auditory encoding of speech. The relationship between these measures is taken to be an indication of how semantic context leading up to a word influences how its low-level acoustic and phonetic features are processed. We record EEG from human participants (both male and female) listening to continuous natural speech and find that the early cortical tracking of a word's speech envelope is enhanced by its semantic similarity to its sentential context. Using a forward modeling approach, we find that prediction accuracy of the EEG signal also shows the same effect. Furthermore, this effect shows distinct temporal patterns of correlation depending on the type of speech input representation (acoustic or phonological) used for the model, implicating a top-down propagation of information through the processing hierarchy. These results suggest a mechanism that links top-down prior information with the early cortical entrainment of words in natural, continuous speech.SIGNIFICANCE STATEMENT During natural speech comprehension, we use semantic context when processing information about new incoming words. However, precisely how the neural processing of bottom-up sensory information is affected by top-down context-based predictions remains controversial. We address this discussion using a novel approach that indexes a word's similarity to context and how well a word's acoustic and phonetic features are processed by the brain at the time of its utterance. We relate these two measures and show that lower-level auditory tracking of speech improves for words that are more related to their preceding context. These results suggest a mechanism that links top-down prior information with bottom-up sensory processing in the context of natural, narrative speech listening.
| S-EPMC6750931 | biostudies-literature
Project description:When we memorize multiple words simultaneously, semantic relatedness among those words assists memory. For example, the information about "apple", "banana," and "orange" will be connected via a common concept of "fruits" and become easy to retain and recall. Neural mechanisms underlying this semantic integration in verbal working memory remain unclear. Here I used electroencephalography (EEG) and investigated neural signals when healthy human participants memorized five nouns semantically related (Sem trial) or not (NonSem trial). The regularity of oscillatory signals (8-30 Hz) during the retention period was found to be lower in NonSem than Sem trials, indicating that memorizing words unrelated to each other induced a non-harmonic (irregular) waveform in the temporal cortex. These results suggest that (i) semantic features of a word are retained as a set of neural oscillations at specific frequencies and (ii) memorizing words sharing a common semantic feature produces harmonic brain responses through a resonance or integration (sharing) of the oscillatory signals.
| S-EPMC10866900 | biostudies-literature
Project description:Transmitter receptors constitute a key component of the molecular machinery for intercellular communication in the brain. Recent efforts have mapped the density of diverse transmitter receptors across the human cerebral cortex with an unprecedented level of detail. Here, we distill these observations into key organizational principles. We demonstrate that receptor densities form a natural axis in the human cerebral cortex, reflecting decreases in differentiation at the level of laminar organization and a sensory-to-association axis at the functional level. Along this natural axis, key organizational principles are discerned: progressive molecular diversity (increase of the diversity of receptor density); excitation/inhibition (increase of the ratio of excitatory-to-inhibitory receptor density); and mirrored, orderly changes of the density of ionotropic and metabotropic receptors. The uncovered natural axis formed by the distribution of receptors aligns with the axis that is formed by other dimensions of cortical organization, such as the myelo- and cytoarchitectonic levels. Therefore, the uncovered natural axis constitutes a unifying organizational feature linking multiple dimensions of the cerebral cortex, thus bringing order to the heterogeneity of cortical organization.
| S-EPMC7826352 | biostudies-literature
Project description:How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex.
| S-EPMC3269422 | biostudies-literature
Project description:C. and O. Vogt had set up a research program with the aim of establishing a detailed cartography of the medullary fiber distribution of the human brain. As part of this program, around 200 cortical fields were differentiated based on their myeloarchitectural characteristics and mapped with regard to their exact location in the isocortex. The typical features were graphically documented and classified by a sophisticated linguistic coding. Their results have only recently received adequate attention and applications. The reasons for the revival of this spectrum of their research include interest in the myeloarchitecture of the cortex as a differentiating feature of the cortex architecture and function, as well as the importance for advanced imaging methodologies, particularly tractography and molecular imaging. Here, we describe our approach to exploit the original work of the Vogts and their co-workers to construct a myeloarchitectonic map that is referenced to the Atlas of the Human Brain (AHB) in standard space. We developed a semi-automatic pipeline for processing and integrating the various original maps into a single coherent map. To optimize the precision of the registration between the published maps and the AHB, we augmented the maps with topographic landmarks of the brains that were originally analyzed. Registration of all maps into the AHB opened several possibilities. First, for the majority of the fields, multiple maps from different authors are available, which allows for sophisticated statistical integration, for example, unification with a label-fusion technique. Second, each field in the myeloarchitectonic surface map can be visualized on the myelin-stained cross-section of the AHB at the best possible correspondence. The features of each field can be correlated with the fiber-stained cross-sections in the AHB and with the extensive published materials from the Vogt school and, if necessary, corrected. Third, mapping to the AHB allows the relationship between fiber characteristics of the cortex and the subcortex to be examined. Fourth, the cytoarchitectonic maps from Brodmann and von Economo and Koskinas, which are also registered to the AHB, can be compared. This option allows the study of the correspondence between cyto- and myeloarchitecture in each field. Finally, by using our "stripe" technology - where any other feature registered to the same space can be directly compared owing to the linear and parallel representation of the correlated cortex segments - this map becomes part of a multidimensional co-registration platform.
| S-EPMC10751573 | biostudies-literature
Project description:An integral part of human language is the capacity to extract meaning from spoken and written words, but the precise relationship between brain representations of information perceived by listening versus reading is unclear. Prior neuroimaging studies have shown that semantic information in spoken language is represented in multiple regions in the human cerebral cortex, while amodal semantic information appears to be represented in a few broad brain regions. However, previous studies were too insensitive to determine whether semantic representations were shared at a fine level of detail rather than merely at a coarse scale. We used fMRI to record brain activity in two separate experiments while participants listened to or read several hours of the same narrative stories, and then created voxelwise encoding models to characterize semantic selectivity in each voxel and in each individual participant. We find that semantic tuning during listening and reading are highly correlated in most semantically selective regions of cortex, and models estimated using one modality accurately predict voxel responses in the other modality. These results suggest that the representation of language semantics is independent of the sensory modality through which the semantic information is received.SIGNIFICANCE STATEMENT Humans can comprehend the meaning of words from both spoken and written language. It is therefore important to understand the relationship between the brain representations of spoken or written text. Here, we show that although the representation of semantic information in the human brain is quite complex, the semantic representations evoked by listening versus reading are almost identical. These results suggest that the representation of language semantics is independent of the sensory modality through which the semantic information is received.
| S-EPMC6764208 | biostudies-literature
Project description:Direct intracranial recording of human brain activity is an important approach for deciphering neural mechanisms of cognition. Such recordings, usually made in patients with epilepsy undergoing inpatient monitoring for seizure localization, are limited in duration and depend on patients' tolerance for the challenges associated with recovering from brain surgery. Thus, typical intracranial recordings, similar to most non-invasive approaches in humans, provide snapshots of brain activity in acute, highly constrained settings, limiting opportunities to understand long timescale and natural, real-world phenomena. A new device for treating some forms of drug-resistant epilepsy, the NeuroPace RNS® System, includes a cranially-implanted neurostimulator and intracranial electrodes that continuously monitor brain activity and respond to incipient seizures with electrical counterstimulation. The RNS System can record epileptic brain activity over years, but whether it can record meaningful, behavior-related physiological responses has not been demonstrated. Here, in a human subject with electrodes implanted over high-level speech-auditory cortex (Wernicke's area; posterior superior temporal gyrus), we report that cortical evoked responses to spoken sentences are robust, selective to phonetic features, and stable over nearly 1.5 years. In a second subject with RNS System electrodes implanted over frontal cortex (Broca's area, posterior inferior frontal gyrus), we found that word production during a naming task reliably evokes cortical responses preceding speech onset. The spatiotemporal resolution, high signal-to-noise, and wireless nature of this system's intracranial recordings make it a powerful new approach to investigate the neural correlates of human cognition over long timescales in natural ambulatory settings.
| S-EPMC5482367 | biostudies-literature
Project description:The Semantic Coherence Dataset has been designed to experiment with semantic coherence metrics. More specifically, the dataset has been built to the ends of testing whether probabilistic measures, such as perplexity, provide stable scores to analyze spoken language. Perplexity, which was originally conceived as an information-theoretic measure to assess the probabilistic inference properties of language models, has recently been proven to be an appropriate tool to categorize speech transcripts based on semantic coherence accounts. More specifically, perplexity has been successfully employed to discriminate subjects suffering from Alzheimer Disease and healthy controls. Collected data include speech transcripts, intended to investigate semantic coherence at different levels: data are thus arranged into two classes, to investigate intra-subject semantic coherence, and inter-subject semantic coherence. In the former case transcripts from a single speaker can be employed to train and test language models and to explore whether the perplexity metric provides stable scores in assessing talks from that speaker, while allowing to distinguish between two different forms of speech, political rallies and interviews. In the latter case, models can be trained by employing transcripts from a given speaker, and then used to measure how stable the perplexity metric is when computed using the model from that user and transcripts from different users. Transcripts were extracted from talks lasting almost 13 hours (overall 12:45:17 and 120,326 tokens) for the former class; and almost 30 hours (29:47:34 and 252,270 tokens) for the latter one. Data herein can be reused to perform analyses on measures built on top of language models, and more in general on measures that are aimed at exploring the linguistic features of text documents.
| S-EPMC9761592 | biostudies-literature