The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension.
Ontology highlight
ABSTRACT:
SUBMITTER: Nastase SA
PROVIDER: S-EPMC8479122 | biostudies-literature | 2021 Sep
REPOSITORIES: biostudies-literature
Publications
The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension.
Nastase Samuel A SA Liu Yun-Fei YF Hillman Hanna H Zadbood Asieh A Hasenfratz Liat L Keshavarzian Neggin N Chen Janice J Honey Christopher J CJ Yeshurun Yaara Y Regev Mor M Nguyen Mai M Chang Claire H C CHC Baldassano Christopher C Lositsky Olga O Simony Erez E Chow Michael A MA Leong Yuan Chang YC Brooks Paula P PP Micciche Emily E Choe Gina G Goldstein Ariel A Vanderwal Tamara T Halchenko Yaroslav O YO Norman Kenneth A KA Hasson Uri U
Scientific data 20210928 1
The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardiz ...[more]
Similar Datasets
Project description:Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such predictive coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that predictive coding in the brain's response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.
| S-EPMC7140726 | biostudies-literature
Project description:Facial electromyography research shows that corrugator supercilii ("frowning muscle") activity tracks the emotional valence of linguistic stimuli. Grounded or embodied accounts of language processing take such activity to reflect the simulation or "re-enactment" of emotion, as part of the retrieval of word meaning (e.g., of "furious") and/or of building a situation model (e.g., for "Mark is furious"). However, the same muscle also expresses our primary emotional evaluation of things we encounter. Language-driven affective simulation can easily be at odds with the reader's affective evaluation of what language describes (e.g., when we like Mark being furious). In a previous experiment ('t Hart et al., 2018) we demonstrated that neither language-driven simulation nor affective evaluation alone seem sufficient to explain the corrugator patterns that emerge during online language comprehension in these complex cases. Those results showed support for a multiple-drivers account of corrugator activity, where both simulation and evaluation processes contribute to the activation patterns observed in the corrugator. The study at hand replicates and extends these findings. With more refined control over when precisely affective information became available in a narrative, we again find results that speak against an interpretation of corrugator activity in terms of simulation or evaluation alone, and as such support the multiple-drivers account. Additional evidence suggests that the simulation driver involved reflects simulation at the level of situation model construction, rather than at the level of retrieving concepts from long-term memory. In all, by giving insights into how language-driven simulation meshes with the reader's evaluative responses during an unfolding narrative, this study contributes to the understanding of affective language comprehension.
| S-EPMC6398452 | biostudies-literature
Project description:Speech comprehension is a complex process that draws on humans' abilities to extract lexical information, parse syntax, and form semantic understanding. These sub-processes have traditionally been studied using separate neuroimaging experiments that attempt to isolate specific effects of interest. More recently it has become possible to study all stages of language comprehension in a single neuroimaging experiment using narrative natural language stimuli. The resulting data are richly varied at every level, enabling analyses that can probe everything from spectral representations to high-level representations of semantic meaning. We provide a dataset containing BOLD fMRI responses recorded while 8 participants each listened to 27 complete, natural, narrative stories (~6 hours). This dataset includes pre-processed and raw MRIs, as well as hand-constructed 3D cortical surfaces for each participant. To address the challenges of analyzing naturalistic data, this dataset is accompanied by a python library containing basic code for creating voxelwise encoding models. Altogether, this dataset provides a large and novel resource for understanding speech and language processing in the human brain.
| S-EPMC10447563 | biostudies-literature
Project description:Recently, cognitive neuroscientists have increasingly studied the brain responses to narratives. At the same time, we are witnessing exciting developments in natural language processing where large-scale neural network models can be used to instantiate cognitive hypotheses in narrative processing. Yet, they learn from text alone and we lack ways of incorporating biological constraints during training. To mitigate this gap, we provide a narrative comprehension magnetoencephalography (MEG) data resource that can be used to train neural network models directly on brain data. We recorded from 3 participants, 10 separate recording hour-long sessions each, while they listened to audiobooks in English. After story listening, participants answered short questions about their experience. To minimize head movement, the participants wore MEG-compatible head casts, which immobilized their head position during recording. We report a basic evoked-response analysis showing that the responses accurately localize to primary auditory areas. The responses are robust and conserved across 10 sessions for every participant. We also provide usage notes and briefly outline possible future uses of the resource.
| S-EPMC9177538 | biostudies-literature
Project description:Do embodied semantic systems play different roles depending on when and how well a given language was learned? Emergent evidence suggests that this is the case for isolated, decontextualized stimuli, but no study has addressed the issue considering naturalistic narratives. Seeking to bridge this gap, we assessed motor-system dynamics in 26 Spanish-English bilinguals as they engaged in free, unconstrained reading of naturalistic action texts (ATs, highlighting the characters' movements) and neutral texts (NTs, featuring low motility) in their first and second language (L1, L2). To explore functional connectivity spread over each reading session, we recorded ongoing high-density electroencephalographic signals and subjected them to functional connectivity analysis via a spatial clustering approach. Results showed that, in L1, AT (relative to NT) reading involved increased connectivity between left and right central electrodes consistently implicated in action-related processes, as well as distinct source-level modulations in motor regions. In L2, despite null group-level effects, enhanced motor-related connectivity during AT reading correlated positively with L2 proficiency and negatively with age of L2 learning. Taken together, these findings suggest that action simulations during unconstrained narrative reading involve neural couplings between motor-sensitive mechanisms, in proportion to how consolidated a language is. More generally, such evidence addresses recent calls to test the ecological validity of motor-resonance effects while offering new insights on their relation with experiential variables.
| S-EPMC7412856 | biostudies-literature
Project description:The collection of eye gaze information during functional magnetic resonance imaging (fMRI) is important for monitoring variations in attention and task compliance, particularly for naturalistic viewing paradigms (e.g., movies). However, the complexity and setup requirements of current in-scanner eye tracking solutions can preclude many researchers from accessing such information. Predictive eye estimation regression (PEER) is a previously developed support vector regression-based method for retrospectively estimating eye gaze from the fMRI signal in the eye's orbit using a 1.5-min calibration scan. Here, we provide confirmatory validation of the PEER method's ability to infer eye gaze on a TR-by-TR basis during movie viewing, using simultaneously acquired eye tracking data in five individuals (median angular deviation < 2°). Then, we examine variations in the predictive validity of PEER models across individuals in a subset of data (n = 448) from the Child Mind Institute Healthy Brain Network Biobank, identifying head motion as a primary determinant. Finally, we accurately classify which of the two movies is being watched based on the predicted eye gaze patterns (area under the curve = 0.90 ± 0.02) and map the neural correlates of eye movements derived from PEER. PEER is a freely available and easy-to-use tool for determining eye fixations during naturalistic viewing.
| S-EPMC7132907 | biostudies-literature
Project description:Parkinson's disease (PD) affects the language processes, with a significant impact on the patients' daily communication. We aimed to describe specific alterations in the comprehension of syntactically complex sentences in patients with PD (PwPD) as compared to healthy controls (HC) and to identify the neural underpinnings of these deficits using a functional connectivity analysis of the striatum. A total of 20 patients PwPD and 15 HC participated in the fMRI study. We analyzed their performance of a Test of sentence comprehension (ToSC) adjusted for fMRI. A task-dependent functional connectivity analysis of the striatum was conducted using the psychophysiological interaction method (PPI). On the behavioral level, the PwPD scored significantly lower (mean ± sd: 77.3 ± 12.6) in the total ToSC score than the HC did (mean ± sd: 86.6 ± 8.0), p = 0.02, and the difference was also significant specifically for sentences with a non-canonical word order (PD-mean ± sd: 69.9 ± 14.1, HC-mean ± sd: 80.2 ± 11.5, p = 0.04). Using PPI, we found a statistically significant difference between the PwPD and the HC in connectivity from the right striatum to the supplementary motor area [SMA, (4 8 53)] for non-canonical sentences. This PPI connectivity was negatively correlated with the ToSC accuracy of non-canonical sentences in the PwPD. Our results showed disturbed sentence reading comprehension in the PwPD with altered task-dependent functional connectivity from the right striatum to the SMA, which supports the synchronization of the temporal and sequential aspects of language processing. The study revealed that subcortical-cortical networks (striatal-frontal loop) in PwPD are compromised, leading to impaired comprehension of syntactically complex sentences.
| S-EPMC10033839 | biostudies-literature
Project description:One ultimate goal of visual neuroscience is to understand how the brain processes visual stimuli encountered in the natural environment. Achieving this goal requires records of brain responses under massive amounts of naturalistic stimuli. Although the scientific community has put a lot of effort into collecting large-scale functional magnetic resonance imaging (fMRI) data under naturalistic stimuli, more naturalistic fMRI datasets are still urgently needed. We present here the Natural Object Dataset (NOD), a large-scale fMRI dataset containing responses to 57,120 naturalistic images from 30 participants. NOD strives for a balance between sampling variation between individuals and sampling variation between stimuli. This enables NOD to be utilized not only for determining whether an observation is generalizable across many individuals, but also for testing whether a response pattern is generalized to a variety of naturalistic stimuli. We anticipate that the NOD together with existing naturalistic neuroimaging datasets will serve as a new impetus for our understanding of the visual processing of naturalistic stimuli.
| S-EPMC10447576 | biostudies-literature
Project description:Intracranial human recordings are a valuable and rare resource of information about the brain. Making such data publicly available not only helps tackle reproducibility issues in science, it helps make more use of these valuable data. This is especially true for data collected using naturalistic tasks. Here, we describe a dataset collected from a large group of human subjects while they watched a short audiovisual film. The dataset has several unique features. First, it includes a large amount of intracranial electroencephalography (iEEG) data (51 participants, age range of 5-55 years, who all performed the same task). Second, it includes functional magnetic resonance imaging (fMRI) recordings (30 participants, age range of 7-47) during the same task. Eighteen participants performed both iEEG and fMRI versions of the task, non-simultaneously. Third, the data were acquired using a rich audiovisual stimulus, for which we provide detailed speech and video annotations. This dataset can be used to study neural mechanisms of multimodal perception and language comprehension, and similarity of neural signals across brain recording modalities.
| S-EPMC8938409 | biostudies-literature
Project description:The neurobiology of sentence comprehension is well-studied but the properties and characteristics of sentence processing networks remain unclear and highly debated. Sign languages (i.e., visual-manual languages), like spoken languages, have complex grammatical structures and thus can provide valuable insights into the specificity and function of brain regions supporting sentence comprehension. The present study aims to characterize how these well-studied spoken language networks can adapt in adults to be responsive to sign language sentences, which contain combinatorial semantic and syntactic visual-spatial linguistic information. Twenty native English-speaking undergraduates who had completed introductory American Sign Language (ASL) courses viewed videos of the following conditions during fMRI acquisition: signed sentences, signed word lists, English sentences and English word lists. Overall our results indicate that native language (L1) sentence processing resources are responsive to ASL sentence structures in late L2 learners, but that certain L1 sentence processing regions respond differently to L2 ASL sentences, likely due to the nature of their contribution to language comprehension. For example, L1 sentence regions in Broca's area were significantly more responsive to L2 than L1 sentences, supporting the hypothesis that Broca's area contributes to sentence comprehension as a cognitive resource when increased processing is required. Anterior temporal L1 sentence regions were sensitive to L2 ASL sentence structure, but demonstrated no significant differences in activation to L1 than L2, suggesting its contribution to sentence processing is modality-independent. Posterior superior temporal L1 sentence regions also responded to ASL sentence structure but were more activated by English than ASL sentences. An exploratory analysis of the neural correlates of L2 ASL proficiency indicates that ASL proficiency is positively correlated with increased activations in response to ASL sentences in L1 sentence processing regions. Overall these results suggest that well-established fronto-temporal spoken language networks involved in sentence processing exhibit functional plasticity with late L2 ASL exposure, and thus are adaptable to syntactic structures widely different than those in an individual's native language. Our findings also provide valuable insights into the unique contributions of the inferior frontal and superior temporal regions that are frequently implicated in sentence comprehension but whose exact roles remain highly debated.
| S-EPMC6136263 | biostudies-literature