Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

The CANDOR corpus: Insights from a large multimodal dataset of naturalistic conversation.

ABSTRACT: People spend a substantial portion of their lives engaged in conversation, and yet, our scientific understanding of conversation is still in its infancy. Here, we introduce a large, novel, and multimodal corpus of 1656 conversations recorded in spoken English. This 7+ million word, 850-hour corpus totals more than 1 terabyte of audio, video, and transcripts, with moment-to-moment measures of vocal, facial, and semantic expression, together with an extensive survey of speakers' postconversation reflections. By taking advantage of the considerable scope of the corpus, we explore many examples of how this large-scale public dataset may catalyze future research, particularly across disciplinary boundaries, as scholars from a variety of fields appear increasingly interested in the study of conversation.

SUBMITTER: Reece A

PROVIDER: S-EPMC10065445 | biostudies-literature | 2023 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

The CANDOR corpus: Insights from a large multimodal dataset of naturalistic conversation.

Reece Andrew A Cooney Gus G Bull Peter P Chung Christine C Dawson Bryn B Fitzpatrick Casey C Glazer Tamara T Knox Dean D Liebscher Alex A Marin Sebastian S

Science advances 20230331 13

People spend a substantial portion of their lives engaged in conversation, and yet, our scientific understanding of conversation is still in its infancy. Here, we introduce a large, novel, and multimodal corpus of 1656 conversations recorded in spoken English. This 7+ million word, 850-hour corpus totals more than 1 terabyte of audio, video, and transcripts, with moment-to-moment measures of vocal, facial, and semantic expression, together with an extensive survey of speakers' postconversation r ...[more]

PMID: 37000886

Similar Datasets

K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations.

Project description:Recognizing emotions during social interactions has many potential applications with the popularization of low-cost mobile sensors, but a challenge remains with the lack of naturalistic affective interaction data. Most existing emotion datasets do not support studying idiosyncratic emotions arising in the wild as they were collected in constrained environments. Therefore, studying emotions in the context of social interactions requires a novel dataset, and K-EmoCon is such a multimodal dataset with comprehensive annotations of continuous emotions during naturalistic conversations. The dataset contains multimodal measurements, including audiovisual recordings, EEG, and peripheral physiological signals, acquired with off-the-shelf devices from 16 sessions of approximately 10-minute long paired debates on a social issue. Distinct from previous datasets, it includes emotion annotations from all three available perspectives: self, debate partner, and external observers. Raters annotated emotional displays at intervals of every 5 seconds while viewing the debate footage, in terms of arousal-valence and 18 additional categorical emotions. The resulting K-EmoCon is the first publicly available emotion dataset accommodating the multiperspective assessment of emotions during social interactions.

| S-EPMC7479607 | biostudies-literature

Open multimodal iEEG-fMRI dataset from naturalistic stimulation with a short audiovisual film.

Project description:Intracranial human recordings are a valuable and rare resource of information about the brain. Making such data publicly available not only helps tackle reproducibility issues in science, it helps make more use of these valuable data. This is especially true for data collected using naturalistic tasks. Here, we describe a dataset collected from a large group of human subjects while they watched a short audiovisual film. The dataset has several unique features. First, it includes a large amount of intracranial electroencephalography (iEEG) data (51 participants, age range of 5-55 years, who all performed the same task). Second, it includes functional magnetic resonance imaging (fMRI) recordings (30 participants, age range of 7-47) during the same task. Eighteen participants performed both iEEG and fMRI versions of the task, non-simultaneously. Third, the data were acquired using a rich audiovisual stimulus, for which we provide detailed speech and video annotations. This dataset can be used to study neural mechanisms of multimodal perception and language comprehension, and similarity of neural signals across brain recording modalities.

| S-EPMC8938409 | biostudies-literature

Le Petit Prince multilingual naturalistic fMRI corpus.

Project description:Neuroimaging using more ecologically valid stimuli such as audiobooks has advanced our understanding of natural language comprehension in the brain. However, prior naturalistic stimuli have typically been restricted to a single language, which limited generalizability beyond small typological domains. Here we present the Le Petit Prince fMRI Corpus (LPPC-fMRI), a multilingual resource for research in the cognitive neuroscience of speech and language during naturalistic listening (OpenNeuro: ds003643). 49 English speakers, 35 Chinese speakers and 28 French speakers listened to the same audiobook The Little Prince in their native language while multi-echo functional magnetic resonance imaging was acquired. We also provide time-aligned speech annotation and word-by-word predictors obtained using natural language processing tools. The resulting timeseries data are shown to be of high quality with good temporal signal-to-noise ratio and high inter-subject correlation. Data-driven functional analyses provide further evidence of data quality. This annotated, multilingual fMRI dataset facilitates future re-analysis that addresses cross-linguistic commonalities and differences in the neural substrate of language processing on multiple perceptual and linguistic levels.

| S-EPMC9424229 | biostudies-literature

The COUGHVID crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms.

Project description:Cough audio signal classification has been successfully used to diagnose a variety of respiratory conditions, and there has been significant interest in leveraging Machine Learning (ML) to provide widespread COVID-19 screening. The COUGHVID dataset provides over 25,000 crowdsourced cough recordings representing a wide range of participant ages, genders, geographic locations, and COVID-19 statuses. First, we contribute our open-sourced cough detection algorithm to the research community to assist in data robustness assessment. Second, four experienced physicians labeled more than 2,800 recordings to diagnose medical abnormalities present in the coughs, thereby contributing one of the largest expert-labeled cough datasets in existence that can be used for a plethora of cough audio classification tasks. Finally, we ensured that coughs labeled as symptomatic and COVID-19 originate from countries with high infection rates. As a result, the COUGHVID dataset contributes a wealth of cough recordings for training ML models to address the world's most urgent health crises.

| S-EPMC8222356 | biostudies-literature

Working Together: Contributions of Corpus Analyses and Experimental Psycholinguistics to Understanding Conversation.

Project description:As conversation is the most important way of using language, linguists and psychologists should combine forces to investigate how interlocutors deal with the cognitive demands arising during conversation. Linguistic analyses of corpora of conversation are needed to understand the structure of conversations, and experimental work is indispensable for understanding the underlying cognitive processes. We argue that joint consideration of corpus and experimental data is most informative when the utterances elicited in a lab experiment match those extracted from a corpus in relevant ways. This requirement to compare like with like seems obvious but is not trivial to achieve. To illustrate this approach, we report two experiments where responses to polar (yes/no) questions were elicited in the lab and the response latencies were compared to gaps between polar questions and answers in a corpus of conversational speech. We found, as expected, that responses were given faster when they were easy to plan and planning could be initiated earlier than when they were harder to plan and planning was initiated later. Overall, in all but one condition, the latencies were longer than one would expect based on the analyses of corpus data. We discuss the implication of this partial match between the data sets and more generally how corpus and experimental data can best be combined in studies of conversation.

| S-EPMC5906713 | biostudies-literature

SpeakingFaces: A Large-Scale Multimodal Dataset of Voice Commands with Visual and Thermal Video Streams.

Project description:We present SpeakingFaces as a publicly-available large-scale multimodal dataset developed to support machine learning research in contexts that utilize a combination of thermal, visual, and audio data streams; examples include human-computer interaction, biometric authentication, recognition systems, domain transfer, and speech recognition. SpeakingFaces is comprised of aligned high-resolution thermal and visual spectra image streams of fully-framed faces synchronized with audio recordings of each subject speaking approximately 100 imperative phrases. Data were collected from 142 subjects, yielding over 13,000 instances of synchronized data (∼3.8 TB). For technical validation, we demonstrate two baseline examples. The first baseline shows classification by gender, utilizing different combinations of the three data streams in both clean and noisy environments. The second example consists of thermal-to-visual facial image translation, as an instance of domain transfer.

| S-EPMC8156799 | biostudies-literature

Multimodal Neurocognitive Markers of Naturalistic Discourse Typify Diverse Neurodegenerative Diseases.

Project description:Neurodegeneration has multiscalar impacts, including behavioral, neuroanatomical, and neurofunctional disruptions. Can disease-differential alterations be captured across such dimensions using naturalistic stimuli? To address this question, we assessed comprehension of four naturalistic stories, highlighting action, nonaction, social, and nonsocial events, in Parkinson's disease (PD) and behavioral variant frontotemporal dementia (bvFTD) relative to Alzheimer's disease patients and healthy controls. Text-specific correlates were evaluated via voxel-based morphometry, spatial (fMRI), and temporal (hd-EEG) functional connectivity. PD patients presented action-text deficits related to the volume of action-observation regions, connectivity across motor-related and multimodal-semantic hubs, and frontal hd-EEG hypoconnectivity. BvFTD patients exhibited social-text deficits, associated with atrophy and spatial connectivity patterns along social-network hubs, alongside right frontotemporal hd-EEG hypoconnectivity. Alzheimer's disease patients showed impairments in all stories, widespread atrophy and spatial connectivity patterns, and heightened occipitotemporal hd-EEG connectivity. Our framework revealed disease-specific signatures across behavioral, neuroanatomical, and neurofunctional dimensions, highlighting the sensitivity and specificity of a single naturalistic task. This investigation opens a translational agenda combining ecological approaches and multimodal cognitive neuroscience for the study of neurodegeneration.

| S-EPMC9376869 | biostudies-literature

A high-density diffuse optical tomography dataset of naturalistic viewing.

Project description:Traditional laboratory tasks offer tight experimental control but lack the richness of our everyday human experience. As a result many cognitive neuroscientists have been motivated to adopt experimental paradigms that are more natural, such as stories and movies. Here we describe data collected from 58 healthy adult participants (aged 18-76 years) who viewed 10 minutes of a movie (The Good, the Bad, and the Ugly, 1966). Most (36) participants viewed the clip more than once, resulting in 106 sessions of data. Cortical responses were mapped using high-density diffuse optical tomography (first- through fourth nearest neighbor separations of 1.3, 3.0, 3.9, and 4.7 cm), covering large portions of superficial occipital, temporal, parietal, and frontal lobes. Consistency of measured activity across subjects was quantified using intersubject correlation analysis. Data are provided in both channel format (SNIRF) and projected to standard space (NIfTI), using an atlas-based light model. These data are suitable for methods exploration as well as investigating a wide variety of cognitive phenomena.

| S-EPMC10659362 | biostudies-literature

The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension.

Project description:The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

| S-EPMC8479122 | biostudies-literature

ComPara: A corpus linguistics in English of computation in architecture dataset.

Project description:ComPara is a corpus linguistics dataset in English focused on computational architecture or architecture where technology functions as a driver for its conceptualization, design, and materialization. Sometimes computational architecture is also referred to as digital, parametric, algorithmic or generative architecture, and, as has been shown, each of these terms has different flavours [9]. Other corpus linguistics for architecture have been built containing texts written over a relatively limited time span and focusing on the language used in the profession in general [1,2]. The text which makes up ComPara is written between 2005 and 2019 and focuses on computational architecture. The corpus is built from two sources: the journal Architectural Design[3] and the eVolo skyscraper competition[4]. The former is one of the journals which has focused most on the theoretical discourse surrounding computation in architecture [5], while the latter is one of the most prestigious competitions focusing on 'technological advancements in architecture' [4]. The corpus includes the titles of Architectural Design's journal issues, titles of all articles and the keywords which are associated to the Introduction article in the journal's web page for each issue for the period between 2005 to 2019. From the eVolo Skyscraper competition, the titles of all winning projects and honorable mentions as well as all abstracts describing the projects between 2006 and 2019 were collected. This amounts to around 100.000 words. The purpose of building this dataset was to help gain a better understanding of the digitalization of architecture over 15 year time-span [8]. Further quantitative, qualitative or mixed method analysis can be carried out using the ComPara corpus by following specific topics or trends over time or by comparing the corpus to other sources.

| S-EPMC9062263 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data