Dataset Information

A multimodal dataset of spontaneous speech and movement production on object affordances.

ABSTRACT: In the longstanding effort of defining object affordances, a number of resources have been developed on objects and associated knowledge. These resources, however, have limited potential for modeling and generalization mainly due to the restricted, stimulus-bound data collection methodologies adopted. To-date, therefore, there exists no resource that truly captures object affordances in a direct, multimodal, and naturalistic way. Here, we present the first such resource of 'thinking aloud', spontaneously-generated verbal and motoric data on object affordances. This resource was developed from the reports of 124 participants divided into three behavioural experiments with visuo-tactile stimulation, which were captured audiovisually from two camera-views (frontal/profile). This methodology allowed the acquisition of approximately 95?hours of video, audio, and text data covering: object-feature-action data (e.g., perceptual features, namings, functions), Exploratory Acts (haptic manipulation for feature acquisition/verification), gestures and demonstrations for object/feature/action description, and reasoning patterns (e.g., justifications, analogies) for attributing a given characterization. The wealth and content of the data make this corpus a one-of-a-kind resource for the study and modeling of object affordances.

SUBMITTER: Vatakis A

PROVIDER: S-EPMC4718047 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A multimodal dataset of spontaneous speech and movement production on object affordances.

Vatakis Argiro A Pastra Katerina K

Scientific data 20160119

In the longstanding effort of defining object affordances, a number of resources have been developed on objects and associated knowledge. These resources, however, have limited potential for modeling and generalization mainly due to the restricted, stimulus-bound data collection methodologies adopted. To-date, therefore, there exists no resource that truly captures object affordances in a direct, multimodal, and naturalistic way. Here, we present the first such resource of 'thinking aloud', spon ...[more]

PMID: 26784391

Similar Datasets

Project description:Individuals typically produce auditory sequences, such as speech or music, at a consistent spontaneous rate or tempo. We addressed whether spontaneous rates would show patterns of convergence across the domains of music and language production when the same participants spoke sentences and performed melodic phrases on a piano. Although timing plays a critical role in both domains, different communicative and motor constraints apply in each case and so it is not clear whether music and speech would display similar timing mechanisms. We report the results of two experiments in which adult participants produced sequences from memory at a comfortable spontaneous (uncued) rate. In Experiment 1, monolingual pianists in Buffalo, New York engaged in three production tasks: speaking sentences from memory, performing short melodies from memory, and tapping isochronously. In Experiment 2, English-French bilingual pianists in Montréal, Canada produced melodies on a piano as in Experiment 1, and spoke short rhythmically-structured phrases repeatedly. Both experiments led to the same pattern of results. Participants exhibited consistent spontaneous rates within each task. People who produced one spoken phrase rapidly were likely to produce another spoken phrase rapidly. This consistency across stimuli was also found for performance of different musical melodies. In general, spontaneous rates across speech and music tasks were not correlated, whereas rates of tapping and music were correlated. Speech rates (for syllables) were faster than music rates (for tones) and speech showed a smaller range of spontaneous rates across individuals than did music or tapping rates. Taken together, these results suggest that spontaneous rate reflects cumulative influences of endogenous rhythms (in consistent self-generated rates within domain), peripheral motor constraints (in finger movements across tapping and music), and communicative goals based on the cultural transmission of auditory information (slower rates for to-be-synchronized music than for speech).

Project description:We built normative brain atlases that animate millisecond-scale intra- and inter-hemispheric white matter-level connectivity dynamics supporting object recognition and speech production. We quantified electrocorticographic modulations during three naming tasks using event-related high-gamma activity from 1,114 nonepileptogenic intracranial electrodes (i.e., non-lesional areas unaffected by epileptiform discharges). Using this electrocorticography data, we visualized functional connectivity modulations defined as significant naming-related high-gamma modulations occurring simultaneously at two sites connected by direct white matter streamlines on diffusion-weighted imaging tractography. Immediately after stimulus onset, intra- and inter-hemispheric functional connectivity enhancements were confined mainly across modality-specific perceptual regions. During response preparation, left intra-hemispheric connectivity enhancements propagated in a posterior-to-anterior direction, involving the left precentral and prefrontal areas. After overt response onset, inter- and intra-hemispheric connectivity enhancements mainly encompassed precentral, postcentral, and superior-temporal (STG) gyri. We found task-specific connectivity enhancements during response preparation as follows. Picture naming enhanced activity along the left arcuate fasciculus between the inferior-temporal and precentral/posterior inferior-frontal (pIFG) gyri. Nonspeech environmental sound naming augmented functional connectivity via the left inferior longitudinal and fronto-occipital fasciculi between the medial-occipital and STG/pIFG. Auditory descriptive naming task enhanced usage of the left frontal U-fibers, involving the middle-frontal gyrus. Taken together, the commonly observed network enhancements include inter-hemispheric connectivity optimizing perceptual processing exerted in each hemisphere, left intra-hemispheric connectivity supporting semantic and lexical processing, and inter-hemispheric connectivity for symmetric oral movements during overt speech. Our atlases improve the currently available models of object recognition and speech production by adding neural dynamics via direct intra- and inter-hemispheric white matter tracts.

Project description:Perception and action are tightly linkedobjects may be perceived not only in terms of visual features, but also in terms of possibilities for action. Previous studies showed that when a centrally located object has a salient graspable feature (e.g., a handle), it facilitates motor responses corresponding with the feature's position. However, such so-called affordance effects have been criticized as resulting from spatial compatibility effects, due to the visual asymmetry created by the graspable feature, irrespective of any affordances. In order to dissociate between affordance and spatial compatibility effects, we asked participants to perform a simple reaction-time task to typically graspable and non-graspable objects with similar visual features (e.g., lollipop and stop sign). Responses were measured using either electromyography (EMG) on proximal arm muscles during reaching-like movements, or with finger key-presses. In both EMG and button press measurements, participants responded faster when the object was either presented in the same location as the responding hand, or was affordable, resulting in significant and independent spatial compatibility and affordance effects, but no interaction. Furthermore, while the spatial compatibility effect was present from the earliest stages of movement preparation and throughout the different stages of movement execution, the affordance effect was restricted to the early stages of movement execution. Finally, we tested a small group of unilateral arm amputees using EMG, and found residual spatial compatibility but no affordance, suggesting that spatial compatibility effects do not necessarily rely on individuals' available affordances. Our results show dissociation between affordance and spatial compatibility effects, and suggest that rather than evoking the specific motor action most suitable for interaction with the viewed object, graspable objects prompt the motor system in a general, body-part independent fashion.

Dataset Information

A multimodal dataset of spontaneous speech and movement production on object affordances.

Publications

A multimodal dataset of spontaneous speech and movement production on object affordances.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets