Dataset Information

The communicative advantage: how kinematic signaling supports semantic comprehension.

ABSTRACT: Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.g., velocity, size) of such gestures. If and how this modulation influences addressees' comprehension of gestures have not been investigated. Here we ask whether communicative kinematic modulation enhances semantic comprehension (i.e., identification) of gestures. We additionally investigate whether any comprehension advantage is due to enhanced early identification or late identification. Participants (n?=?20) watched videos of representational gestures produced in a more- (n?=?60) or less-communicative (n?=?60) context and performed a forced-choice recognition task. We tested the isolated role of kinematics by removing visibility of actor's faces in Experiment I, and by reducing the stimuli to stick-light figures in Experiment II. Three video lengths were used to disentangle early identification from late identification. Accuracy and response time quantified main effects. Kinematic modulation was tested for correlations with task performance. We found higher gesture identification performance in more- compared to less-communicative gestures. However, early identification was only enhanced within a full visual context, while late identification occurred even when viewing isolated kinematics. Additionally, temporally segmented acts with more post-stroke holds were associated with higher accuracy. Our results demonstrate that communicative signaling, interacting with other visual cues, generally supports gesture identification, while kinematic modulation specifically enhances late identification in the absence of other cues. Results provide insights into mutual understanding processes as well as creating artificial communicative agents.

SUBMITTER: Trujillo JP

PROVIDER: S-EPMC7772160 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The communicative advantage: how kinematic signaling supports semantic comprehension.

Trujillo James P JP Simanova Irina I Bekkering Harold H Özyürek Asli A

Psychological research 20190511 7

Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.g., velocity, size) of such gestures. If and how this modulation influences addressees' comprehension of gestures have not been investigated. Here we ask whether communicative kinematic modulation enhances semantic comprehension (i.e., iden ...[more]

PMID: 31079227

Similar Datasets

Project description:Previous studies have shown that at moderate levels of spectral degradation, semantic predictability facilitates language comprehension. It is argued that when speech is degraded, listeners have narrowed expectations about the sentence endings; i.e., semantic prediction may be limited to only most highly predictable sentence completions. The main objectives of this study were to (i) examine whether listeners form narrowed expectations or whether they form predictions across a wide range of probable sentence endings, (ii) assess whether the facilitatory effect of semantic predictability is modulated by perceptual adaptation to degraded speech, and (iii) use and establish a sensitive metric for the measurement of language comprehension. For this, we created 360 German Subject-Verb-Object sentences that varied in semantic predictability of a sentence-final target word in a graded manner (high, medium, and low) and levels of spectral degradation (1, 4, 6, and 8 channels noise-vocoding). These sentences were presented auditorily to two groups: One group (n =48) performed a listening task in an unpredictable channel context in which the degraded speech levels were randomized, while the other group (n =50) performed the task in a predictable channel context in which the degraded speech levels were blocked. The results showed that at 4 channels noise-vocoding, response accuracy was higher in high-predictability sentences than in the medium-predictability sentences, which in turn was higher than in the low-predictability sentences. This suggests that, in contrast to the narrowed expectations view, comprehension of moderately degraded speech, ranging from low- to high- including medium-predictability sentences, is facilitated in a graded manner; listeners probabilistically preactivate upcoming words from a wide range of semantic space, not limiting only to highly probable sentence endings. Additionally, in both channel contexts, we did not observe learning effects; i.e., response accuracy did not increase over the course of experiment, and response accuracy was higher in the predictable than in the unpredictable channel context. We speculate from these observations that when there is no trial-by-trial variation of the levels of speech degradation, listeners adapt to speech quality at a long timescale; however, when there is a trial-by-trial variation of the high-level semantic feature (e.g., sentence predictability), listeners do not adapt to low-level perceptual property (e.g., speech quality) at a short timescale.

Project description:Sentence comprehension involves maintaining and continuously integrating linguistic information and, thus, makes demands on working memory (WM). Past research has demonstrated that semantic WM, but not phonological WM, is critical for integrating word meanings across some distance and resolving semantic interference in sentence comprehension. Here, we examined the relation between phonological and semantic WM and the comprehension of center-embedded relative clause sentences, often argued to make heavy demands on WM. Additionally, we examined the relation between phonological and semantic WM and the comprehension of transitive and dative active and passive sentences, which may also draw on WM resources depending on the number of propositions that must be maintained and the difficulty of processing passive clauses. In a large sample of individuals with aphasia (N = 56), we assessed whether comprehension performance on more complex vs. simpler active-passive or embedded relative clause sentences would be predicted by semantic but not phonological WM when controlling for single word comprehension. For performance on the active-passive comprehension task, we found that semantic WM, but not phonological WM, predicted comprehension of dative sentences when controlling for comprehension of transitive sentences. We also found that phonological WM, but not semantic WM, predicted mean comprehension for reversible active-passive sentences when controlling for trials with lexical distractors. On the relative clause comprehension task, consistent with prior results, we found that semantic WM, but not phonological WM, predicted comprehension of object relative clause sentences and relative clause sentences with a passive construction. However, both phonological WM and semantic WM predicted mean comprehension across all relative clause types for reversible trials when controlling for trials with lexical distractors. While we found evidence of semantic WM’s role in comprehension, we also observed unpredicted relations between phonological WM and comprehension in some conditions. Post-hoc analyses provided preliminary evidence that phonological WM maintains a backup phonological representation of the sentence that may be accessed when sentence comprehension processing is less efficient. Future work should investigate possible roles that phonological WM may play across sentence types.

Dataset Information

The communicative advantage: how kinematic signaling supports semantic comprehension.

Publications

The communicative advantage: how kinematic signaling supports semantic comprehension.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets