Dataset Information

Inconsistent use of gesture space during abstract pointing impairs language comprehension.

ABSTRACT: Pointing toward concrete objects is a well-known and efficient communicative strategy. Much less is known about the communicative effectiveness of abstract pointing where the pointing gestures are directed to "empty space." McNeill's (2003) observations suggest that abstract pointing can be used to establish referents in gesture space, without the referents being physically present. Recently, however, it has been shown that abstract pointing typically provides redundant information to the uttered speech thereby suggesting a very limited communicative value (So et al., 2009). In a first approach to tackle this issue we were interested to know whether perceivers are sensitive at all to this gesture cue or whether it is completely discarded as irrelevant add-on information. Sensitivity to for instance a gesture-speech mismatch would suggest a potential communicative function of abstract pointing. Therefore, we devised a mismatch paradigm in which participants watched a video where a female was interviewed on various topics. During her responses, she established two concepts in space using abstract pointing (e.g., pointing to the left when saying Donald, and pointing to the right when saying Mickey). In the last response to each topic, the pointing gesture accompanying a target word (e.g., Donald) was either consistent or inconsistent with the previously established location. Event related brain potentials showed an increased N400 and P600 when gesture and speech referred to different referents, indicating that inconsistent use of gesture space impairs language comprehension. Abstract pointing was found to influence comprehension even though gesture was not crucial to understanding the sentences or conducting the experimental task. These data suggest that a referent was retrieved via abstract pointing and that abstract pointing can potentially be used for referent indication in a discourse. We conclude that abstract pointing has a potential communicative function.

SUBMITTER: Gunter TC

PROVIDER: S-EPMC4321330 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Inconsistent use of gesture space during abstract pointing impairs language comprehension.

Gunter Thomas C TC Weinbrenner J E Douglas JE Holle Henning H

Frontiers in psychology 20150209

Pointing toward concrete objects is a well-known and efficient communicative strategy. Much less is known about the communicative effectiveness of abstract pointing where the pointing gestures are directed to "empty space." McNeill's (2003) observations suggest that abstract pointing can be used to establish referents in gesture space, without the referents being physically present. Recently, however, it has been shown that abstract pointing typically provides redundant information to the uttere ...[more]

PMID: 25709591

Similar Datasets

Project description:An ongoing issue of interest in second language research concerns what transfers from a speaker's first language to their second. For learners of a sign language, gesture is a potential substrate for transfer. Our study provides a novel test of gestural production by eliciting silent gesture from novices in a controlled environment. We focus on spatial relationships, which in sign languages are represented in a very iconic way using the hands, and which one might therefore predict to be easy for adult learners to acquire. However, a previous study by Marshall and Morgan (2015) revealed that this was only partly the case: in a task that required them to express the relative locations of objects, hearing adult learners of British Sign Language (BSL) could represent objects' locations and orientations correctly, but had difficulty selecting the correct handshapes to represent the objects themselves. If hearing adults are indeed drawing upon their gestural resources when learning sign languages, then their difficulties may have stemmed from their having in manual gesture only a limited repertoire of handshapes to draw upon, or, alternatively, from having too broad a repertoire. If the first hypothesis is correct, the challenge for learners is to extend their handshape repertoire, but if the second is correct, the challenge is instead to narrow down to the handshapes appropriate for that particular sign language. 30 sign-naïve hearing adults were tested on Marshall and Morgan's task. All used some handshapes that were different from those used by native BSL signers and learners, and the set of handshapes used by the group as a whole was larger than that employed by native signers and learners. Our findings suggest that a key challenge when learning to express locative relations might be reducing from a very large set of gestural resources, rather than supplementing a restricted one, in order to converge on the conventionalized classifier system that forms part of the grammar of the language being learned.

Project description:Using functional magnetic resonance imaging (fMRI), we identified cortical regions mediating interpretive processes that take place during language comprehension. We manipulated participants' interpretation of texts by asking them to focus on action-, space-, or time-related features while listening to identical short stories. We identify several cortical regions where activity varied significantly in response to this attention manipulation, even though the content being processed was exactly the same. Activity in the posterior and anterior sections of the left inferior frontal gyrus (IFG), which are thought to have different sensitivities to high-level language processing, was modulated by the listeners' attentional focus, but in ways that were quite different. The posterior left IFG (Pars Opercularis) showed different activity levels for the three conditions. However, a population coding analysis demonstrated similar distributions of activity across conditions. This suggests that while the gain of the response in the Pars Opercularis was modulated, its core organization was relatively invariant across the experimental conditions. In the anterior left IFG (Pars Triangularis), the analysis of population codes revealed different activity patterns between conditions: there was little similarity between activity during time-attention and action- and space-attention, however there were similar activity patterns while attending to space and action information. In addition, both the left superior temporal gyrus and sulcus showed greater activity in the space and action attention conditions when contrasted with time attention. We discuss these findings in light of work on the role of left IFG in processing semantic information in language, and in light of theories suggesting that temporal information in language is processed in the brain using similar mechanisms as spatial information. Our findings suggest that a substantial source of variance in neural activity during language comprehension emerges from the internally-driven, information-seeking preferences of listeners rather than the syntactic or semantic properties of a text.

Dataset Information

Inconsistent use of gesture space during abstract pointing impairs language comprehension.

Publications

Inconsistent use of gesture space during abstract pointing impairs language comprehension.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets