Dataset Information

Invariant recognition drives neural representations of action sequences.

ABSTRACT: Recognizing the actions of others from visual stimuli is a crucial aspect of human perception that allows individuals to respond to social cues. Humans are able to discriminate between similar actions despite transformations, like changes in viewpoint or actor, that substantially alter the visual appearance of a scene. This ability to generalize across complex transformations is a hallmark of human visual intelligence. Advances in understanding action recognition at the neural level have not always translated into precise accounts of the computational principles underlying what representations of action sequences are constructed by human visual cortex. Here we test the hypothesis that invariant action discrimination might fill this gap. Recently, the study of artificial systems for static object perception has produced models, Convolutional Neural Networks (CNNs), that achieve human level performance in complex discriminative tasks. Within this class, architectures that better support invariant object recognition also produce image representations that better match those implied by human and primate neural data. However, whether these models produce representations of action sequences that support recognition across complex transformations and closely follow neural representations of actions remains unknown. Here we show that spatiotemporal CNNs accurately categorize video stimuli into action classes, and that deliberate model modifications that improve performance on an invariant action recognition task lead to data representations that better match human neural recordings. Our results support our hypothesis that performance on invariant discrimination dictates the neural representations of actions computed in the brain. These results broaden the scope of the invariant recognition framework for understanding visual intelligence from perception of inanimate objects and faces in static images to the study of human perception of action sequences.

SUBMITTER: Tacchetti A

PROVIDER: S-EPMC5749869 | biostudies-literature | 2017 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Invariant recognition drives neural representations of action sequences.

Tacchetti Andrea A Isik Leyla L Poggio Tomaso T

PLoS computational biology 20171218 12

Recognizing the actions of others from visual stimuli is a crucial aspect of human perception that allows individuals to respond to social cues. Humans are able to discriminate between similar actions despite transformations, like changes in viewpoint or actor, that substantially alter the visual appearance of a scene. This ability to generalize across complex transformations is a hallmark of human visual intelligence. Advances in understanding action recognition at the neural level have not alw ...[more]

PMID: 29253864

Similar Datasets

Project description:How do athletes represent actions from their sport? How are these representations structured and which knowledge is shared among experts in the same discipline? To address these questions, the event segmentation task was used. Experts in Taekwondo and novices indicated how they would subjectively split videos of Taekwondo form sequences into meaningful units. In previous research, this procedure was shown to unveil the structure of internal action representations and to be affected by sensorimotor knowledge. Without specific instructions on the grain size of segmentation, experts tended to integrate over longer episodes which resulted in a lower number of single units. Moreover, in accordance with studies in figure-skating and basketball, we expected higher agreement among experts on where to place segmentation marks, i.e., boundaries. In line with this hypothesis, significantly more overlap of boundaries was found within the expert group as compared to the control group. This was observed even though the interindividual differences in the selected grain size were huge and expertise had no systematic influence here. The absence of obvious goals or objects to structure Taekwondo forms underlines the importance of shared expert knowledge. Further, experts might have benefited from sensorimotor skills which allowed to simulate the observed actions more precisely. Both aspects may explain stronger agreement among experts even in unfamiliar Taekwondo forms. These interpretations are descriptively supported by the participants' statements about features which guided segmentation and by an overlap of the group's agreed boundaries with those of an experienced referee. The study shows that action segmentation can be used to provide insights into structure and content of action representations specific to experts. The mechanisms underlying shared knowledge among Taekwondoists and among experts in general are discussed on the background of current theoretic frameworks.

Dataset Information

Invariant recognition drives neural representations of action sequences.

Publications

Invariant recognition drives neural representations of action sequences.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets