Dataset Information

Computational mechanisms underlying cortical responses to the affordance properties of visual scenes.

ABSTRACT: Biologically inspired deep convolutional neural networks (CNNs), trained for computer vision tasks, have been found to predict cortical responses with remarkable accuracy. However, the internal operations of these models remain poorly understood, and the factors that account for their success are unknown. Here we develop a set of techniques for using CNNs to gain insights into the computational mechanisms underlying cortical responses. We focused on responses in the occipital place area (OPA), a scene-selective region of dorsal occipitoparietal cortex. In a previous study, we showed that fMRI activation patterns in the OPA contain information about the navigational affordances of scenes; that is, information about where one can and cannot move within the immediate environment. We hypothesized that this affordance information could be extracted using a set of purely feedforward computations. To test this idea, we examined a deep CNN with a feedforward architecture that had been previously trained for scene classification. We found that responses in the CNN to scene images were highly predictive of fMRI responses in the OPA. Moreover the CNN accounted for the portion of OPA variance relating to the navigational affordances of scenes. The CNN could thus serve as an image-computable candidate model of affordance-related responses in the OPA. We then ran a series of in silico experiments on this model to gain insights into its internal operations. These analyses showed that the computation of affordance-related features relied heavily on visual information at high-spatial frequencies and cardinal orientations, both of which have previously been identified as low-level stimulus preferences of scene-selective visual cortex. These computations also exhibited a strong preference for information in the lower visual field, which is consistent with known retinotopic biases in the OPA. Visualizations of feature selectivity within the CNN suggested that affordance-based responses encoded features that define the layout of the spatial environment, such as boundary-defining junctions and large extended surfaces. Together, these results map the sensory functions of the OPA onto a fully quantitative model that provides insights into its visual computations. More broadly, they advance integrative techniques for understanding visual cortex across multiple level of analysis: from the identification of cortical sensory functions to the modeling of their underlying algorithms.

SUBMITTER: Bonner MF

PROVIDER: S-EPMC5933806 | biostudies-literature | 2018 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Computational mechanisms underlying cortical responses to the affordance properties of visual scenes.

Bonner Michael F MF Epstein Russell A RA

PLoS computational biology 20180423 4

Biologically inspired deep convolutional neural networks (CNNs), trained for computer vision tasks, have been found to predict cortical responses with remarkable accuracy. However, the internal operations of these models remain poorly understood, and the factors that account for their success are unknown. Here we develop a set of techniques for using CNNs to gain insights into the computational mechanisms underlying cortical responses. We focused on responses in the occipital place area (OPA), a ...[more]

PMID: 29684011

Similar Datasets

Project description:Hemianopia patients have lost vision from the contralateral hemifield, but make behavioural adjustments to compensate for this field loss. As a result, their visual performance and behaviour contrast with those of hemineglect patients who fail to attend to objects contralateral to their lesion. These conditions differ in their ocular fixations and perceptual judgments. During visual search, hemianopic patients make more fixations in contralesional space while hemineglect patients make fewer. During line bisection, hemianopic patients fixate the contralesional line segment more and make a small contralesional bisection error, while hemineglect patients make few contralesional fixations and a larger ipsilesional bisection error. Hence, there is an attentional failure for contralesional space in hemineglect but a compensatory adaptation to attend more to the blind side in hemianopia. A challenge for models of visual attentional processes is to show how compensation is achieved in hemianopia, and why such processes are hindered or inaccessible in hemineglect. We used a neurophysiology-derived computational model to examine possible cortical compensatory processes in simulated hemianopia from a V1 lesion and compared results with those obtained with the same processes under conditions of simulated hemineglect from a parietal lesion. A spatial compensatory bias to increase attention contralesionally replicated hemianopic scanning patterns during visual search but not during line bisection. To reproduce the latter required a second process, an extrastriate lateral connectivity facilitating form completion into the blind field: this allowed accurate placement of fixations on contralesional stimuli and reproduced fixation patterns and the contralesional bisection error of hemianopia. Neither of these two cortical compensatory processes was effective in ameliorating the ipsilesional bias in the hemineglect model. Our results replicate normal and pathological patterns of visual scanning, line bisection, and differences between hemianopia and hemineglect, and may explain why compensatory processes that counter the effects of hemianopia are ineffective in hemineglect.

Dataset Information

Computational mechanisms underlying cortical responses to the affordance properties of visual scenes.

Publications

Computational mechanisms underlying cortical responses to the affordance properties of visual scenes.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets