Dataset Information

A spiking neural network model of model-free reinforcement learning with high-dimensional sensory input and perceptual ambiguity.

ABSTRACT: A theoretical framework of reinforcement learning plays an important role in understanding action selection in animals. Spiking neural networks provide a theoretically grounded means to test computational hypotheses on neurally plausible algorithms of reinforcement learning through numerical simulation. However, most of these models cannot handle observations which are noisy, or occurred in the past, even though these are inevitable and constraining features of learning in real environments. This class of problem is formally known as partially observable reinforcement learning (PORL) problems. It provides a generalization of reinforcement learning to partially observable domains. In addition, observations in the real world tend to be rich and high-dimensional. In this work, we use a spiking neural network model to approximate the free energy of a restricted Boltzmann machine and apply it to the solution of PORL problems with high-dimensional observations. Our spiking network model solves maze tasks with perceptually ambiguous high-dimensional observations without knowledge of the true environment. An extended model with working memory also solves history-dependent tasks. The way spiking neural networks handle PORL problems may provide a glimpse into the underlying laws of neural information processing which can only be discovered through such a top-down approach.

SUBMITTER: Nakano T

PROVIDER: S-EPMC4347982 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:The brain uses its intrinsic dynamics to actively predict observed sensory inputs, especially under perceptual ambiguity. However, it remains unclear how this inference process is neurally implemented in biasing perception of ambiguous inputs towards the predicted percepts. The process of perceptual inference can be well illustrated by the phenomenon of bistable apparent motion in the Ternus display, in which subjective perception spontaneously alternates between element motion (EM) and group motion (GM) percepts depending on whether two consecutively presented frames are grouped over time or not. The frequency of alpha-band oscillations has long been hypothesized to gate the temporal window of perceptual grouping over time. Under this hypothesis, variation in the intrinsic alpha frequency should predict perceptual outcome of the bistable Ternus display. Moreover, we hypothesize that the perception system employs this prior knowledge on intrinsic alpha frequency to resolve perceptual ambiguity, by shifting perceptual inference towards the predicted percepts. Using electroencephalography and intracranial recordings, we showed that both between and within subjects, lower prestimulus alpha frequencies (PAFs) predicted the EM percepts since the two frames fell in the same alpha cycle and got temporally integrated, while higher PAFs predicted the GM percepts since the two frames fell in different alpha cycles. Multivariate decoding analysis between the EM percepts with lower PAFs and the GM percepts with higher PAFs further revealed a representation of the subsequently reported bistable percept in the neural signals shortly before the actual appearance of the second frame. Therefore, perceptual inference, based on variation in intrinsic PAFs, biases poststimulus neural representations by inducing preactivation of the predicted percepts. In addition, enhanced prestimulus blood-oxygen-level-dependent (BOLD) signals and network dynamics in the frontoparietal network, together with reduced prestimulus alpha power, upon perceiving the EM percepts suggest that temporal grouping is an attention-demanding process.

Dataset Information

A spiking neural network model of model-free reinforcement learning with high-dimensional sensory input and perceptual ambiguity.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets