Unknown

Dataset Information

0

A neural model of hierarchical reinforcement learning.

ABSTRACT: We develop a novel, biologically detailed neural model of reinforcement learning (RL) processes in the brain. This model incorporates a broad range of biological features that pose challenges to neural RL, such as temporally extended action sequences, continuous environments involving unknown time delays, and noisy/imprecise computations. Most significantly, we expand the model into the realm of hierarchical reinforcement learning (HRL), which divides the RL process into a hierarchy of actions at different levels of abstraction. Here we implement all the major components of HRL in a neural model that captures a variety of known anatomical and physiological properties of the brain. We demonstrate the performance of the model in a range of different environments, in order to emphasize the aim of understanding the brain's general reinforcement learning ability. These results show that the model compares well to previous modelling work and demonstrates improved performance as a result of its hierarchical ability. We also show that the model's behaviour is consistent with available data on human hierarchical RL, and generate several novel predictions.

SUBMITTER: Rasmussen D 

PROVIDER: S-EPMC5500327 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

Json Xml
altmetric image

Publications

A neural model of hierarchical reinforcement learning.

Rasmussen Daniel D   Voelker Aaron A   Eliasmith Chris C  

PloS one 20170706 7

We develop a novel, biologically detailed neural model of reinforcement learning (RL) processes in the brain. This model incorporates a broad range of biological features that pose challenges to neural RL, such as temporally extended action sequences, continuous environments involving unknown time delays, and noisy/imprecise computations. Most significantly, we expand the model into the realm of hierarchical reinforcement learning (HRL), which divides the RL process into a hierarchy of actions a  ...[more]

Similar Datasets

Unknown A neural signature of hierarchical reinforcement learning.
| S-EPMC3145918 | biostudies-literature
Unknown Optimal Hierarchical Learning Path Design With Reinforcement Learning.
| S-EPMC7711250 | biostudies-literature
Unknown States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.
| S-EPMC2895323 | biostudies-other
Unknown Hierarchical deep reinforcement learning reveals novel mechanism of cell movement.
| S-EPMC9387359 | biostudies-literature
Unknown Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis.
| S-EPMC3278315 | biostudies-literature
Unknown A neural network model for the orbitofrontal cortex and task space acquisition during reinforcement learning.
| S-EPMC5771635 | biostudies-literature
Unknown A spiking neural network model of model-free reinforcement learning with high-dimensional sensory input and perceptual ambiguity.
| S-EPMC4347982 | biostudies-literature
Unknown Neural Correlates of Reinforcement Learning in Mid-lateral Cerebellum.
| S-EPMC8015782 | biostudies-literature
Unknown Neural computations underlying inverse reinforcement learning in the human brain.
| S-EPMC5662289 | biostudies-literature
Unknown A multiplicative reinforcement learning model capturing learning dynamics and interindividual variability in mice.
| S-EPMC3856837 | biostudies-other