Dataset Information

Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis.

ABSTRACT: While the neurobiology of simple and habitual choices is relatively well known, our current understanding of goal-directed choices and planning in the brain is still limited. Theoretical work suggests that goal-directed computations can be productively associated to model-based (reinforcement learning) computations, yet a detailed mapping between computational processes and neuronal circuits remains to be fully established. Here we report a computational analysis that aligns Bayesian nonparametrics and model-based reinforcement learning (MB-RL) to the functioning of the hippocampus (HC) and the ventral striatum (vStr)-a neuronal circuit that increasingly recognized to be an appropriate model system to understand goal-directed (spatial) decisions and planning mechanisms in the brain. We test the MB-RL agent in a contextual conditioning task that depends on intact hippocampus and ventral striatal (shell) function and show that it solves the task while showing key behavioral and neuronal signatures of the HC-vStr circuit. Our simulations also explore the benefits of biological forms of look-ahead prediction (forward sweeps) during both learning and control. This article thus contributes to fill the gap between our current understanding of computational algorithms and biological realizations of (model-based) reinforcement learning.

SUBMITTER: Stoianov IP

PROVIDER: S-EPMC6160242 | biostudies-literature | 2018 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis.

Stoianov Ivilin Peev IP Pennartz Cyriel M A CMA Lansink Carien S CS Pezzulo Giovani G

PLoS computational biology 20180917 9

While the neurobiology of simple and habitual choices is relatively well known, our current understanding of goal-directed choices and planning in the brain is still limited. Theoretical work suggests that goal-directed computations can be productively associated to model-based (reinforcement learning) computations, yet a detailed mapping between computational processes and neuronal circuits remains to be fully established. Here we report a computational analysis that aligns Bayesian nonparametr ...[more]

PMID: 30222746

Similar Datasets

Project description:Neurodevelopmental explanations for adolescent substance use have focused on heightened sensitivity of mesolimbic circuitry, centered on the ventral striatum (VS). Recent evidence suggests that, relative to adults, adolescents show a stronger link between reinforcement learning and episodic memory for rewarding outcomes and greater functional connectivity between the VS and hippocampus, which may reflect a heightened reward modulation of memory. However, a link between VS-hippocampal circuitry and adolescent substance use has yet to be established. Two separate studies were conducted to evaluate whether variation in VS-hippocampal resting-state functional connectivity (rs-FC) predicts subsequent adolescent substance use exposure. A pilot study (Study 1) was conducted in 19 youth recruited from a high sociodemographic risk population (N = 19; mean age = 13.3 SD = 1.4; 14 females; 47% Black Non-Hispanic, 32% White Non-Hispanic). To replicate results of Study 1, Study 2 utilized data from the National Consortium on Adolescent Neurodevelopment and Alcohol (N = 644; mean age = 16.3 SD = 2.5; 339 females; 11% Black Non-Hispanic, 11% Hispanic/Latino, 66% White Non-Hispanic). Resting-state fMRI data were collected at a baseline time point and lifetime and past year self-reported substance use was collected at a follow up visit. Regression models tested whether baseline VS-hippocampal rs-FC predicted substance use exposure at follow up, as measured by an index score reflecting the number of substance classes (e.g., alcohol, marijuana) tried and overall frequency of use. Across both studies, higher VS-hippocampal rs-FC at baseline predicted greater substance use exposure at follow up (pFWE < 0.05). These data provide the first evidence linking increased VS-hippocampal connectivity with greater adolescent substance use exposure. Results fit with the emerging idea that variation in adolescent substance use may relate to not only individual differences in mesolimbic sensitivity to reward, but also to an individuals' memory sensitivity to reward as measured by connectivity between canonical memory and reward regions.

Project description:Humans and non-human animals show great flexibility in spatial navigation, including the ability to return to specific locations based on as few as one single experience. To study spatial navigation in the laboratory, watermaze tasks, in which rats have to find a hidden platform in a pool of cloudy water surrounded by spatial cues, have long been used. Analogous tasks have been developed for human participants using virtual environments. Spatial learning in the watermaze is facilitated by the hippocampus. In particular, rapid, one-trial, allocentric place learning, as measured in the delayed-matching-to-place variant of the watermaze task, which requires rodents to learn repeatedly new locations in a familiar environment, is hippocampal dependent. In this article, we review some computational principles, embedded within a reinforcement learning framework, that utilise hippocampal spatial representations for navigation in watermaze tasks. We consider which key elements underlie their efficacy, and discuss their limitations in accounting for hippocampus-dependent navigation, both in terms of behavioural performance (i.e. how well do they reproduce behavioural measures of rapid place learning) and neurobiological realism (i.e. how well do they map to neurobiological substrates involved in rapid place learning). We discuss how an actor-critic architecture, enabling simultaneous assessment of the value of the current location and of the optimal direction to follow, can reproduce one-trial place learning performance as shown on watermaze and virtual delayed-matching-to-place tasks by rats and humans, respectively, if complemented with map-like place representations. The contribution of actor-critic mechanisms to delayed-matching-to-place performance is consistent with neurobiological findings implicating the striatum and hippocampo-striatal interaction in delayed-matching-to-place performance, given that the striatum has been associated with actor-critic mechanisms. Moreover, we illustrate that hierarchical computations embedded within an actor-critic architecture may help to account for aspects of flexible spatial navigation. The hierarchical reinforcement learning approach separates trajectory control via a temporal-difference error from goal selection via a goal prediction error and may account for flexible, trial-specific, navigation to familiar goal locations, as required in some arm-maze place memory tasks, although it does not capture one-trial learning of new goal locations, as observed in open field, including watermaze and virtual, delayed-matching-to-place tasks. Future models of one-shot learning of new goal locations, as observed on delayed-matching-to-place tasks, should incorporate hippocampal plasticity mechanisms that integrate new goal information with allocentric place representation, as such mechanisms are supported by substantial empirical evidence.

Dataset Information

Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis.

Publications

Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets