Dataset Information

Neural correlates of forward planning in a spatial decision task in humans.

ABSTRACT: Although reinforcement learning (RL) theories have been influential in characterizing the mechanisms for reward-guided choice in the brain, the predominant temporal difference (TD) algorithm cannot explain many flexible or goal-directed actions that have been demonstrated behaviorally. We investigate such actions by contrasting an RL algorithm that is model based, in that it relies on learning a map or model of the task and planning within it, to traditional model-free TD learning. To distinguish these approaches in humans, we used functional magnetic resonance imaging in a continuous spatial navigation task, in which frequent changes to the layout of the maze forced subjects continually to relearn their favored routes, thereby exposing the RL mechanisms used. We sought evidence for the neural substrates of such mechanisms by comparing choice behavior and blood oxygen level-dependent (BOLD) signals to decision variables extracted from simulations of either algorithm. Both choices and value-related BOLD signals in striatum, although most often associated with TD learning, were better explained by the model-based theory. Furthermore, predecessor quantities for the model-based value computation were correlated with BOLD signals in the medial temporal lobe and frontal cortex. These results point to a significant extension of both the computational and anatomical substrates for RL in the brain.

SUBMITTER: Simon DA

PROVIDER: S-EPMC3108440 | biostudies-other | 2011 Apr

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Neural correlates of forward planning in a spatial decision task in humans.

Simon Dylan Alexander DA Daw Nathaniel D ND

The Journal of neuroscience : the official journal of the Society for Neuroscience 20110401 14

Although reinforcement learning (RL) theories have been influential in characterizing the mechanisms for reward-guided choice in the brain, the predominant temporal difference (TD) algorithm cannot explain many flexible or goal-directed actions that have been demonstrated behaviorally. We investigate such actions by contrasting an RL algorithm that is model based, in that it relies on learning a map or model of the task and planning within it, to traditional model-free TD learning. To distinguis ...[more]

PMID: 21471389

Similar Datasets

Project description:IntroductionPeople exhibit a strong attachment to possessions, observed in behavioral economics through loss aversion using new items in the Endowment or IKEA effects and in clinical psychology through pathological trouble discarding domestic items in Hoarding Disorder. These fields rarely intersect, but both document a reticence to relinquish a possessed item, even at a cost, which is associated with feelings of loss but can include enhanced positive states as well.MethodsTo demonstrate the shared properties of these loss-related ownership effects, we developed the Pretzel Decorating Task (PDT), which concurrently measures overvaluation of one's own over others' items and feelings of loss associated with losing a possession, alongside enhanced positive appraisals of one's items and an effort to save them. The PDT was piloted with 31 participants who decorated pretzels and responded to their own or others' items during functional neuroimaging (fMRI). Participants observed one item per trial (self or other) and could work to save it (high or low probability loss) before learning the fate of the item (trashed or saved). Finally, participants rated items and completed hoarding tendency scales.ResultsThe hypotheses were supported, as even non-clinical participants overvalued, viewed as nicer, feared losing, and worked harder to save their items over others'-a response that correlated with hoarding tendencies and motor-motivational brain activation. Our region of interest in the nucleus accumbens (NAcc) was engaged when viewing one's own items to the extent that people worked harder to save them and was more active when their items were saved when they felt emotionally attached to possessions in real life. When their items were trashed, NAcc activity negatively correlated with trouble discarding and emotional attachments to possessions. Right anterior insula was more active when working to save one's own over others' items. Extensive motor-motivational areas were engaged when working to save one's own over others' items, including cerebellum, primary motor and somatosensory regions, and retrosplenial/parahippocampal regions-even after controlling for tapping.DiscussionOur attachments to items are emotional, continuous across typical and pathological populations, and drive us to save possessions that we value.

Dataset Information

Neural correlates of forward planning in a spatial decision task in humans.

Publications

Neural correlates of forward planning in a spatial decision task in humans.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets