Dataset Information

Testing the reward prediction error hypothesis with an axiomatic model.

ABSTRACT: Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

SUBMITTER: Rutledge RB

PROVIDER: S-EPMC2957369 | biostudies-literature | 2010 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Testing the reward prediction error hypothesis with an axiomatic model.

Rutledge Robb B RB Dean Mark M Caplin Andrew A Glimcher Paul W PW

The Journal of neuroscience : the official journal of the Society for Neuroscience 20101001 40

Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neu ...[more]

PMID: 20926678

Similar Datasets

Project description:AimsTo advance translational studies of the role of reward prediction error (PE) in alcohol use disorder, the present study sought to develop and conduct an initial test of an alcohol-specific PE task paradigm using functional magnetic resonance imaging in humans.MethodsAlcohol dependent or social drinkers received small tastes of their preferred alcohol beverage or control beverage, with preceding visual cues indicating whether alcohol (or water) would be delivered. To assess both positive and negative PE signals, expectancies were systematically violated in both positive (i.e. expecting water and receiving alcohol) and negative (i.e. expecting alcohol and receiving water) directions. Exploratory trial-by-trial analyses were conducted to explore temporal fluctuations of activation within a priori-defined regions of interest that have been implicated in cue reactivity and PE processing.ResultsAcross the entire sample of participants, positive PE-related brain activation was found in a large cluster comprised of frontal lobe regions, as well as insular cortex, and motor/sensory cortices. Compared to social drinking subjects, alcohol dependent subjects had greater positive PE-related brain activity in left superior parietal lobule, lateral occipital cortex and postcentral gyrus. Exploratory trial-by-trial analyses indicated differences in activation specific to type of taste, mostly at earlier trials.ConclusionsThis task-development oriented pilot study found that PE signaling may not be detected in expected brain regions when image analyses average across all PE trials of the task. Rather, a trial-by-trial analysis approach may help detect sparse, temporally distinct PE signaling in expected reward processing regions.Short summaryThis fMRI study of reward prediction error found greater positive prediction error-related activity (i.e. expecting water taste, receiving alcohol taste) in alcohol dependent individuals relative to social drinkers in parietal and occipital cortices. Trial-by-trial analyses may be able to better detect sparse prediction error signaling in expected reward processing regions.

Dataset Information

Testing the reward prediction error hypothesis with an axiomatic model.

Publications

Testing the reward prediction error hypothesis with an axiomatic model.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets