Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Hedging your bets by learning reward correlations in the human brain.

ABSTRACT: Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling.

SUBMITTER: Wunderlich K

PROVIDER: S-EPMC3183226 | biostudies-literature | 2011 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Hedging your bets by learning reward correlations in the human brain.

Wunderlich Klaus K Symmonds Mkael M Bossaerts Peter P Dolan Raymond J RJ

Neuron 20110921 6

Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome ...[more]

PMID: 21943609

Similar Datasets

Two spatiotemporally distinct value systems shape reward-based learning in the human brain.

Project description:Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions. Yet, the neural underpinnings of the value systems that encode different decision-outcomes remain elusive. Here coupling single-trial electroencephalography with simultaneously acquired functional magnetic resonance imaging, we uncover the spatiotemporal dynamics of two separate but interacting value systems encoding decision-outcomes. Consistent with a role in regulating alertness and switching behaviours, an early system is activated only by negative outcomes and engages arousal-related and motor-preparatory brain structures. Consistent with a role in reward-based learning, a later system differentially suppresses or activates regions of the human reward network in response to negative and positive outcomes, respectively. Following negative outcomes, the early system interacts and downregulates the late system, through a thalamic interaction with the ventral striatum. Critically, the strength of this coupling predicts participants' switching behaviour and avoidance learning, directly implicating the thalamostriatal pathway in reward-based learning.

| S-EPMC4569710 | biostudies-literature

Neural structure mapping in human probabilistic reward learning.

Project description:Humans can learn abstract concepts that describe invariances over relational patterns in data. One such concept, known as magnitude, allows stimuli to be compactly represented on a single dimension (i.e. on a mental line). Here, we measured representations of magnitude in humans by recording neural signals whilst they viewed symbolic numbers. During a subsequent reward-guided learning task, the neural patterns elicited by novel complex visual images reflected their payout probability in a way that suggested they were encoded onto the same mental number line, with 'bad' bandits sharing neural representation with 'small' numbers and 'good' bandits with 'large' numbers. Using neural network simulations, we provide a mechanistic model that explains our findings and shows how structural alignment can promote transfer learning. Our findings suggest that in humans, learning about reward probability is accompanied by structural alignment of value representations with neural codes for the abstract concept of magnitude.

| S-EPMC6405242 | biostudies-literature

Psychedelics Reopen the Social Reward Learning Critical Period

Project description:Psychedelics are a broad class of drugs defined by their ability to induce an altered state of consciousness. These drugs have been used for millennia in both spiritual and medicinal contexts, and a number of recent clinical successes have spurred a renewed interest in developing psychedelic therapies. Nevertheless, a unifying mechanism that can account for these shared phenomenological and therapeutic properties remains unknown. Here we demonstrate in mice that the ability to reopen the social reward learning critical period is a shared property across psychedelics. Interestingly, the time course of critical period reopening is proportional to the duration of acute subjective effects reported in humans. Furthermore, the ability to reinstate social reward learning in adulthood is paralleled by metaplastic restoration of oxytocin mediated long-term depression (OT-LTD) in the Nucleus Accumbens (NAc). Finally, identification of differentially expressed genes in the ‘open state’ versus ‘closed state’, provides evidence that reorganization of the extracellular matrix (ECM) is a common downstream mechanism underlying psychedelic mediated critical period reopening. Together these results have significant implications for the implementation of psychedelics in clinical practice, as well as the design of novel compounds for the treatment of neuropsychiatric disease.

2023-06-14 | GSE230679 | GEO

Signals of anticipation of reward and of mean reward rates in the human brain.

Project description:Theoretical models of dopamine function stemming from reinforcement learning theory have emphasized the importance of prediction errors, which signal changes in the expectation of impending rewards. Much less is known about the effects of mean reward rates, which may be of motivational significance due to their role in computing the optimal effort put into exploiting reward opportunities. Here, we used a reinforcement learning model to design three functional neuroimaging studies and disentangle the effects of changes in reward expectations and mean reward rates, showing recruitment of specific regions in the brainstem regardless of prediction errors. While changes in reward expectations activated ventral striatal areas as in previous studies, mean reward rates preferentially modulated the substantia nigra/ventral tegmental area, deep layers of the superior colliculi, and a posterior pontomesencephalic region. These brainstem structures may work together to set motivation and attentional efforts levels according to perceived reward opportunities.

| S-EPMC7062891 | biostudies-literature

Disentangling pleasure from incentive salience and learning signals in brain reward circuitry.

Project description:Multiple signals for reward-hedonic impact, motivation, and learned associative prediction-are funneled through brain mesocorticolimbic circuits involving the nucleus accumbens and ventral pallidum. Here, we show how the hedonic "liking" and motivation "wanting" signals for a sweet reward are distinctly modulated and tracked in this circuit separately from signals for Pavlovian predictions (learning). Animals first learned to associate a fixed sequence of Pavlovian cues with sucrose reward. Subsequent intraaccumbens microinjections of an opioid-stimulating drug increased the hedonic liking impact of sucrose in behavior and firing signals of ventral pallidum neurons, and likewise, they increased incentive salience signals in firing to the reward-proximal incentive cue (but did not alter firing signals to the learned prediction value of a reward-distal cue). Microinjection of a dopamine-stimulating drug instead enhanced only the motivation component but did not alter hedonic impact or learned prediction signals. Different dedicated neuronal subpopulations in the ventral pallidum tracked signal enhancements for hedonic impact vs. incentive salience, and a faster firing pattern also distinguished incentive signals from slower hedonic signals, even for a third overlapping population. These results reveal separate neural representations of wanting, liking, and prediction components of the same reward within the nucleus accumbens to ventral pallidum segment of mesocorticolimbic circuitry.

| S-EPMC3131314 | biostudies-literature

Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex.

Project description:Reward-guided decision-making depends on a network of brain regions. Among these are the orbitofrontal and the anterior cingulate cortex. However, it is difficult to ascertain if these areas constitute anatomical and functional unities, and how these areas correspond between monkeys and humans. To address these questions we looked at connectivity profiles of these areas using resting-state functional MRI in 38 humans and 25 macaque monkeys. We sought brain regions in the macaque that resembled 10 human areas identified with decision making and brain regions in the human that resembled six macaque areas identified with decision making. We also used diffusion-weighted MRI to delineate key human orbital and medial frontal brain regions. We identified 21 different regions, many of which could be linked to particular aspects of reward-guided learning, valuation, and decision making, and in many cases we identified areas in the macaque with similar coupling profiles.

| S-EPMC4443352 | biostudies-literature

Attentional modulation of reward processing in the human brain.

Project description:Although neural signals of reward anticipation have been studied extensively, the functional relationship between reward and attention has remained unclear: Neural signals implicated in reward processing could either reflect attentional biases towards motivationally salient stimuli, or proceed independently of attentional processes. Here, we sought to disentangle reward and attention-related neural processes by independently modulating reward value and attentional task demands in a functional magnetic resonance imaging study in healthy human participants. During presentation of a visual reward cue that indicated whether monetary reward could be obtained in a subsequent reaction time task, participants either attended to the reward cue or performed an unrelated attention-demanding task at two different levels of difficulty. In ventral striatum and ventral tegmental area, neural responses were modulated by reward anticipation irrespective of attentional demands, thus indicating attention-independent processing of reward cues. By contrast, additive effects of reward and attention were observed in visual cortex. Critically, reward-related activations in right anterior insula strongly depended on attention to the reward cue. Dynamic causal modelling revealed that the attentional modulation of reward processing in insular cortex was mediated by enhanced effective connectivity from ventral striatum to anterior insula. Our results provide evidence for distinct functional roles of the brain regions involved in the processing of reward-indicating information: While subcortical structures signal the motivational salience of reward cues even when attention is fully engaged elsewhere, reward-related responses in anterior insula depend on available attentional resources, likely reflecting the conscious evaluation of sensory information with respect to motivational value.

| S-EPMC6869517 | biostudies-literature

First Nanoparticles of a Conductor Based on the Organic Donor Molecule BETS: κ-(BETS)2FeCl4.

Project description:Nanoparticles of the molecular superconductor (BETS)2FeCl4 were obtained by the electrochemical oxidation of BETS in the presence of [(C2H5)4N]FeCl4 and an amphiphilic imine (OATM), acting as a growth controlling agent. When the reaction was carried out with a molar ratio OATM/BETS of 10, roughly spherical nanoparticles exhibiting sizes in the 10-40 nm range were observed. X-ray diffraction patterns evidenced the growth of (BETS)2FeCl4 nanoparticles with the κ-type structure. The current-voltage characteristic recorded on an individual nanoparticle aggregate was fitted with a Shockley diode model. A saturation current of 1216 pA and a threshold voltage of 0.62 V were extracted from this model. This latter value was consistent with roughly half of the energy gap of the semiconducting nano-crystalline aggregate.

| S-EPMC8398930 | biostudies-literature

Learning reward frequency over reward probability: A tale of two learning rules.

Project description:Learning about the expected value of choice alternatives associated with reward is critical for adaptive behavior. Although human choice preferences are affected by the presentation frequency of reward-related alternatives, this may not be captured by some dominant models of value learning, such as the delta rule. In this study, we examined whether reward learning is driven more by learning the probability of reward provided by each option, or how frequently each option has been rewarded, and assess how well models based on average reward (e.g. the delta model) and models based on cumulative reward (e.g. the decay model) can account for choice preferences. In a binary-outcome choice task, participants selected between pairs of options that had reward probabilities of 0.65 (A) versus 0.35 (B) or 0.75 (C) versus 0.25 (D). Crucially, during training there were twice the number of AB trials as CD trials, such that option A was associated with higher cumulative reward, while option C gave higher average reward. Participants then decided between novel combinations of options (e.g., AC). Most participants preferred option A over C, a result predicted by the Decay model, but not the Delta model. We also compared the Delta and Decay models to both more simplified as well as more complex models that assumed additional mechanisms, such as representation of uncertainty. Overall, models that assume learning about cumulative reward provided the best account of the data.

| S-EPMC6814570 | biostudies-literature

How does reward expectation influence cognition in the human brain?

Project description:The prospect of reward changes how we think and behave. We investigated how this occurs in the brain using a novel continuous performance task in which fluctuating reward expectations biased cognitive processes between competing spatial and verbal tasks. Critically, effects of reward expectancy could be distinguished from induced changes in task-related networks. Behavioral data confirm specific bias toward a reward-relevant modality. Increased reward expectation improves reaction time and accuracy in the relevant dimension while reducing sensitivity to modulations of stimuli characteristics in the irrelevant dimension. Analysis of functional magnetic resonance imaging data shows that the proximity to reward over successive trials is associated with increased activity of the medial frontal cortex regardless of the modality. However, there are modality-specific changes in brain activity in the lateral frontal, parietal, and temporal cortex. Analysis of effective connectivity suggests that reward expectancy enhances coupling in both early visual pathways and within the prefrontal cortex. These distributed changes in task-related cortical networks arise from subjects' representations of future events and likelihood of reward.

| S-EPMC3886193 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data