Action controls dopaminergic enhancement of reward representations.
Ontology highlight
ABSTRACT:
SUBMITTER: Guitart-Masip M
PROVIDER: S-EPMC3358848 | biostudies-other | 2012 May
REPOSITORIES: biostudies-other
Publications
Action controls dopaminergic enhancement of reward representations.
Guitart-Masip Marc M Chowdhury Rumana R Sharot Tali T Dayan Peter P Duzel Emrah E Dolan Raymond J RJ
Proceedings of the National Academy of Sciences of the United States of America 20120423 19
Dopamine is widely observed to signal anticipation of future rewards and thus thought to be a key contributor to affectively charged decision making. However, the experiments supporting this view have not dissociated rewards from the actions that lead to, or are occasioned by, them. Here, we manipulated dopamine pharmacologically and examined the effect on a task that explicitly dissociates action and reward value. We show that dopamine enhanced the neural representation of rewarding actions, wi ...[more]
Similar Datasets
Project description:ObjectiveMajor depressive disorder is characterized by reduced reward-related striatal activation and dysfunctional reward learning, putatively reflecting decreased dopaminergic signaling. The goal of this study was to test whether a pharmacological challenge designed to facilitate dopaminergic transmission can enhance striatal responses to reward and improve reward learning in depressed individuals.MethodIn a double-blind placebo-controlled design, 46 unmedicated depressed participants and 43 healthy control participants were randomly assigned to receive either placebo or a single low dose (50 mg) of the D2/D3 receptor antagonist amisulpride, which is believed to increase dopamine signaling through presynaptic autoreceptor blockade. To investigate the effects of increased dopaminergic transmission on reward-related striatal function and behavior, a monetary incentive delay task (in conjunction with functional MRI) and a probabilistic reward learning task were administered at absorption peaks of amisulpride.ResultsDepressed participants selected previously rewarded stimuli less frequently than did control participants, indicating reduced reward learning, but this effect was not modulated by amisulpride. Relative to depressed participants receiving placebo (and control participants receiving amisulpride), depressed participants receiving amisulpride exhibited increased striatal activation and potentiated corticostriatal functional connectivity between the nucleus accumbens and the midcingulate cortex in response to monetary rewards. Stronger corticostriatal connectivity in response to rewards predicted better reward learning among depressed individuals receiving amisulpride as well as among control participants receiving placebo.ConclusionsAcute enhancement of dopaminergic transmission potentiated reward-related striatal activation and corticostriatal functional connectivity in depressed individuals but had no behavioral effects. Taken together, the results suggest that targeted pharmacological treatments may normalize neural correlates of reward processing in depression; despite such acute effects on neural function, behavioral modification may require more chronic exposure. This is consistent with previous reports that antidepressant effects of amisulpride in depression emerged after sustained administration.
| S-EPMC5378658 | biostudies-literature
Project description:The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference (TD) reinforcement learning. The TD framework predicts that some neuronal elements should represent the reward prediction error (RPE), which means they signal the difference between the expected future rewards and the actual rewards. The prominence of the TD theory arises from the observation that firing properties of dopaminergic neurons in the ventral tegmental area appear similar to those of RPE model-neurons in TD learning. Previous implementations of TD learning assume a fixed temporal basis for each stimulus that might eventually predict a reward. Here we show that such a fixed temporal basis is implausible and that certain predictions of TD learning are inconsistent with experiments. We propose instead an alternative theoretical framework, coined FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, feature specific representations of time are learned, allowing for neural representations of stimuli to adjust their timing and relation to rewards in an online manner. In FLEX dopamine acts as an instructive signal which helps build temporal models of the environment. FLEX is a general theoretical framework that has many possible biophysical implementations. In order to show that FLEX is a feasible approach, we present a specific biophysically plausible model which implements the principles of FLEX. We show that this implementation can account for various reinforcement learning paradigms, and that its results and predictions are consistent with a preponderance of both existing and reanalyzed experimental data.
| S-EPMC10543312 | biostudies-literature
Project description:BackgroundTheoretical models have emphasized systems-level abnormalities in major depressive disorder (MDD). For unbiased yet rigorous evaluations of pathophysiological mechanisms underlying MDD, it is critically important to develop data-driven approaches that harness whole-brain data to classify MDD and evaluate possible normalizing effects of targeted interventions. Here, using an experimental therapeutics approach coupled with machine learning, we investigated the effect of a pharmacological challenge aiming to enhance dopaminergic signaling on whole-brain response to reward-related stimuli in MDD.MethodsUsing a double-blind, placebo-controlled design, we analyzed functional magnetic resonance imaging data from 31 unmedicated MDD participants receiving a single dose of 50 mg amisulpride (MDDAmisulpride), 26 MDD participants receiving placebo (MDDPlacebo), and 28 healthy control subjects receiving placebo (HCPlacebo) recruited through two independent studies. An importance-guided machine learning technique for model selection was used on whole-brain functional magnetic resonance imaging data probing reward anticipation and consumption to identify features linked to MDD (MDDPlacebo vs. HCPlacebo) and dopaminergic enhancement (MDDAmisulpride vs. MDDPlacebo).ResultsHighly predictive classification models emerged that distinguished MDDPlacebo from HCPlacebo (area under the curve = 0.87) and MDDPlacebo from MDDAmisulpride (area under the curve = 0.89). Although reward-related striatal activation and connectivity were among the most predictive features, the best truncated models based on whole-brain features were significantly better relative to models trained using striatal features only.ConclusionsResults indicate that in MDD, enhanced dopaminergic signaling restores abnormal activation and connectivity in a widespread network of regions. These findings provide new insights into the pathophysiology of MDD and pharmacological mechanism of antidepressants at the system level in addressing reward processing deficits among depressed individuals.
| S-EPMC7010544 | biostudies-literature
Project description:Neurons comprising nigrostriatal system play important roles in action selection. However, it remains unclear how this system integrates recent outcome information with current action (movement) and outcome (reward or no reward) information to achieve appropriate subsequent action. We examined how neuronal activity of substantia nigra pars compacta (SNc) and dorsal striatum reflects the level of reward expectation from recent outcomes in rats performing a reward-based choice task. Movement-related activity of direct and indirect pathway striatal projection neurons (dSPNs and iSPNs, respectively) were enhanced by reward expectation, similarly to the SNc dopaminergic neurons, in both medial and lateral nigrostriatal projections. Given the classical basal ganglia model wherein dopamine stimulates dSPNs and suppresses iSPNs through distinct dopamine receptors, dopamine might not be the primary driver of iSPN activity increasing following higher reward expectation. In contrast, outcome-related activity was affected by reward expectation in line with the classical model and reinforcement learning theory, suggesting purposive effects of reward expectation.
| S-EPMC10482957 | biostudies-literature
Project description:Although both males and females become addicted to cocaine, females transition to addiction faster and experience greater difficulties remaining abstinent. We demonstrate an oestrous cycle-dependent mechanism controlling increased cocaine reward in females. During oestrus, ventral tegmental area (VTA) dopamine neuron activity is enhanced and drives post translational modifications at the dopamine transporter (DAT) to increase the ability of cocaine to inhibit its function, an effect mediated by estradiol. Female mice conditioned to associate cocaine with contextual cues during oestrus have enhanced mesolimbic responses to these cues in the absence of drug. Using chemogenetic approaches, we increase VTA activity to mechanistically link oestrous cycle-dependent enhancement of VTA firing to enhanced cocaine affinity at DAT and subsequent reward processing. These data have implications for sexual dimorphism in addiction vulnerability and define a mechanism by which cellular activity results in protein alterations that contribute to dysfunctional learning and reward processing.
| S-EPMC5234081 | biostudies-other
Project description:Reward representation in ventral striatum is boosted by perceptual novelty, although the mechanism of this effect remains elusive. Animal studies indicate a functional loop (Lisman and Grace, 2005) that includes hippocampus, ventral striatum, and midbrain as being important in regulating salience attribution within the context of novel stimuli. According to this model, reward responses in ventral striatum or midbrain should be enhanced in the context of novelty even if reward and novelty constitute unrelated, independent events. Using fMRI, we show that trials with reward-predictive cues and subsequent outcomes elicit higher responses in the striatum if preceded by an unrelated novel picture, indicating that reward representation is enhanced in the context of novelty. Notably, this effect was observed solely when reward occurrence, and hence reward-related salience, was low. These findings support a view that contextual novelty enhances neural responses underlying reward representation in the striatum and concur with the effects of novelty processing as predicted by the model of Lisman and Grace (2005).
| S-EPMC2838369 | biostudies-literature
Project description:Midbrain dopamine (DA) neurons encode both reward- and movement-related events and are implicated in disorders of reward processing as well as movement. Consequently, disentangling the contribution of DA neurons in reinforcing versus generating movements is challenging and has led to lasting controversy. In this study, we dissociated these functions by parametrically varying the timing of optogenetic manipulations in a Pavlovian conditioning task and examining the influence on anticipatory licking before reward delivery. Inhibiting both ventral tegmental area and substantia nigra pars compacta DA neurons in the post-reward period had a significantly greater behavioral effect than inhibition in the pre-reward period of the task. Furthermore, the contribution of DA neurons to behavior decreased linearly as a function of elapsed time after reward. Together, the results indicate a temporally restricted role of DA neurons primarily related to reinforcing stimulus-reward associations and suggest that directly generating movements is a comparatively less important function.
| S-EPMC7007363 | biostudies-literature
Project description:Resource-seeking behaviours are ordinarily constrained by physiological needs and threats of danger, and the loss of these controls is associated with pathological reward seeking1. Although dysfunction of the dopaminergic valuation system of the brain is known to contribute towards unconstrained reward seeking2,3, the underlying reasons for this behaviour are unclear. Here we describe dopaminergic neural mechanisms that produce reward seeking despite adverse consequences in Drosophila melanogaster. Odours paired with optogenetic activation of a defined subset of reward-encoding dopaminergic neurons become cues that starved flies seek while neglecting food and enduring electric shock punishment. Unconstrained seeking of reward is not observed after learning with sugar or synthetic engagement of other dopaminergic neuron populations. Antagonism between reward-encoding and punishment-encoding dopaminergic neurons accounts for the perseverance of reward seeking despite punishment, whereas synthetic engagement of the reward-encoding dopaminergic neurons also impairs the ordinary need-dependent dopaminergic valuation of available food. Connectome analyses reveal that the population of reward-encoding dopaminergic neurons receives highly heterogeneous input, consistent with parallel representation of diverse rewards, and recordings demonstrate state-specific gating and satiety-related signals. We propose that a similar dopaminergic valuation system dysfunction is likely to contribute to maladaptive seeking of rewards by mammals.
| S-EPMC10632144 | biostudies-literature
Project description:In computer science, reinforcement learning is a powerful framework with which artificial agents can learn to maximize their performance for any given Markov decision process (MDP). Advances over the last decade, in combination with deep neural networks, have enjoyed performance advantages over humans in many difficult task settings. However, such frameworks perform far less favorably when evaluated in their ability to generalize or transfer representations across different tasks. Existing algorithms that facilitate transfer typically are limited to cases in which the transition function or the optimal policy is portable to new contexts, but achieving "deep transfer" characteristic of human behavior has been elusive. Such transfer typically requires discovery of abstractions that permit analogical reuse of previously learned representations to superficially distinct tasks. Here, we demonstrate that abstractions that minimize error in predictions of reward outcomes generalize across tasks with different transition and reward functions. Such reward-predictive representations compress the state space of a task into a lower dimensional representation by combining states that are equivalent in terms of both the transition and reward functions. Because only state equivalences are considered, the resulting state representation is not tied to the transition and reward functions themselves and thus generalizes across tasks with different reward and transition functions. These results contrast with those using abstractions that myopically maximize reward in any given MDP and motivate further experiments in humans and animals to investigate if neural and cognitive systems involved in state representation perform abstractions that facilitate such equivalence relations.
| S-EPMC7591094 | biostudies-literature
Project description:BackgroundAdolescent internalizing symptoms and trauma exposure have been linked with altered reward learning processes and decreased ventral striatal responses to rewarding cues. Recent computational work on decision-making highlights an important role for prospective representations of the imagined outcomes of different choices. This study tested whether internalizing symptoms and trauma exposure among youth impact the generation of prospective reward representations during decision-making and potentially mediate altered behavioral strategies during reward learning.MethodsSixty-one adolescent females with varying exposure to interpersonal violence exposure (n = 31 with histories of physical or sexual assault) and severity of internalizing symptoms completed a social reward learning task during fMRI. Multivariate pattern analyses (MVPA) were used to decode neural reward representations at the time of choice.ResultsMVPA demonstrated that rewarding outcomes could accurately be decoded within several large-scale distributed networks (e.g. frontoparietal and striatum networks), that these reward representations were reactivated prospectively at the time of choice in proportion to the expected probability of receiving reward, and that youth with behavioral strategies that favored exploiting high reward options demonstrated greater prospective generation of reward representations. Youth internalizing symptoms, but not trauma exposure characteristics, were negatively associated with both the behavioral strategy of exploiting high reward options as well as the prospective generation of reward representations in the striatum.ConclusionsThese data suggest diminished prospective mental simulation of reward as a mechanism of altered reward learning strategies among youth with internalizing symptoms.
| S-EPMC10600826 | biostudies-literature