Does insufficient sleep affect how you learn from reward or punishment? Reinforcement learning after 2 nights of sleep restriction.
Ontology highlight
ABSTRACT:
SUBMITTER: Gerhardsson A
PROVIDER: S-EPMC8365707 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:In the DSM-5, psychiatric diagnoses are made based on self-reported symptoms and clinician-identified signs. Though helpful in choosing potential interventions based on the available regimens, this conceptualization of psychiatric diseases can limit basic science investigation into their underlying causes. The reward prediction error (RPE) hypothesis of dopamine neuron function posits that phasic dopamine signals encode the difference between the rewards a person expects and experiences. The computational framework from which this hypothesis was derived, temporal difference reinforcement learning (TDRL), is largely focused on reward processing rather than punishment learning. Many psychiatric disorders are characterized by aberrant behaviors, expectations, reward processing, and hypothesized dopaminergic signaling, but also characterized by suffering and the inability to change one's behavior despite negative consequences. In this review, we provide an overview of the RPE theory of phasic dopamine neuron activity and review the gains that have been made through the use of computational reinforcement learning theory as a framework for understanding changes in reward processing. The relative dearth of explicit accounts of punishment learning in computational reinforcement learning theory and its application in neuroscience is highlighted as a significant gap in current computational psychiatric research. Four disorders comprise the main focus of this review: two disorders of traditionally hypothesized hyperdopaminergic function, addiction and schizophrenia, followed by two disorders of traditionally hypothesized hypodopaminergic function, depression and post-traumatic stress disorder (PTSD). Insights gained from a reward processing based reinforcement learning framework about underlying dopaminergic mechanisms and the role of punishment learning (when available) are explored in each disorder. Concluding remarks focus on the future directions required to characterize neuropsychiatric disorders with a hypothesized cause of underlying dopaminergic transmission.
| S-EPMC9630918 | biostudies-literature
Project description:In real-world settings, learning is often characterised as intentional: learners are aware of the goal during the learning process, and the goal of learning is readily dissociable from the awareness of what is learned. Recent evidence has shown that reward and punishment (collectively referred to as valenced feedback) are important factors that influence performance during learning. Presently, however, studies investigating the impact of valenced feedback on skill learning have only considered unintentional learning, and therefore the interaction between intentionality and valenced feedback has not been systematically examined. The present study investigated how reward and punishment impact behavioural performance when participants are instructed to learn in a goal-directed fashion (i.e. intentionally) rather than unintentionally. In Experiment 1, participants performed the serial response time task with reward, punishment, or control feedback and were instructed to ignore the presence of the sequence, i.e., learn unintentionally. Experiment 2 followed the same design, but participants were instructed to intentionally learn the sequence. We found that punishment significantly benefitted performance during learning only when participants learned unintentionally, and we observed no effect of punishment when participants learned intentionally. Thus, the impact of feedback on performance may be influenced by goal of the learner.
| S-EPMC7264311 | biostudies-literature
Project description:ObjectivesAnorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior.MethodsThis study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge.ResultsAN had lower learning rates than HC following both positive and negative PE (p < .02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p < .001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome.ConclusionsThis is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.
| S-EPMC9148374 | biostudies-literature
Project description:Decision-making invokes two fundamental axes of control: affect or valence, spanning reward and punishment, and effect or action, spanning invigoration and inhibition. We studied the acquisition of instrumental responding in healthy human volunteers in a task in which we orthogonalized action requirements and outcome valence. Subjects were much more successful in learning active choices in rewarded conditions, and passive choices in punished conditions. Using computational reinforcement-learning models, we teased apart contributions from putatively instrumental and Pavlovian components in the generation of the observed asymmetry during learning. Moreover, using model-based fMRI, we showed that BOLD signals in striatum and substantia nigra/ventral tegmental area (SN/VTA) correlated with instrumentally learnt action values, but with opposite signs for go and no-go choices. Finally, we showed that successful instrumental learning depends on engagement of bilateral inferior frontal gyrus. Our behavioral and computational data showed that instrumental learning is contingent on overcoming inherent and plastic Pavlovian biases, while our neuronal data showed this learning is linked to unique patterns of brain activity in regions implicated in action and inhibition respectively.
| S-EPMC3387384 | biostudies-literature
Project description:Reward and punishment are potent modulators of associative learning in instrumental and classical conditioning. However, the effect of reward and punishment on procedural learning is not known. The striatum is known to be an important locus of reward-related neural signals and part of the neural substrate of procedural learning. Here, using an implicit motor learning task, we show that reward leads to enhancement of learning in human subjects, whereas punishment is associated only with improvement in motor performance. Furthermore, these behavioral effects have distinct neural substrates with the learning effect of reward being mediated through the dorsal striatum and the performance effect of punishment through the insula. Our results suggest that reward and punishment engage separate motivational systems with distinctive behavioral effects and neural substrates.
| S-EPMC2765863 | biostudies-other
Project description:Suicide rates are high in old age, and the contribution of cognitive risk factors remains poorly understood. Suicide may be viewed as an outcome of an altered decision process. The authors hypothesized that impairment in reward/punishment-based learning, a component of affective decision making, is associated with attempted suicide in late-life depression. They expected that suicide attempters would discount past reward/punishment history, focusing excessively on the most recent rewards and punishments. The authors further hypothesized that this impairment could be dissociated from executive abilities, such as forward planning.The authors assessed reward/punishment-based learning using the probabilistic reversal learning task in 65 individuals age 60 and older: suicide attempters, suicide ideators, nonsuicidal depressed elderly, and nondepressed comparison subjects. The authors used a reinforcement learning computational model to decompose reward/punishment processing over time. The Stockings of Cambridge test served as a control measure of executive function.Suicide attempters but not suicide ideators showed impaired probabilistic reversal learning compared to both nonsuicidal depressed elderly and nondepressed comparison subjects, after controlling for effects of education, global cognitive function, and substance use. Model-based analyses revealed that suicide attempters discounted previous history to a higher degree relative to comparison subjects, basing their choice largely on reward/punishment received on the last trial. Groups did not differ in their performance on the Stockings of Cambridge test.Older suicide attempters display impaired reward/punishment-based learning. The authors propose a hypothesis that older suicide attempters make overly present-focused decisions, ignoring past experiences. Modification of this "myopia for the past" may have therapeutic potential.
| S-EPMC3020386 | biostudies-literature
Project description:To examine how stress affects cognitive functioning, individual differences in trait vulnerability (punishment sensitivity) and state reactivity (negative affect) to social evaluative threat were examined during concurrent reinforcement learning. Lower trait-level punishment sensitivity predicted better reward learning and poorer punishment learning; the opposite pattern was found in more punishment sensitive individuals. Increasing state-level negative affect was directly related to punishment learning accuracy in highly punishment sensitive individuals, but these measures were inversely related in less sensitive individuals. Combined electrophysiological measurement, performance accuracy and computational estimations of learning parameters suggest that trait and state vulnerability to stress alter cortico-striatal functioning during reinforcement learning, possibly mediated via medio-frontal cortical systems.
| S-EPMC3110431 | biostudies-literature
Project description:STUDY OBJECTIVES:Sleep disturbances increase vulnerability for depression, but the mechanisms underlying this relationship are not well known. We investigated the effects of experimental sleep disruption on response bias (RB), a measure of reward learning previously linked to depression, and the moderating role of positive affect responses. METHODS:Participants (N = 42) were healthy adults enrolled in a within-subject crossover sleep disruption experiment that incorporated one night of uninterrupted sleep (US) and one night of forced awakenings (FA) in random order. On the day following each experimental sleep night, participants completed a probabilistic reward task to assess RB, and the Positive and Negative Affect Schedule-X. Participants were subgrouped according to positive affect responses: Preserved Positive Affect (i.e. positive affect scores maintained or increased; n = 15) or Reduced Positive Affect (i.e. positive affect scores decreased; n = 27) following FA. RESULTS:Contrary to our hypotheses, across participants, RB did not significantly differ between the US and FA sleep conditions (p = .67). However, the effect of sleep condition on RB was moderated by positive affect response (p = .01); those with preserved positive affect showed heightened RB following FA, whereas those with reduced positive affect showed diminished RB following FA. Changes in negative affect between US and FA did not moderate RB. CONCLUSION:The inability to preserve positive affect through periods of sleep disruption may be a marker of diminished reward learning capability. Understanding how sleep disruption impacts positive affect responses and reward learning identifies a pathway by which sleep disturbances may confer risk for depression.
| S-EPMC6519913 | biostudies-literature
Project description:Managing multiple goals is essential to adaptation, yet we are only beginning to understand computations by which we navigate the resource demands entailed in so doing. Here, we sought to elucidate how humans balance reward seeking and punishment avoidance goals, and relate this to variation in its expression within anxious individuals. To do so, we developed a novel multigoal pursuit task that includes trial-specific instructed goals to either pursue reward (without risk of punishment) or avoid punishment (without the opportunity for reward). We constructed a computational model of multigoal pursuit to quantify the degree to which participants could disengage from the pursuit goals when instructed to, as well as devote less model-based resources toward goals that were less abundant. In general, participants (n = 192) were less flexible in avoiding punishment than in pursuing reward. Thus, when instructed to pursue reward, participants often persisted in avoiding features that had previously been associated with punishment, even though at decision time these features were unambiguously benign. In a similar vein, participants showed no significant downregulation of avoidance when punishment avoidance goals were less abundant in the task. Importantly, we show preliminary evidence that individuals with chronic worry may have difficulty disengaging from punishment avoidance when instructed to seek reward. Taken together, the findings demonstrate that people avoid punishment less flexibly than they pursue reward. Future studies should test in larger samples whether a difficulty to disengage from punishment avoidance contributes to chronic worry.
| S-EPMC8912924 | biostudies-literature
Project description:Perception is an inferential process, which becomes immediately evident when sensory information is conflicting or ambiguous and thus allows for more than one perceptual interpretation. Thinking the idea of perception as inference through to the end results in a blurring of boundaries between perception and action selection, as perceptual inference implies the construction of a percept as an active process. Here we therefore wondered whether perception shares a key characteristic of action selection, namely that it is shaped by reinforcement learning. In two behavioral experiments, we used binocular rivalry to examine whether perceptual inference can be influenced by the association of perceptual outcomes with reward or punishment, respectively, in analogy to instrumental conditioning. Binocular rivalry was evoked by two orthogonal grating stimuli presented to the two eyes, resulting in perceptual alternations between the two gratings. Perception was tracked indirectly and objectively through a target detection task, which allowed us to preclude potential reporting biases. Monetary reward or punishments were given repeatedly during perception of only one of the two rivaling stimuli. We found an increase in dominance durations for the percept associated with reward, relative to the non-rewarded percept. In contrast, punishment led to an increase of the non-punished compared to a relative decrease of the punished percept. Our results show that perception shares key characteristics with action selection, in that it is influenced by reward and punishment in opposite directions, thus narrowing the gap between the conceptually separated domains of perception and action selection. We conclude that perceptual inference is an adaptive process that is shaped by its consequences.
| S-EPMC4253824 | biostudies-literature