Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved.
Ontology highlight
ABSTRACT:
SUBMITTER: Cutler J
PROVIDER: S-EPMC8295324 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ADHD is associated with altered dopamine regulated reinforcement learning on prediction errors. Despite evidence of categorically altered error processing in ADHD, neuroimaging advances have largely investigated models of normal reinforcement learning in greater detail. Further, although reinforcement leaning critically relies on ventral striatum exerting error magnitude related thresholding influences on substantia nigra (SN) and dorsal striatum, these thresholding influences have never been identified with neuroimaging. To identify such thresholding influences, we propose that error magnitude related activities must first be separated from opposite activities in overlapping neural regions during error detection. Here we separate error detection from magnitude related adjustment (post-error slowing) during inhibition errors in the stop signal task in typically developing (TD) and ADHD adolescents using fMRI. In TD, we predicted that: 1) deactivation of dorsal striatum on error detection interrupts ongoing processing, and should be proportional to right frontoparietal response phase activity that has been observed in the SST; 2) deactivation of ventral striatum on post-error slowing exerts thresholding influences on, and should be proportional to activity in dorsal striatum. In ADHD, we predicted that ventral striatum would instead correlate with heightened amygdala responses to errors. We found deactivation of dorsal striatum on error detection correlated with response-phase activity in both groups. In TD, post-error slowing deactivation of ventral striatum correlated with activation of dorsal striatum. In ADHD, ventral striatum correlated with heightened amygdala activity. Further, heightened activities in locus coeruleus (norepinephrine), raphe nucleus (serotonin) and medial septal nuclei (acetylcholine), which all compete for control of DA, and are altered in ADHD, exhibited altered correlations with SN. All correlations in TD were replicated in healthy adults. Results in TD are consistent with dopamine regulated reinforcement learning on post-error slowing. In ADHD, results are consistent with heightened activities in the amygdala and non-dopaminergic neurotransmitter nuclei preventing reinforcement learning.
| S-EPMC6382150 | biostudies-literature
Project description:Dysfunction in the neural circuits underlying salience signaling is implicated in symptoms of psychosis and may predict conversion to a psychotic disorder in youth at clinical high risk (CHR) for psychosis. Additionally, negative symptom severity, including consummatory and anticipatory aspects of anhedonia, may predict functional outcome in individuals with schizophrenia-spectrum disorders. However, it is unclear whether anhedonia is related to the ability to attribute incentive salience to stimuli (through reinforcement learning [RL]) and whether measures of anhedonia and RL predict functional outcome in a younger, help-seeking population. We administered the Salience Attribution Test (SAT) to 33 participants who met criteria for either CHR or a recent-onset psychotic disorder and 29 help-seeking youth with nonpsychotic disorders. In the SAT, participants must identify relevant and irrelevant stimulus dimensions and be sensitive to different reinforcement probabilities for the 2 levels of the relevant dimension ("adaptive salience"). Adaptive salience attribution was positively related to both consummatory pleasure and functioning in the full sample. Analyses also revealed an indirect effect of adaptive salience on the relation between consummatory pleasure and both role (αβ = .22, 95% CI = 0.02, 0.48) and social functioning (αβ = .14, 95% CI = 0.02, 0.30). These findings suggest a distinct pathway to poor global functioning in help-seeking youth, via impaired reward sensitivity and RL.
| S-EPMC8530392 | biostudies-literature
Project description:While there is no doubt that social signals affect human reinforcement learning, there is still no consensus about how this process is computationally implemented. To address this issue, we compared three psychologically plausible hypotheses about the algorithmic implementation of imitation in reinforcement learning. The first hypothesis, decision biasing (DB), postulates that imitation consists in transiently biasing the learner's action selection without affecting their value function. According to the second hypothesis, model-based imitation (MB), the learner infers the demonstrator's value function through inverse reinforcement learning and uses it to bias action selection. Finally, according to the third hypothesis, value shaping (VS), the demonstrator's actions directly affect the learner's value function. We tested these three hypotheses in 2 experiments (N = 24 and N = 44) featuring a new variant of a social reinforcement learning task. We show through model comparison and model simulation that VS provides the best explanation of learner's behavior. Results replicated in a third independent experiment featuring a larger cohort and a different design (N = 302). In our experiments, we also manipulated the quality of the demonstrators' choices and found that learners were able to adapt their imitation rate, so that only skilled demonstrators were imitated. We proposed and tested an efficient meta-learning process to account for this effect, where imitation is regulated by the agreement between the learner and the demonstrator. In sum, our findings provide new insights and perspectives on the computational mechanisms underlying adaptive imitation in human reinforcement learning.
| S-EPMC7723279 | biostudies-literature
Project description:Changes in the stability of microtubules regulate many biological processes, but their role in memory remains unclear. Here we show that learning causes biphasic changes in the microtubule-associated network in the hippocampus. In the early phase, stathmin is dephosphorylated, enhancing its microtubule-destabilizing activity by promoting stathmin-tubulin binding, whereas in the late phase these processes are reversed leading to an increase in microtubule/KIF5-mediated localization of the GluA2 subunit of AMPA receptors at synaptic sites. A microtubule stabilizer paclitaxel decreases or increases memory when applied at the early or late phases, respectively. Stathmin mutations disrupt changes in microtubule stability, GluA2 localization, synaptic plasticity and memory. Aged wild-type mice show impairments in stathmin levels, changes in microtubule stability and GluA2 localization. Blocking GluA2 endocytosis rescues memory deficits in stathmin mutant and aged wild-type mice. These findings demonstrate a role for microtubules in memory in young adult and aged individuals.
| S-EPMC4137320 | biostudies-other
Project description:In this paper, we introduce a new type of tree-based method, reinforcement learning trees (RLT), which exhibits significantly improved performance over traditional methods such as random forests (Breiman, 2001) under high-dimensional settings. The innovations are three-fold. First, the new method implements reinforcement learning at each selection of a splitting variable during the tree construction processes. By splitting on the variable that brings the greatest future improvement in later splits, rather than choosing the one with largest marginal effect from the immediate split, the constructed tree utilizes the available samples in a more efficient way. Moreover, such an approach enables linear combination cuts at little extra computational cost. Second, we propose a variable muting procedure that progressively eliminates noise variables during the construction of each individual tree. The muting procedure also takes advantage of reinforcement learning and prevents noise variables from being considered in the search for splitting rules, so that towards terminal nodes, where the sample size is small, the splitting rules are still constructed from only strong variables. Last, we investigate asymptotic properties of the proposed method under basic assumptions and discuss rationale in general settings.
| S-EPMC4760114 | biostudies-literature
Project description:IntroductionLongitudinal Integrated Clerkships (LICs) have demonstrated benefits for students' learning and development in medical education globally. The model emphasises importance with respect to continuity and time in the workplace for learners. There is a need to explore how LICs become a viable training model for learners. An amalgamative LIC model was drawn upon to inform the design of a placement for trainee pharmacists on a hospital ward. This study sought to determine the local viability of a longitudinal placement for trainee pharmacists, using communities of practice learning theory to interpret findings.MethodsA design-based research approach informed study design. A longitudinal placement was implemented on two hospital wards for 13 weeks. Trainees (n = 3) were interviewed four times over a 14-week period. Ward staff (n = 14) were interviewed at week 14. Interviews were transcribed verbatim. Qualitative longitudinal analysis, using the trajectory approach, was conducted using abductive analysis. The coded data were organised into a framework and subthemes were created.ResultsTrainees acquired membership within the multi-disciplinary ward team over time. This led to an enriched learning experience and the trainee's professional development improved as they attained more responsibilities. This enabled them to make a greater contribution to patient care; more medication consultations occurred, and discharge times improved.DiscussionThe local viability of the longitudinal placement appears to be linked to the trainee's ability to acquire membership within the ward community of practice. Membership gave trainee's access to learning opportunities, supporting their development, and they earnt the trust of staff, leading to more responsibilities for providing patient care. Further research into developing longitudinal placements that support trainee healthcare professionals to acquire membership within communities of practice is warranted.
| S-EPMC9298918 | biostudies-literature
Project description:Reinforcement learning in complex natural environments is a challenging task because the agent should generalize from the outcomes of actions taken in one state of the world to future actions in different states of the world. The extent to which human experts find the proper level of generalization is unclear. Here we show, using the sequences of field goal attempts made by professional basketball players, that the outcome of even a single field goal attempt has a considerable effect on the rate of subsequent 3 point shot attempts, in line with standard models of reinforcement learning. However, this change in behaviour is associated with negative correlations between the outcomes of successive field goal attempts. These results indicate that despite years of experience and high motivation, professional players overgeneralize from the outcomes of their most recent actions, which leads to decreased performance.
| S-EPMC3247813 | biostudies-other
Project description:Empathy plays an important role in human social interaction. A multifaceted construct, empathy includes a prosocial motivation or intention to help others in need. Although humans are often willing to help others in need, at times (e.g., during intergroup conflict), empathic responses are diminished or absent. Research examining the cognitive mechanisms underlying prosocial tendencies has focused on the facilitating roles of perspective taking and emotion sharing but has not previously elucidated the contributions of episodic simulation and memory to facilitating prosocial intentions. Here, we investigated whether humans' ability to construct episodes by vividly imagining (episodic simulation) or remembering (episodic memory) specific events also supports a willingness to help others. Three experiments provide evidence that, when participants were presented with a situation depicting another person's plight, the act of imagining an event of helping the person or remembering a related past event of helping others increased prosocial intentions to help the present person in need, compared with various control conditions. We also report evidence suggesting that the vividness of constructed episodes--rather than simply heightened emotional reactions or degree of perspective taking--supports this effect. Our results shed light on a role that episodic simulation and memory can play in fostering empathy and begin to offer insight into the underlying mechanisms.
| S-EPMC3970486 | biostudies-literature
Project description:The amygdala is a subcortical structure implicated in both the expression of conditioned fear and social fear recognition. Social fear recognition deficits following amygdala lesions are often interpreted as reflecting perceptual deficits, or the amygdala's role in coordinating responses to threats. But these explanations fail to capture why amygdala lesions impair both physiological and behavioural responses to multimodal fear cues and the ability to identify them. We hypothesized that social fear recognition deficits following amygdala damage reflect impaired conceptual understanding of fear. Supporting this prediction, we found specific impairments in the ability to predict others' fear (but not other emotions) from written scenarios following bilateral amygdala lesions. This finding is consistent with the suggestion that social fear recognition, much like social recognition of states like pain, relies on shared internal representations. Preserved judgements about the permissibility of causing others fear confirms suggestions that social emotion recognition and morality are dissociable.
| S-EPMC7893280 | biostudies-literature
Project description:Childhood sexual abuse (CSA) has been associated with psychopathology, particularly major depressive disorder (MDD), and high-risk behaviors. Despite the epidemiological data available, the mechanisms underlying these maladaptive outcomes remain poorly understood.We examined whether a history of CSA, particularly in conjunction with a past episode of MDD, is associated with behavioral and neural dysfunction in reinforcement learning, and whether such dysfunction is linked to maladaptive behavior.Participants completed a clinical evaluation and a probabilistic reinforcement task while 128-channel event-related potentials were recorded.Academic setting; participants recruited from the community.Fifteen women with a history of CSA and remitted MDD (CSA + rMDD), 16 women with remitted MDD with no history of CSA (rMDD), and 18 healthy women (controls).Three or more episodes of coerced sexual contact (mean [SD] duration, 3.00 [2.20] years) between the ages of 7 and 12 years by at least 1 male perpetrator.Participants' preference for choosing the most rewarded stimulus and avoiding the most punished stimulus was evaluated. The feedback-related negativity and error-related negativity-hypothesized to reflect activation in the anterior cingulate cortex-were used as electrophysiological indices of reinforcement learning.No group differences emerged in the acquisition of reinforcement contingencies. In trials requiring participants to rely partially or exclusively on previously rewarded information, the CSA + rMDD group showed (1) lower accuracy (relative to both controls and the rMDD group), (2) blunted electrophysiological differentiation between correct and incorrect responses (relative to controls), and (3) increased activation in the subgenual anterior cingulate cortex (relative to the rMDD group). A history of CSA was not associated with impairments in avoiding the most punished stimulus. Self-harm and suicidal behaviors correlated with poorer performance of previously rewarded, but not previously punished, trials.Irrespective of past MDD episodes, women with a history of CSA showed neural and behavioral deficits in utilizing previous reinforcement to optimize decision making in the absence of feedback (blunted "Go learning"). Although our study provides initial evidence for reward-specific deficits associated with CSA, future research is warranted to determine if disrupted positive reinforcement learning predicts high-risk behavior following CSA.
| S-EPMC3642225 | biostudies-literature