Reinforcement Learning Performance and Risk for Psychosis in Youth.
Ontology highlight
ABSTRACT:
SUBMITTER: Waltz JA
PROVIDER: S-EPMC5483992 | biostudies-literature | 2015 Dec
REPOSITORIES: biostudies-literature
Publications
Reinforcement Learning Performance and Risk for Psychosis in Youth.
Waltz James A JA Demro Caroline C Schiffman Jason J Thompson Elizabeth E Kline Emily E Reeves Gloria G Xu Ziye Z Gold James J
The Journal of nervous and mental disease 20151201 12
Early identification efforts for psychosis have thus far yielded many more individuals "at risk" than actually develop psychotic illness. Here, we test whether measures of reinforcement learning (RL), known to be impaired in chronic schizophrenia, are related to the severity of clinical risk symptoms. Because of the reliance of RL on dopamine-rich frontostriatal systems and evidence of dopamine system dysfunction in the psychosis prodrome, RL measures are of specific interest in this clinical po ...[more]
Similar Datasets
Project description:Motivational and hedonic impairments are core features of a variety of types of psychopathology. An important aspect of motivational function is reinforcement learning (RL), including implicit (i.e., outside of conscious awareness) and explicit (i.e., including explicit representations about potential reward associations) learning, as well as both positive reinforcement (learning about actions that lead to reward) and punishment (learning to avoid actions that lead to loss). Here we present data from paradigms designed to assess both positive and negative components of both implicit and explicit RL, examine performance on each of these tasks among individuals with schizophrenia, schizoaffective disorder, and bipolar disorder with psychosis, and examine their relative relationships to specific symptom domains transdiagnostically. None of the diagnostic groups differed significantly from controls on the implicit RL tasks in either bias toward a rewarded response or bias away from a punished response. However, on the explicit RL task, both the individuals with schizophrenia and schizoaffective disorder performed significantly worse than controls, but the individuals with bipolar did not. Worse performance on the explicit RL task, but not the implicit RL task, was related to worse motivation and pleasure symptoms across all diagnostic categories. Performance on explicit RL, but not implicit RL, was related to working memory, which accounted for some of the diagnostic group differences. However, working memory did not account for the relationship of explicit RL to motivation and pleasure symptoms. These findings suggest transdiagnostic relationships across the spectrum of psychotic disorders between motivation and pleasure impairments and explicit RL. (PsycINFO Database Record
| S-EPMC5503766 | biostudies-literature
Project description:Numerous studies have identified reinforcement learning (RL) deficits in schizophrenia. Most have focused on chronic patients with longstanding antipsychotic treatment, however, and studies of RL in early-illness patients have produced mixed results, particularly regarding gradual/procedural learning. No study has directly contrasted both rapid and gradual RL in first-episode psychosis (FEP) samples. We examined probabilistic RL in 34 FEP patients and 36 controls, using Go/NoGo (GNG) and Gain vs Loss-Avoidance (GLA) paradigms. Our results were mixed, with FEP patients exhibiting greater impairment in the ability to use positive, as opposed to negative, feedback to drive rapid RL on the GLA, but not the GNG. By contrast, patients and controls showed similar improvement across the acquisition. Finally, we found no significant between-group differences in the postacquisition expression of value-based preference in both tasks. Negative symptoms were modestly associated with RL measures, while the overall bias to engage in Go-responding correlated significantly with psychosis severity in FEP patients, consistent with striatal hyperdopaminergia. Taken together, FEP patients demonstrated more circumscribed RL impairments than previous studies have documented in chronic samples, possibly reflecting differential symptom profiles between first-episode and chronic samples. Our finding of relatively preserved gradual/procedural RL, in briefly medicated FEP patients, might suggest spared or restored basal ganglia function. Our findings of preserved abilities to use representations of expected value to guide decision making, and our mixed results regarding rapid RL, may reflect a lesser degree of prefrontal cortical functional impairment in FEP than in chronic samples. Further longitudinal research, in larger samples, is required.
| S-EPMC5049533 | biostudies-literature
Project description:Dysfunction in the neural circuits underlying salience signaling is implicated in symptoms of psychosis and may predict conversion to a psychotic disorder in youth at clinical high risk (CHR) for psychosis. Additionally, negative symptom severity, including consummatory and anticipatory aspects of anhedonia, may predict functional outcome in individuals with schizophrenia-spectrum disorders. However, it is unclear whether anhedonia is related to the ability to attribute incentive salience to stimuli (through reinforcement learning [RL]) and whether measures of anhedonia and RL predict functional outcome in a younger, help-seeking population. We administered the Salience Attribution Test (SAT) to 33 participants who met criteria for either CHR or a recent-onset psychotic disorder and 29 help-seeking youth with nonpsychotic disorders. In the SAT, participants must identify relevant and irrelevant stimulus dimensions and be sensitive to different reinforcement probabilities for the 2 levels of the relevant dimension ("adaptive salience"). Adaptive salience attribution was positively related to both consummatory pleasure and functioning in the full sample. Analyses also revealed an indirect effect of adaptive salience on the relation between consummatory pleasure and both role (αβ = .22, 95% CI = 0.02, 0.48) and social functioning (αβ = .14, 95% CI = 0.02, 0.30). These findings suggest a distinct pathway to poor global functioning in help-seeking youth, via impaired reward sensitivity and RL.
| S-EPMC8530392 | biostudies-literature
Project description:Several studies have demonstrated that youth at clinical high risk (CHR) of developing psychosis have a high prevalence of comorbid psychiatric disorders. Less is known about the impact of comorbid diagnoses on later conversion to psychosis and the change over time. The aim of this study was to determine the frequency and distribution of psychiatric diagnoses at baseline and over time in the North American Prodrome Longitudinal Study (NAPLS 2) and the role of comorbid diagnoses in conversion to psychosis. The NAPLS 2 sample consisted of 744 CHR youth and 276 healthy controls. Only 21% of the CHR group did not have a comorbid diagnosis with many have 2-3 DSM-IV comorbid diagnoses. The most common diagnoses were anxiety and depressive disorders, which did improve over time. The only diagnosis at baseline that differentiated the converters from the non-converters was cannabis misuse. Comorbidity, except for cannabis use, was essentially independent of clinical outcome. It is possible that those with comorbid diagnoses are preferentially the help-seeking individuals that present for help in our clinics and research projects and that those who are at risk but do not have a comorbid diagnosis may not be seeking help in the prodromal phase.
| S-EPMC5731830 | biostudies-literature
Project description:ImportanceAlthough clinical criteria for identifying youth at risk for psychosis have been validated, they are not sufficiently accurate for predicting outcomes to inform major treatment decisions. The identification of biomarkers may improve outcome prediction among individuals at clinical high risk for psychosis (CHR-P).ObjectiveTo examine whether mismatch negativity (MMN) event-related potential amplitude, which is deficient in schizophrenia, is reduced in young people with the CHR-P syndrome and associated with outcomes, accounting for effects of antipsychotic medication use.Design, setting, and participantsMMN data were collected as part of the multisite case-control North American Prodrome Longitudinal Study (NAPLS-2) from 8 university-based outpatient research programs. Baseline MMN data were collected from June 2009 through April 2013. Clinical outcomes were assessed throughout 24 months. Participants were individuals with the CHR-P syndrome and healthy controls with MMN data. Participants with the CHR-P syndrome who developed psychosis (ie, converters) were compared with those who did not develop psychosis (ie, nonconverters) who were followed up for 24 months. Analysis took place between December 2019 and December 2021.Main outcomes and measuresElectroencephalography was recorded during a passive auditory oddball paradigm. MMN elicited by duration-, pitch-, and duration + pitch double-deviant tones was measured.ResultsThe CHR-P group (n = 580; mean [SD] age, 19.24 [4.39] years) included 247 female individuals (42.6%) and the healthy control group (n = 241; mean age, 20.33 [4.74] years) included 114 female individuals (47.3%). In the CHR-P group, 450 (77.6%) were not taking antipsychotic medication at baseline. Baseline MMN amplitudes, irrespective of deviant type, were deficient in future CHR-P converters to psychosis (n = 77, unmedicated n = 54) compared with nonconverters (n = 238, unmedicated n = 190) in both the full sample (d = 0.27) and the unmedicated subsample (d = 0.33). In the full sample, baseline medication status interacted with group and deviant type indicating that double-deviant MMN, compared with single deviants, was reduced in unmedicated converters compared with nonconverters (d = 0.43). Further, within the unmedicated subsample, deficits in double-deviant MMN were most strongly associated with earlier conversion to psychosis (hazard ratio, 1.40 [95% CI, 1.03-1.90]; P = .03], which persisted over and above positive symptom severity.Conclusions and relevanceThis study found that MMN amplitude deficits were sensitive to future psychosis conversion among individuals at risk of CHR-P, particularly those not taking antipsychotic medication at baseline, although associations were modest. While MMN shows limited promise as a biomarker of psychosis onset on its own, it may contribute novel risk information to multivariate prediction algorithms and serve as a translational neurophysiological target for novel treatment development in a subgroup of at-risk individuals.
| S-EPMC9178501 | biostudies-literature
Project description:The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.
| S-EPMC3359751 | biostudies-literature
Project description:The Wisconsin Card Sorting Test (WCST) is considered a gold standard for the assessment of cognitive flexibility. On the WCST, repeating a sorting category following negative feedback is typically treated as indicating reduced cognitive flexibility. Therefore such responses are referred to as 'perseveration' errors. Recent research suggests that the propensity for perseveration errors is modulated by response demands: They occur less frequently when their commitment repeats the previously executed response. Here, we propose parallel reinforcement-learning models of card sorting performance, which assume that card sorting performance can be conceptualized as resulting from model-free reinforcement learning at the level of responses that occurs in parallel with model-based reinforcement learning at the categorical level. We compared parallel reinforcement-learning models with purely model-based reinforcement learning, and with the state-of-the-art attentional-updating model. We analyzed data from 375 participants who completed a computerized WCST. Parallel reinforcement-learning models showed best predictive accuracies for the majority of participants. Only parallel reinforcement-learning models accounted for the modulation of perseveration propensity by response demands. In conclusion, parallel reinforcement-learning models provide a new theoretical perspective on card sorting and it offers a suitable framework for discerning individual differences in latent processes that subserve behavioral flexibility.
| S-EPMC7508815 | biostudies-literature
Project description:Server load levels affect the performance of cloud task execution, which is rooted in the impact of server performance on cloud task execution. Traditional cloud task scheduling methods usually only consider server load without fully considering the server's real-time load-performance mapping relationship, resulting in the inability to evaluate the server's real-time processing capability accurately. This deficiency directly affects the efficiency, performance, and user experience of cloud task scheduling. Firstly, we construct a performance platform model to monitor server real-time load and performance status information in response to the above problems. In addition, we propose a new deep reinforcement learning task scheduling method based on server real-time performance (SRP-DRL). This method introduces a real-time performance-aware strategy and adds status information about the real-time impact of task load on server performance on top of considering server load. It enhances the perception capability of the deep reinforcement learning (DRL) model in cloud scheduling environments and improves the server's load-balancing ability under latency constraints. Experimental results indicate that the SRP-DRL method has better overall performance regarding task average response time, success rate, and server average load variance compared to Random, Round-Robin, Earliest Idle Time First (EITF), and Best Fit (BEST-FIT) task scheduling methods. In particular, the SRP-DRL is highly effective in reducing server average load variance when numerous tasks arrive within a unit of time, ultimately optimizing the performance of the cloud system.
| S-EPMC11232600 | biostudies-literature
Project description:BackgroundIdentifying young people as at clinical high-risk (CHR) for psychosis affords opportunities for intervention to possibly prevent psychosis onset. Yet such CHR identification could plausibly increase stigma. We do not know whether these youth already perceive themselves to be at psychosis-risk (PR) or how their being told they are at PR might impact how they think about themselves.Methods148 CHR youth were asked about labels they had been given by others (labeling by others) or with which they personally identified (self-labeling). They were then asked which had the greatest impact on how they thought about themselves. We evaluated whether being told vs. thinking they were at PR had stronger effects.FindingsThe majority identified nonpsychotic disorders rather than PR labels as having the greatest impact on sense of self (67.6% vs. 27.7%). However, participants who identified themselves as at PR had an 8.8 (95% CI = 2.0-39.1) increase in the odds of the PR label having the greatest impact (p < 0.01). Additionally, having been told by others that they were at PR was associated with a 4.0 increase in odds (95% CI = 1.1-15.0) that the PR label had the most impact (p < 0.05).InterpretationNonpsychotic disorder labels appear to have a greater impact on CHR youth than psychosis-risk labels. However, thinking they are at PR, and, secondarily, being told they are at PR, appears to increase the relative impact of the PR label. Understanding self- and other-labeling may be important to how young people think of themselves, and may inform early intervention strategies.
| S-EPMC7079577 | biostudies-literature
Project description:In this paper, we introduce a new type of tree-based method, reinforcement learning trees (RLT), which exhibits significantly improved performance over traditional methods such as random forests (Breiman, 2001) under high-dimensional settings. The innovations are three-fold. First, the new method implements reinforcement learning at each selection of a splitting variable during the tree construction processes. By splitting on the variable that brings the greatest future improvement in later splits, rather than choosing the one with largest marginal effect from the immediate split, the constructed tree utilizes the available samples in a more efficient way. Moreover, such an approach enables linear combination cuts at little extra computational cost. Second, we propose a variable muting procedure that progressively eliminates noise variables during the construction of each individual tree. The muting procedure also takes advantage of reinforcement learning and prevents noise variables from being considered in the search for splitting rules, so that towards terminal nodes, where the sample size is small, the splitting rules are still constructed from only strong variables. Last, we investigate asymptotic properties of the proposed method under basic assumptions and discuss rationale in general settings.
| S-EPMC4760114 | biostudies-literature