Dataset Information

Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning.

ABSTRACT:

Background

Serious and debilitating symptoms of anxiety are the most common mental health problem worldwide, accounting for around 5% of all adult years lived with disability in the developed world. Avoidance behavior-avoiding social situations for fear of embarrassment, for instance-is a core feature of such anxiety. However, as for many other psychiatric symptoms the biological mechanisms underlying avoidance remain unclear.

Methods

Reinforcement learning models provide formal and testable characterizations of the mechanisms of decision making; here, we examine avoidance in these terms. A total of 101 healthy participants and individuals with mood and anxiety disorders completed an approach-avoidance go/no-go task under stress induced by threat of unpredictable shock.

Results

We show an increased reliance in the mood and anxiety group on a parameter of our reinforcement learning model that characterizes a prepotent (pavlovian) bias to withhold responding in the face of negative outcomes. This was particularly the case when the mood and anxiety group was under stress.

Conclusions

This formal description of avoidance within the reinforcement learning framework provides a new means of linking clinical symptoms with biophysically plausible models of neural circuitry and, as such, takes us closer to a mechanistic understanding of mood and anxiety disorders.

SUBMITTER: Mkrtchian A

PROVIDER: S-EPMC5598542 | biostudies-literature | 2017 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning.

Mkrtchian Anahit A Aylward Jessica J Dayan Peter P Roiser Jonathan P JP Robinson Oliver J OJ

Biological psychiatry 20170208 7

<h4>Background</h4>Serious and debilitating symptoms of anxiety are the most common mental health problem worldwide, accounting for around 5% of all adult years lived with disability in the developed world. Avoidance behavior-avoiding social situations for fear of embarrassment, for instance-is a core feature of such anxiety. However, as for many other psychiatric symptoms the biological mechanisms underlying avoidance remain unclear.<h4>Methods</h4>Reinforcement learning models provide formal a ...[more]

PMID: 28343697

Similar Datasets

Project description:ImportanceComputational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.ObjectiveTo assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.Data sourcesWeb of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).Study selectionStudies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.Data extraction and synthesisArticles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.Main outcomes and measuresThe primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).ResultsA total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).Conclusions and relevanceThe simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders.

Project description:ImportanceChildren with strabismus have poorer functional vision and decreased quality of life than those without strabismus.ObjectiveTo evaluate the association between strabismus and mental illness among children.Design, setting, and participantsThis cross-sectional study analyzed claims data from the OptumLabs Data Warehouse, a longitudinal deidentified commercial insurance claims database, from 12 005 189 patients enrolled in the health plan between January 1, 2007, and December 31, 2017. Eligibility criteria included age younger than 19 years at the time of strabismus diagnosis, enrollment in the health plan between 2007 and 2018, and having at least 1 strabismus claim based on International Classification of Diseases, Ninth Revision, Clinical Modification and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Clinical Modification codes. Controls were children in the same database with no eye disease codes other than refractive error reported. Demographic characteristics and mental illness claims were compared. Statistical analysis was conducted from December 1, 2018, to July 31, 2021.Main outcomes and measuresPresence of mental illness claims.ResultsAmong the 12 005 189 patients (6 095 523 boys [50.8%]; mean [SD] age, 8.0 [5.9] years) in the study, adjusted odds ratios for the association of mental illnesses with strabismus were 2.01 (95% CI, 1.99-2.04) for anxiety disorder, 1.83 (95% CI, 1.76-1.90) for schizophrenia, 1.64 (95% CI, 1.59-1.70) for bipolar disorder, 1.61 (95% CI, 1.59-1.63) for depressive disorder, and 0.99 (95% CI, 0.97-1.02) for substance use disorder. There was a moderate association between each strabismus type (esotropia, exotropia, and hypertropia) and anxiety disorder, schizophrenia, bipolar disorder, and depressive disorder; odds ratios ranged from 1.23 (95% CI, 1.17-1.29) for the association between esotropia and bipolar disorder to 2.70 (95% CI, 2.66-2.74) for the association between exotropia and anxiety disorder.Conclusions and relevanceThis cross-sectional study suggests that there was a moderate association between strabismus and anxiety disorder, schizophrenia, bipolar disorder, and depressive disorder but not substance use disorder. Recognizing that these associations exist should encourage mental illness screening and treatment for patients with strabismus.

Project description:Avoidance behaviors, in which a learned response causes omission of an upcoming punisher, are a core feature of many psychiatric disorders. While reinforcement learning (RL) models have been widely used to study the development of appetitive behaviors, less attention has been paid to avoidance. Here, we present a RL model of lever-press avoidance learning in Sprague-Dawley (SD) rats and in the inbred Wistar Kyoto (WKY) rat, which has been proposed as a model of anxiety vulnerability. We focus on "warm-up," transiently decreased avoidance responding at the start of a testing session, which is shown by SD but not WKY rats. We first show that a RL model can correctly simulate key aspects of acquisition, extinction, and warm-up in SD rats; we then show that WKY behavior can be simulated by altering three model parameters, which respectively govern the tendency to explore new behaviors vs. exploit previously reinforced ones, the tendency to repeat previous behaviors regardless of reinforcement, and the learning rate for predicting future outcomes. This suggests that several, dissociable mechanisms may contribute independently to strain differences in behavior. The model predicts that, if the "standard" inter-session interval is shortened from 48 to 24 h, SD rats (but not WKY) will continue to show warm-up; we confirm this prediction in an empirical study with SD and WKY rats. The model further predicts that SD rats will continue to show warm-up with inter-session intervals as short as a few minutes, while WKY rats will not show warm-up, even with inter-session intervals as long as a month. Together, the modeling and empirical data indicate that strain differences in warm-up are qualitative rather than just the result of differential sensitivity to task variables. Understanding the mechanisms that govern expression of warm-up behavior in avoidance may lead to better understanding of pathological avoidance, and potential pathways to modify these processes.

Dataset Information

Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning.

Background

Methods

Results

Conclusions

Publications

Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets