Browse
Submit Data
Databases
API
Help

Dataset Information

15 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Imprecise action selection in substance use disorder: Evidence for active learning impairments when solving the explore-exploit dilemma.

ABSTRACT:

Background

Substance use disorders (SUDs) are a major public health risk. However, mechanisms accounting for continued patterns of poor choices in the face of negative life consequences remain poorly understood.

Methods

We use a computational (active inference) modeling approach, combined with multiple regression and hierarchical Bayesian group analyses, to examine how treatment-seeking individuals with one or more SUDs (alcohol, cannabis, sedatives, stimulants, hallucinogens, and/or opioids; N = 147) and healthy controls (HCs; N = 54) make choices to resolve uncertainty within a gambling task. A subset of SUDs (N = 49) and HCs (N = 51) propensity-matched on age, sex, and verbal IQ were also compared to replicate larger group findings.

Results

Results indicate that: (a) SUDs show poorer task performance than HCs (p = 0.03, Cohen's d = 0.33), with model estimates revealing less precise action selection mechanisms (p = 0.004, d = 0.43), a lower learning rate from losses (p = 0.02, d = 0.36), and a greater learning rate from gains (p = 0.04, d = 0.31); and (b) groups do not differ significantly in goal-directed information seeking.

Conclusions

Findings suggest a pattern of inconsistent behavior in response to positive outcomes in SUDs combined with a tendency to attribute negative outcomes to chance. Specifically, individuals with SUDs fail to settle on a behavior strategy despite sufficient evidence of its success. These learning impairments could help account for difficulties in adjusting behavior and maintaining optimal decision-making during and after treatment.

SUBMITTER: Smith R

PROVIDER: S-EPMC7502502 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Differential reinforcement encoding along the hippocampal long axis helps resolve the explore-exploit dilemma.

Project description:When making decisions, should one exploit known good options or explore potentially better alternatives? Exploration of spatially unstructured options depends on the neocortex, striatum, and amygdala. In natural environments, however, better options often cluster together, forming structured value distributions. The hippocampus binds reward information into allocentric cognitive maps to support navigation and foraging in such spaces. Here we report that human posterior hippocampus (PH) invigorates exploration while anterior hippocampus (AH) supports the transition to exploitation on a reinforcement learning task with a spatially structured reward function. These dynamics depend on differential reinforcement representations in the PH and AH. Whereas local reward prediction error signals are early and phasic in the PH tail, global value maximum signals are delayed and sustained in the AH body. AH compresses reinforcement information across episodes, updating the location and prominence of the value maximum and displaying goal cell-like ramping activity when navigating toward it.

| S-EPMC7589536 | biostudies-literature

Subcortical Substrates of Explore-Exploit Decisions in Primates.

Project description:The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational neural circuits facilitate learning based on past choices and outcomes, it is unclear whether they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques as they solved a task that required them to balance novelty-driven exploration with exploitation of what they had already learned. Using a partially observable Markov decision process (POMDP) model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding explore-exploit decisions.

| S-EPMC6687547 | biostudies-literature

Solving the apparent diversity-accuracy dilemma of recommender systems.

Project description:Recommender systems use data on past user preferences to predict possible future likes and interests. A key challenge is that while the most useful individual recommendations are to be found among diverse niche objects, the most reliably accurate results are obtained by methods that recommend objects based on user or object similarity. In this paper we introduce a new algorithm specifically to address the challenge of diversity and show how it can be used to resolve this apparent dilemma when combined in an elegant hybrid with an accuracy-focused algorithm. By tuning the hybrid appropriately we are able to obtain, without relying on any semantic or context-specific information, simultaneous gains in both accuracy and diversity of recommendations.

| S-EPMC2842039 | biostudies-literature

Solving the clinker dilemma with hybrid output-based allocation.

Project description:Output-based allocation (OBA) is one of the main options discussed for addressing carbon leakage in emissions trading systems. This paper studies how different OBA designs affect incentives on mitigation and trade in the cement sector. To do so, we develop an analytical model of sector emissions as a function of technical parameters representing abatement levers. We propose a specific design called hybrid OBA, and show that unlike the alternatives, it provides incentives for firms to reduce the carbon intensity of production without offshoring production. We assess the feasibility of hybrid OBA through expert interviews and find that the main barriers identified, including technical and administrative complexities, are manageable. However, hybrid OBA represents a mid-term solution until more robust anti-leakage measures can be introduced, because of two key limitations of OBA in general - it does not provide incentives to reduce the consumption of cement or to accelerate the development of radical low-carbon technologies, both of which are necessary to deliver deep decarbonisation.

| S-EPMC7175600 | biostudies-literature

Disturbance modifies payoffs in the explore-exploit trade-off.

Project description:Decision-making agents face a fundamental trade-off between exploring new opportunities with risky outcomes versus exploiting familiar options with more certain but potentially suboptimal outcomes. Although mediation of this trade-off is essential to adaptive behavior and has for decades been assumed to modulate performance, the empirical consequences of human exploratory strategies are unknown beyond laboratory or theoretical settings. Leveraging 540,000 vessel position records from 2494 commercial fishing trips along with corresponding revenues, here we find that during undisturbed conditions, there was no relationship between exploration and performance, contrary to theoretical predictions. However, during a major disturbance event which closed the most-utilized fishing grounds, explorers benefited significantly from less-impacted revenues and were also more likely to continue fishing. We conclude that in stochastic natural systems characterized by non-stationary rewards, the role of exploration in buffering against disturbance may be greater than previously thought in humans.

| S-EPMC6662700 | biostudies-literature

Attention-deficit/hyperactivity disorder and the explore/exploit trade-off.

Project description:The ability to maximize rewards and minimize the costs of obtaining them is vital to making advantageous explore/exploit decisions. Exploratory decisions are theorized to be greater among individuals with attention-deficit/hyperactivity disorder (ADHD), potentially due to deficient catecholamine transmission. Here, we examined the effects of ADHD status and methylphenidate, a common ADHD medication, on explore/exploit decisions using a 6-armed bandit task. We hypothesized that ADHD participants would make more exploratory decisions than controls, and that MPH would reduce group differences. On separate study days, adults with (n = 26) and without (n = 23) ADHD completed the bandit task at baseline, and after methylphenidate or placebo in counter-balanced order. Explore/exploit decisions were modeled using reinforcement learning algorithms. ADHD participants made more exploratory decisions (i.e., chose options without the highest expected reward value) and earned fewer points than controls in all three study days, and methylphenidate did not affect these outcomes. Baseline exploratory choices were positively associated with hyperactive ADHD symptoms across all participants. These results support several theoretical models of increased exploratory choices in ADHD and suggest the unexplained variance in ADHD decisions may be due to less value tracking. The inability to suppress actions with little to no reward value may be a key feature of hyperactive ADHD symptoms.

| S-EPMC8027173 | biostudies-literature

Project description:Recommender systems are designed to assist individual users to navigate through the rapidly growing amount of information. One of the most successful recommendation techniques is the collaborative filtering, which has been extensively investigated and has already found wide applications in e-commerce. One of challenges in this algorithm is how to accurately quantify the similarities of user pairs and item pairs. In this paper, we employ the multidimensional scaling (MDS) method to measure the similarities between nodes in user-item bipartite networks. The MDS method can extract the essential similarity information from the networks by smoothing out noise, which provides a graphical display of the structure of the networks. With the similarity measured from MDS, we find that the item-based collaborative filtering algorithm can outperform the diffusion-based recommendation algorithms. Moreover, we show that this method tends to recommend unpopular items and increase the global diversification of the networks in long term.

| S-EPMC4208813 | biostudies-literature

Biases in the Explore-Exploit Tradeoff in Addictions: The Role of Avoidance of Uncertainty.

Project description:We focus on exploratory decisions across disorders of compulsivity, a potential dimensional construct for the classification of mental disorders. Behaviors associated with the pathological use of alcohol or food, in alcohol use disorders (AUD) or binge-eating disorder (BED), suggest a disturbance in explore-exploit decision-making, whereby strategic exploratory decisions in an attempt to improve long-term outcomes may diminish in favor of more repetitive or exploitatory choices. We compare exploration vs exploitation across disorders of natural (obesity with and without BED) and drug rewards (AUD). We separately acquired resting state functional MRI data using a novel multi-echo planar imaging sequence and independent components analysis from healthy individuals to assess the neural correlates underlying exploration. Participants with AUD showed reduced exploratory behavior across gain and loss environments, leading to lower-yielding exploitatory choices. Obese subjects with and without BED did not differ from healthy volunteers but when compared with each other or to AUD subjects, BED had enhanced exploratory behaviors particularly in the loss domain. All subject groups had decreased exploration or greater uncertainty avoidance to losses compared with rewards. More exploratory decisions in the context of reward were associated with frontal polar and ventral striatal connectivity. For losses, exploration was associated with frontal polar and precuneus connectivity. We further implicate the relevance and dimensionality of constructs of compulsivity across disorders of both natural and drug rewards.

| S-EPMC4650253 | biostudies-literature

Impairments in action-outcome learning in schizophrenia.

Project description:Learning the causal relation between actions and their outcomes (AO learning) is critical for goal-directed behavior when actions are guided by desire for the outcome. This can be contrasted with habits that are acquired by reinforcement and primed by prevailing stimuli, in which causal learning plays no part. Recently, we demonstrated that goal-directed actions are impaired in schizophrenia; however, whether this deficit exists alongside impairments in habit or reinforcement learning is unknown. The present study distinguished deficits in causal learning from reinforcement learning in schizophrenia. We tested people with schizophrenia (SZ, n?=?25) and healthy adults (HA, n?=?25) in a vending machine task. Participants learned two action-outcome contingencies (e.g., push left to get a chocolate M&M, push right to get a cracker), and they also learned one contingency was degraded by delivery of noncontingent outcomes (e.g., free M&Ms), as well as changes in value by outcome devaluation. Both groups learned the best action to obtain rewards; however, SZ did not distinguish the more causal action when one AO contingency was degraded. Moreover, action selection in SZ was insensitive to changes in outcome value unless feedback was provided, and this was related to the deficit in AO learning. The failure to encode the causal relation between action and outcome in schizophrenia occurred without any apparent deficit in reinforcement learning. This implies that poor goal-directed behavior in schizophrenia cannot be explained by a more primary deficit in reward learning such as insensitivity to reward value or reward prediction errors.

| S-EPMC5834614 | biostudies-literature

Polymeric nucleic acid vehicles exploit active interorganelle trafficking mechanisms.

Project description:Materials that self-assemble with nucleic acids into nanocomplexes (e.g. polyplexes) are widely used in many fundamental biological and biomedical experiments. However, understanding the intracellular transport mechanisms of these vehicles remains a major hurdle in their effective usage. Here, we investigate two polycation models, Glycofect (which slowly degrades via hydrolysis) and linear polyethyleneimine (PEI) (which does not rapidly hydrolyze), to determine the impact of polymeric structure on intracellular trafficking. Cells transfected using Glycofect underwent increasing transgene expression over the course of 40 h and remained benign over the course of 7 days. Transgene expression in cells transfected with PEI peaked at 16 h post-transfection and resulted in less than 10% survival after 7 days. While saccharide-containing Glycofect has a higher buffering capacity than PEI, polyplexes created with Glycofect demonstrate more sustained endosomal release, possibly suggesting an additional or alternative delivery mechanism to the classical "proton sponge mechanism". PEI appeared to promote release of DNA from acidic organelles more than Glycofect. Immunofluorescence images indicate that both Glycofect and linear PEI traffic oligodeoxynucleotides to the Golgi and endoplasmic reticulum, which may be a route towards nuclear delivery. However, Glycofect polyplexes demonstrated higher co-localization with the ER than PEI polyplexes, and co-localization experiments indicate the retrograde transport of polyplexes via COP I vesicles from the Golgi to the ER. We conclude that slow release and unique trafficking behaviors of Glycofect polyplexes may be due to the presence of saccharide units and the degradable nature of the polymer, allowing more efficacious and benign delivery.

| S-EPMC3586558 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data