Browse
Submit Data
Databases
API
Help

Dataset Information

19 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Dopamine, locus of control, and the exploration-exploitation tradeoff.

ABSTRACT: Whether to continue to exploit a source of reward, or to search for a new one of potentially greater value, is a fundamental and underconstrained decision. Recent computational studies of this exploration-exploitation tradeoff have found that variability in exploration across individuals is influenced by a functional polymorphism (Val158Met) in the catechol-O-methyltransferase (COMT) gene, whose protein product degrades synaptically released dopamine. However, these and other genotype-phenotype associations have rarely been causally tested. To directly test this association and to evaluate additional behavioral characteristics, including perceived locus of control (LOC), here we used the COMT inhibitor tolcapone in a randomized, double-blind, counterbalanced, within-subject study of 66 subjects genotyped for the Val158Met allele to assess the hypothesis that reducing COMT enzymatic activity interacts with genotype to increase uncertainty-driven exploration. In keeping with our initial hypothesis, tolcapone led to an increase in exploratory, but not exploitative, behavior in Met/Met rather than Val/Val subjects. Independent of genotype, those subjects with a more external LOC also showed increases in uncertainty-driven exploration on tolcapone relative to placebo. However, we did not replicate our previous finding that Met/Met subjects show greater exploration at baseline. Together these findings support a model in which exploration is hypothesized to have a dopaminergic basis. Moreover, in keeping with findings in other behavioral and cognitive domains, the response to an increase in presumptively frontal dopamine is dependent upon baseline dopamine tone.

SUBMITTER: Kayser AS

PROVIDER: S-EPMC4443960 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Dopamine blockade impairs the exploration-exploitation trade-off in rats.

Project description:In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine contributes to the control of this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in random exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision-making when facing changing environmental contingencies.

| S-EPMC6494917 | biostudies-literature

Altered Corticostriatal Connectivity and Exploration/Exploitation Imbalance Emerge as Intermediate Phenotypes for a Neonatal Dopamine Dysfunction.

Project description:Findings showing that neonatal lesions of the forebrain dopaminergic system in rodents lead to juvenile locomotor hyperactivity and learning deficits have been taken as evidence of face validity for the attention deficit hyperactivity disorder. However, the core cognitive and physiological intermediate phenotypes underlying this rodent syndrome remain unknown. Here we show that early postnatal dopaminergic lesions cause long-lasting deficits in exploitation of shelter, social and nutritional resources, and an imbalanced exploratory behavior, where nondirected local exploration is exacerbated, whereas sophisticated search behaviors involving sequences of goal directed actions are degraded. Importantly, some behavioral deficits do not diminish after adolescence but instead worsen or mutate, particularly those related to the exploration of wide and spatially complex environments. The in vivo electrophysiological recordings and morphological reconstructions of striatal medium spiny neurons reveal corticostriatal alterations associated to the behavioral phenotype. More specifically, an attenuation of corticostriatal functional connectivity, affecting medial prefrontal inputs more markedly than cingulate and motor inputs, is accompanied by a contraction of the dendritic arbor of striatal projection neurons in this animal model. Thus, dopaminergic neurons are essential during postnatal development for the functional and structural maturation of corticostriatal connections. From a bottom-up viewpoint, our findings suggest that neuropsychiatric conditions presumably linked to developmental alterations of the dopaminergic system should be evaluated for deficits in foraging decision making, alterations in the recruitment of corticostriatal circuits during foraging tasks, and structural disorganization of the frontostriatal connections.

| S-EPMC4569947 | biostudies-literature

Exploration and Exploitation in Natural Viewing Behavior.

Project description:Many eye-tracking studies investigate visual behavior with a focus on image features and the semantic content of a scene. A wealth of results on these aspects is available, and our understanding of the decision process where to look has reached a mature stage. However, the temporal aspect, whether to stay and further scrutinize a region (exploitation) or to move on and explore image regions that were yet not in the focus of attention (exploration) is less well understood. Here, we investigate the trade-off between these two processes across stimuli with varying properties and sizes. In a free viewing task, we examined gaze parameters in humans, involving the central tendency, entropy, saccadic amplitudes, number of fixations and duration of fixations. The results revealed that the central tendency and entropy scaled with stimulus size. The mean saccadic amplitudes showed a linear increase that originated from an interaction between the distribution of saccades and the spatial bias. Further, larger images led to spatially more extensive sampling as indicated by a higher number of fixations at the expense of reduced fixation durations. These results demonstrate a profound shift from exploitation to exploration as an adaptation of main gaze parameters with increasing image size.

| S-EPMC5442137 | biostudies-literature

Bumblebees learn foraging routes through exploitation-exploration cycles.

Project description:How animals explore and acquire knowledge from the environment is a key question in movement ecology. For pollinators that feed on multiple small replenishing nectar resources, the challenge is to learn efficient foraging routes while dynamically acquiring spatial information about new resource locations. Here, we use the behavioural mapping t-Stochastic Neighbouring Embedding algorithm and Shannon entropy to statistically analyse previously published sampling patterns of bumblebees feeding on artificial flowers in the field. We show that bumblebees modulate foraging excursions into distinctive behavioural strategies, characterizing the trade-off dynamics between (i) visiting and exploiting flowers close to the nest, (ii) searching for new routes and resources, and (iii) exploiting learned flower visitation sequences. Experienced bees combine these behavioural strategies even after they find an optimal route minimizing travel distances between flowers. This behavioural variability may help balancing energy costs-benefits and facilitate rapid adaptation to changing environments and the integration of more profitable resources in their routes.

| S-EPMC6685008 | biostudies-literature

Human collective intelligence under dual exploration-exploitation dilemmas.

Project description:The exploration-exploitation dilemma is a recurrent adaptive problem for humans as well as non-human animals. Given a fixed time/energy budget, every individual faces a fundamental trade-off between exploring for better resources and exploiting known resources to optimize overall performance under uncertainty. Colonies of eusocial insects are known to solve this dilemma successfully via evolved coordination mechanisms that function at the collective level. For humans and other non-eusocial species, however, this dilemma operates within individuals as well as between individuals, because group members may be motivated to take excessive advantage of others' exploratory findings through social learning. Thus, even though social learning can reduce collective exploration costs, the emergence of disproportionate "information scroungers" may severely undermine its potential benefits. We investigated experimentally whether social learning opportunities might improve the performance of human participants working on a "multi-armed bandit" problem in groups, where they could learn about each other's past choice behaviors. Results showed that, even though information scroungers emerged frequently in groups, social learning opportunities reduced total group exploration time while increasing harvesting from better options, and consequentially improved collective performance. Surprisingly, enriching social information by allowing participants to observe others' evaluations of chosen options (e.g., Amazon's 5-star rating system) in addition to choice-frequency information had a detrimental impact on performance compared to the simpler situation with only the choice-frequency information. These results indicate that humans groups can handle the fundamental "dual exploration-exploitation dilemmas" successfully, and that social learning about simple choice-frequencies can help produce collective intelligence.

| S-EPMC3995913 | biostudies-literature

KOR Control over Addiction Processing: An Exploration of the Mesolimbic Dopamine Pathway.

Project description:Drug addiction is a complex, persistent, and chronically relapsing neurological disorder exacerbated by acute and chronic stress. It is well known that the dynorphin/kappa opioid receptor (KOR) system regulates stress perception and responsivity, while the mesolimbic dopamine system plays a role in reward and reinforcement associated with alcohol and substance use disorders. Interestingly, the dopamine and dynorphin/KOR systems are highly integrated in mesolimbic areas, with KOR activation leading to inhibition of dopamine release, further altering the perception of reinforcing and aversive stimuli. Chronic or repeated exposure to stress or drugs potentiates KOR function ultimately contributing to a hypodopaminergic state. This hypodopaminergic state is one of the hallmarks of hyperkatifeia, defined as the hypersensitivity to emotional distress that is exacerbated during drug withdrawal and abstinence. The relationship between stress and drug addiction is bidirectional; repeated/chronic stress promotes pro-addictive behaviors, and repeated cycles of drug exposure and withdrawal, across various drug classes, produces stress. Neuroadaptations driven by this bidirectional relationship ultimately influence the perception of the reinforcing value of rewarding stimuli. In this chapter, we address the involvement of the dopamine and dynorphin/KOR systems and their interactions in shaping reinforcement value processing after drug and stress exposure, as well as a combinatorial impact of both drugs and stress.

| S-EPMC8192597 | biostudies-literature

Pure correlates of exploration and exploitation in the human brain.

Project description:Balancing exploration and exploitation is a fundamental problem in reinforcement learning. Previous neuroimaging studies of the exploration-exploitation dilemma could not completely disentangle these two processes, making it difficult to unambiguously identify their neural signatures. We overcome this problem using a task in which subjects can either observe (pure exploration) or bet (pure exploitation). Insula and dorsal anterior cingulate cortex showed significantly greater activity on observe trials compared to bet trials, suggesting that these regions play a role in driving exploration. A model-based analysis of task performance suggested that subjects chose to observe until a critical evidence threshold was reached. We observed a neural signature of this evidence accumulation process in the ventromedial prefrontal cortex. These findings support theories positing an important role for anterior cingulate cortex in exploration, while also providing a new perspective on the roles of insula and ventromedial prefrontal cortex.

| S-EPMC5825268 | biostudies-other

Confidence modulates exploration and exploitation in value-based learning.

Project description:Uncertainty is ubiquitous in cognitive processing. In this study, we aim to investigate the ability agents possess to track and report the noise inherent in their mental operations, often in the form of confidence judgments. Here, we argue that humans can use uncertainty inherent in their representations of value beliefs to arbitrate between exploration and exploitation. Such uncertainty is reflected in explicit confidence judgments. Using a novel variant of a multi-armed bandit paradigm, we studied how beliefs were formed and how uncertainty in the encoding of these value beliefs (belief confidence) evolved over time. We found that people used uncertainty to arbitrate between exploration and exploitation, reflected in a higher tendency toward exploration when their confidence in their value representations was low. We furthermore found that value uncertainty can be linked to frameworks of metacognition in decision making in two ways. First, belief confidence drives decision confidence, i.e. people's evaluation of their own choices. Second, individuals with higher metacognitive insight into their choices were also better at tracing the uncertainty in their environment. Together, these findings argue that such uncertainty representations play a key role in the context of cognitive control.

| S-EPMC6505439 | biostudies-literature

K-ATP channels in dopamine substantia nigra neurons control bursting and novelty-induced exploration.

Project description:Phasic activation of the dopamine (DA) midbrain system in response to unexpected reward or novelty is critical for adaptive behavioral strategies. This activation of DA midbrain neurons occurs via a synaptically triggered switch from low-frequency background spiking to transient high-frequency burst firing. We found that, in medial DA neurons of the substantia nigra (SN), activity of ATP-sensitive potassium (K-ATP) channels enabled NMDA-mediated bursting in vitro as well as spontaneous in vivo burst firing in anesthetized mice. Cell-selective silencing of K-ATP channel activity in medial SN DA neurons revealed that their K-ATP channel-gated burst firing was crucial for novelty-dependent exploratory behavior. We also detected a transcriptional upregulation of K-ATP channel and NMDA receptor subunits, as well as high in vivo burst firing, in surviving SN DA neurons from Parkinson's disease patients, suggesting that burst-gating K-ATP channel function in DA neurons affects phenotypes in both disease and health.

| S-EPMC4242970 | biostudies-literature

The effects of risk-taking, exploitation, and exploration on creativity.

Project description:The purpose of this paper was to investigate the effects of risk-taking, exploitation, and exploration on creativity by taking a model-based computational approach to both divergent and convergent thinking as primary ingredients of creativity. We adopted a reinforcement learning framework of Q learning to provide a simple, rigorous account of behavior in the decision-making process and examined the determinants of divergent and convergent thinking. Our findings revealed that risk-taking has positive effects on divergent thinking, but not related to convergent thinking. In particular, divergent thinkers with a high learning capacity were more likely to engage in risk-taking when facing losses than when facing gains. This risk-taking behavior not only contributes to the rapid achievement of learning convergence, but is also associated with high performance in divergent thinking tasks. Conversely, both exploitation and exploration had no significant effects on creativity once these risk attitudes were considered. Moreover, while convergent thinking relied on personality characteristics, it was not associated with risk-taking, exploitation, or exploration.

| S-EPMC7392310 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data