Dataset Information

Temporal discounting correlates with directed exploration but not with random exploration.

ABSTRACT: The explore-exploit dilemma describes the trade off that occurs any time we must choose between exploring unknown options and exploiting options we know well. Implicit in this trade off is how we value future rewards - exploiting is usually better in the short term, but in the longer term the benefits of exploration can be huge. Thus, in theory there should be a tight connection between how much people value future rewards, i.e. how much they discount future rewards relative to immediate rewards, and how likely they are to explore, with less 'temporal discounting' associated with more exploration. By measuring individual differences in temporal discounting and correlating them with explore-exploit behavior, we tested whether this theoretical prediction holds in practice. We used the 27-item Delay-Discounting Questionnaire to estimate temporal discounting and the Horizon Task to quantify two strategies of explore-exploit behavior: directed exploration, where information drives exploration by choice, and random exploration, where behavioral variability drives exploration by chance. We find a clear correlation between temporal discounting and directed exploration, with more temporal discounting leading to less directed exploration. Conversely, we find no relationship between temporal discounting and random exploration. Unexpectedly, we find that the relationship with directed exploration appears to be driven by a correlation between temporal discounting and uncertainty seeking at short time horizons, rather than information seeking at long horizons. Taken together our results suggest a nuanced relationship between temporal discounting and explore-exploit behavior that may be mediated by multiple factors.

SUBMITTER: Sadeghiyeh H

PROVIDER: S-EPMC7055215 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Temporal discounting correlates with directed exploration but not with random exploration.

Sadeghiyeh Hashem H Wang Siyu S Alberhasky Maxwell R MR Kyllo Hannah M HM Shenhav Amitai A Wilson Robert C RC

Scientific reports 20200304 1

The explore-exploit dilemma describes the trade off that occurs any time we must choose between exploring unknown options and exploiting options we know well. Implicit in this trade off is how we value future rewards - exploiting is usually better in the short term, but in the longer term the benefits of exploration can be huge. Thus, in theory there should be a tight connection between how much people value future rewards, i.e. how much they discount future rewards relative to immediate rewards ...[more]

PMID: 32132573

Similar Datasets

Project description:Schizophrenia is associated with a number of deficits in decision-making, but the scope, nature, and cause of these deficits are not completely understood. Here we focus on a particular type of decision, known as the explore/exploit dilemma, in which people must choose between exploiting options that yield relatively known rewards and exploring more ambiguous options of uncertain reward probability or magnitude. Previous work has shown that healthy people use two distinct strategies to decide when to explore: directed exploration, which involves choosing options that would reduce uncertainty about the reward values (information seeking), and random exploration (exploring by chance), which describes behavioral variability that is not goal directed. We administered a recently developed gambling task designed to quantify both directed and random exploration to 108 patients with schizophrenia (PSZ) and 33 healthy volunteers (HVs). We found that PSZ patients show reduced directed exploration relative to HVs, but no difference in random exploration. Moreover, patients' directed exploration behavior clusters into two qualitatively different behavioral phenotypes. In the first phenotype, which accounts for the majority of the patients (79%) and is consistent with previously reported behavior, directed exploration is only marginally (but significantly) reduced, suggesting that these patients can use directed exploration, but at a slightly lower level than community controls. In contrast, the second phenotype, comprising 21% of patients, exhibit a form of "extreme ambiguity aversion," in which they almost never choose more informative options, even when they are clearly of higher value. Moreover, in PSZ, deficits in directed exploration were related to measures of intellectual function, whereas random exploration was related to positive symptoms. Taken together, our results suggest that schizophrenia has differential effects on directed and random exploration and that investigating the explore/exploit dilemma in psychosis patients may reveal subgroups of patients with qualitatively different patterns of exploration.

Project description:Brain computer interfaces (BCIs) provide a direct communication channel by using brain signals, enabling patients with motor impairments to interact with external devices. Motion intention detection is useful for intuitive movement-based BCI as movement is the fundamental mode of interaction with the environment. The aim of this paper is to investigate the temporal dynamics of brain processes using electroencephalography (EEG) to explore novel neural correlates of motion intention. We investigate the changes in temporal dependencies of the EEG by characterising the decay of autocorrelation during asynchronous voluntary finger tapping movement. The evolution of the autocorrelation function is characterised by its relaxation time, which is used as a robust marker for motion intention. We observed that there was reorganisation of temporal dependencies in EEG during motion intention. The autocorrelation decayed slower during movement intention and faster during the resting state. There was an increase in temporal dependence during movement intention. The relaxation time of the autocorrelation function showed significant (p < 0.05) discrimination between movement and resting state with the mean sensitivity of 78.37 ± 8.83%. The relaxation time provides movement related information that is complementary to the well-known event-related desynchronisation (ERD) by characterising the broad band EEG dynamics which is frequency independent in contrast to ERD. It can also detect motion intention on average 0.51s before the actual movement onset. We have thoroughly compared autocorrelation relaxation time features with ERD in four frequency bands. The relaxation time may therefore, complement the well-known features used in motion-based BCI leading to more robust and intuitive BCI solutions. The results obtained suggest that changes in autocorrelation decay may involve reorganisation of temporal dependencies of brain activity over longer duration during motion intention. This opens the possibilities of investigating further the temporal dynamics of fundamental neural processes underpinning motion intention.

Dataset Information

Temporal discounting correlates with directed exploration but not with random exploration.

Publications

Temporal discounting correlates with directed exploration but not with random exploration.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets