Browse
Submit Data
Databases
API
Help

Dataset Information

18 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

What to Choose Next? A Paradigm for Testing Human Sequential Decision Making.

ABSTRACT: Many of the decisions we make in our everyday lives are sequential and entail sparse rewards. While sequential decision-making has been extensively investigated in theory (e.g., by reinforcement learning models) there is no systematic experimental paradigm to test it. Here, we developed such a paradigm and investigated key components of reinforcement learning models: the eligibility trace (i.e., the memory trace of previous decision steps), the external reward, and the ability to exploit the statistics of the environment's structure (model-free vs. model-based mechanisms). We show that the eligibility trace decays not with sheer time, but rather with the number of discrete decision steps made by the participants. We further show that, unexpectedly, neither monetary rewards nor the environment's spatial regularity significantly modulate behavioral performance. Finally, we found that model-free learning algorithms describe human performance better than model-based algorithms.

SUBMITTER: Tartaglia EM

PROVIDER: S-EPMC5339299 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

What to Choose Next? A Paradigm for Testing Human Sequential Decision Making.

Tartaglia Elisa M EM Clarke Aaron M AM Herzog Michael H MH

Frontiers in psychology 20170307

Many of the decisions we make in our everyday lives are sequential and entail sparse rewards. While sequential decision-making has been extensively investigated in theory (e.g., by reinforcement learning models) there is no systematic experimental paradigm to test it. Here, we developed such a paradigm and investigated key components of reinforcement learning models: the eligibility trace (i.e., the memory trace of previous decision steps), the external reward, and the ability to exploit the sta ...[more]

PMID: 28326050

Similar Datasets

Social Influences in Sequential Decision Making.

Project description:People often make decisions in a social environment. The present work examines social influence on people's decisions in a sequential decision-making situation. In the first experimental study, we implemented an information cascade paradigm, illustrating that people infer information from decisions of others and use this information to make their own decisions. We followed a cognitive modeling approach to elicit the weight people give to social as compared to private individual information. The proposed social influence model shows that participants overweight their own private information relative to social information, contrary to the normative Bayesian account. In our second study, we embedded the abstract decision problem of Study 1 in a medical decision-making problem. We examined whether in a medical situation people also take others' authority into account in addition to the information that their decisions convey. The social influence model illustrates that people weight social information differentially according to the authority of other decision makers. The influence of authority was strongest when an authority's decision contrasted with private information. Both studies illustrate how the social environment provides sources of information that people integrate differently for their decisions.

| S-EPMC4718651 | biostudies-literature

Subjective optimality in finite sequential decision-making.

Project description:Many decisions in life are sequential and constrained by a time window. Although mathematically derived optimal solutions exist, it has been reported that humans often deviate from making optimal choices. Here, we used a secretary problem, a classic example of finite sequential decision-making, and investigated the mechanisms underlying individuals' suboptimal choices. Across three independent experiments, we found that a dynamic programming model comprising subjective value function explains individuals' deviations from optimality and predicts the choice behaviors under fewer and more opportunities. We further identified that pupil dilation reflected the levels of decision difficulty and subsequent choices to accept or reject the stimulus at each opportunity. The value sensitivity, a model-based estimate that characterizes each individual's subjective valuation, correlated with the extent to which individuals' physiological responses tracked stimuli information. Our results provide model-based and physiological evidence for subjective valuation in finite sequential decision-making, rediscovering human suboptimality in subjectively optimal decision-making processes.

| S-EPMC8675647 | biostudies-literature

Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making.

Project description:Classic reinforcement learning (RL) theories cannot explain human behavior in the absence of external reward or when the environment changes. Here, we employ a deep sequential decision-making paradigm with sparse reward and abrupt environmental changes. To explain the behavior of human participants in these environments, we show that RL theories need to include surprise and novelty, each with a distinct role. While novelty drives exploration before the first encounter of a reward, surprise increases the rate of learning of a world-model as well as of model-free action-values. Even though the world-model is available for model-based RL, we find that human decisions are dominated by model-free action choices. The world-model is only marginally used for planning, but it is important to detect surprising events. Our theory predicts human action choices with high probability and allows us to dissociate surprise, novelty, and reward in EEG signals.

| S-EPMC8205159 | biostudies-literature

Heuristic and optimal policy computations in the human brain during sequential decision-making.

Project description:Optimal decisions across extended time horizons require value calculations over multiple probabilistic future states. Humans may circumvent such complex computations by resorting to easy-to-compute heuristics that approximate optimal solutions. To probe the potential interplay between heuristic and optimal computations, we develop a novel sequential decision-making task, framed as virtual foraging in which participants have to avoid virtual starvation. Rewards depend only on final outcomes over five-trial blocks, necessitating planning over five sequential decisions and probabilistic outcomes. Here, we report model comparisons demonstrating that participants primarily rely on the best available heuristic but also use the normatively optimal policy. FMRI signals in medial prefrontal cortex (MPFC) relate to heuristic and optimal policies and associated choice uncertainties. Crucially, reaction times and dorsal MPFC activity scale with discrepancies between heuristic and optimal policies. Thus, sequential decision-making in humans may emerge from integration between heuristic and optimal policies, implemented by controllers in MPFC.

| S-EPMC5780427 | biostudies-literature

Behavioral preference in sequential decision-making and its association with anxiety.

Project description:In daily life, people often make consecutive decisions before the ultimate goal is reached (i.e., sequential decision-making). However, this kind of decision-making has been largely overlooked in the literature. The current study investigated whether behavioral preference would change during sequential decisions, and the neural processes underlying the potential changes. For this purpose, we revised the classic balloon analogue risk task and recorded the electroencephalograph (EEG) signals associated with each step of decision-making. Independent component analysis performed on EEG data revealed that four EEG components elicited by periodic feedback in the current step predicted participants' decisions (gamble vs. no gamble) in the next step. In order of time sequence, these components were: bilateral occipital alpha rhythm, bilateral frontal theta rhythm, middle frontal theta rhythm, and bilateral sensorimotor mu rhythm. According to the information flows between these EEG oscillations, we proposed a brain model that describes the temporal dynamics of sequential decision-making. Finally, we found that the tendency to gamble (as well as the power intensity of bilateral frontal theta rhythms) was sensitive to the individual level of trait anxiety in certain steps, which may help understand the role of emotion in decision-making.

| S-EPMC6866314 | biostudies-literature

One-shot learning and behavioral eligibility traces in sequential decision making.

Project description:In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of sequences. We developed a novel paradigm to directly observe which actions and states along a multi-step sequence are reinforced after a single reward. By focusing our analysis on those states for which RL with and without eligibility trace make qualitatively distinct predictions, we find direct behavioral (choice probability) and physiological (pupil dilation) signatures of reinforcement learning with eligibility trace across multiple sensory modalities.

| S-EPMC6897511 | biostudies-literature

Sequential sampling from memory underlies action selection during abstract decision-making.

Project description:The study of perceptual decision-making in monkeys has provided insights into the process by which sensory evidence is integrated toward a decision. When monkeys make decisions with the knowledge of the motor actions the decisions bear upon, the process of evidence integration is instantiated by neurons involved in the selection of said actions. It is less clear how monkeys make decisions when unaware of the actions required to communicate their choice-what we refer to as "abstract" decisions. We investigated this by training monkeys to associate the direction of motion of a noisy random-dot display with the color of two targets. Crucially, the targets were displayed at unpredictable locations after the motion stimulus was extinguished. We found that the monkeys postponed decision formation until the targets were revealed. Neurons in the parietal association area LIP represented the integration of evidence leading to a choice, but as the stimulus was no longer visible, the samples of evidence must have been retrieved from short-term memory. Our results imply that when decisions are temporally unyoked from the motor actions they bear upon, decision formation is protracted until they can be framed in terms of motor actions.

| S-EPMC9090972 | biostudies-literature

Enhancing livestock vaccination decision-making through rapid diagnostic testing.

Project description:•Compared to vaccination, the collective approach to diagnostic testing presents a low-fixed cost.•Existing household livestock-health behaviors increase the likelihood for uptake of preventative health practices.•Initial evidence to support household investments in livestock preventative health over therapeutic treatments.

| S-EPMC7067263 | biostudies-literature

Women's preferences for and experiences with prenatal genetic testing decision making: Sociodemographic disparities in preference-concordant decision making.

Project description:ObjectiveTo investigate women's preferred approach to prenatal genetic testing decision making and assess concordance between preference and experience.MethodsWe conducted a secondary analysis of data from two randomized trials conducted between 2007-2012. Survey items assessed participants' preferred approach to decision making and whether they experienced a preference-concordant decision process. Logistic regression estimated relationships between patient characteristics and these outcomes.Results56% of women preferred autonomous decision making, 39% preferred shared decision making, and 5% preferred a provider-driven approach. Only 57% experienced preference-concordant decision making. On bivariate analysis, black women, Spanish-speaking Latinas and women with less education were less likely to experience this outcome than white, more educated women. Numeracy and preferring a provider-driven approach fully mediated observed disparities in preference-concordant decision making for most participants, except for Spanish-speaking Latinas, who were still less likely to have experienced this outcome after accounting for these factors.ConclusionNumeracy, preference for provider-driven decision processes, and language barriers were key drivers of disparities in preference-concordant decision making.Practice implicationsGiven the values-sensitive and quantitative nature of prenatal testing decisions, nuanced counseling and interventions to address language barriers, numeracy gaps, and decision-making preferences are needed to tailor counseling to patient's backgrounds and desires.

| S-EPMC6421100 | biostudies-literature

Adversarial vulnerabilities of human decision-making.

Project description:Adversarial examples are carefully crafted input patterns that are surprisingly poorly classified by artificial and/or natural neural networks. Here we examine adversarial vulnerabilities in the processes responsible for learning and choice in humans. Building upon recent recurrent neural network models of choice processes, we propose a general framework for generating adversarial opponents that can shape the choices of individuals in particular decision-making tasks toward the behavioral patterns desired by the adversary. We show the efficacy of the framework through three experiments involving action selection, response inhibition, and social decision-making. We further investigate the strategy used by the adversary in order to gain insights into the vulnerabilities of human choice. The framework may find applications across behavioral sciences in helping detect and avoid flawed choice.

| S-EPMC7682379 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data