Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making.
Ontology highlight
ABSTRACT:
SUBMITTER: Xu HA
PROVIDER: S-EPMC8205159 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:People often make decisions in a social environment. The present work examines social influence on people's decisions in a sequential decision-making situation. In the first experimental study, we implemented an information cascade paradigm, illustrating that people infer information from decisions of others and use this information to make their own decisions. We followed a cognitive modeling approach to elicit the weight people give to social as compared to private individual information. The proposed social influence model shows that participants overweight their own private information relative to social information, contrary to the normative Bayesian account. In our second study, we embedded the abstract decision problem of Study 1 in a medical decision-making problem. We examined whether in a medical situation people also take others' authority into account in addition to the information that their decisions convey. The social influence model illustrates that people weight social information differentially according to the authority of other decision makers. The influence of authority was strongest when an authority's decision contrasted with private information. Both studies illustrate how the social environment provides sources of information that people integrate differently for their decisions.
| S-EPMC4718651 | biostudies-literature
Project description:In recent years, studies have increasingly dealt with the interaction of gaze behavior and decision making of team sports athletes. However, there is still a variety of important game situations, for example, in the case of penalty corners in field hockey, in which this interaction has not been investigated in detail yet. Penalty corners present a meaningful goal scoring opportunity by providing a relatively free shot. This paper considers two studies. The first study investigated a possible connection between the gaze behavior and the quality of decisions of experienced field hockey players and evaluated the level of success of different gaze strategies. A preliminary study (Study 1) was designed as a survey questionnaire with the aim of preparing for the main study by obtaining subjective assessments of the individual gaze behavior and decision making of professional athletes. In the second and the main study (Study 2), the gaze behavior of experienced field hockey players was recorded using mobile eye-tracking systems to analyze different strategical approaches in associated gaze behavior and decision making. Study 1 showed that players consider reacting to the defenders' behavior during a penalty corner a promising avenue for improving success at penalty corner attempts. It also indicated that such defense-dependent strategies are currently only rarely employed. Study 2 demonstrated how gaze behavior differs between different strategical approaches of the offense. It was shown that the gaze direction on the ball, the stopper, and the goal area is important to allow for a more optimal adaptation to the tactical behavior of defense. It can be concluded that adaptive decision making (i.e., choosing which variation will be carried out just after the "injection" of the ball) seems promising but requires further training to improve the success rate of penalty corner.
| S-EPMC8366230 | biostudies-literature
Project description:Many decisions in life are sequential and constrained by a time window. Although mathematically derived optimal solutions exist, it has been reported that humans often deviate from making optimal choices. Here, we used a secretary problem, a classic example of finite sequential decision-making, and investigated the mechanisms underlying individuals' suboptimal choices. Across three independent experiments, we found that a dynamic programming model comprising subjective value function explains individuals' deviations from optimality and predicts the choice behaviors under fewer and more opportunities. We further identified that pupil dilation reflected the levels of decision difficulty and subsequent choices to accept or reject the stimulus at each opportunity. The value sensitivity, a model-based estimate that characterizes each individual's subjective valuation, correlated with the extent to which individuals' physiological responses tracked stimuli information. Our results provide model-based and physiological evidence for subjective valuation in finite sequential decision-making, rediscovering human suboptimality in subjectively optimal decision-making processes.
| S-EPMC8675647 | biostudies-literature
Project description:Many of the decisions we make in our everyday lives are sequential and entail sparse rewards. While sequential decision-making has been extensively investigated in theory (e.g., by reinforcement learning models) there is no systematic experimental paradigm to test it. Here, we developed such a paradigm and investigated key components of reinforcement learning models: the eligibility trace (i.e., the memory trace of previous decision steps), the external reward, and the ability to exploit the statistics of the environment's structure (model-free vs. model-based mechanisms). We show that the eligibility trace decays not with sheer time, but rather with the number of discrete decision steps made by the participants. We further show that, unexpectedly, neither monetary rewards nor the environment's spatial regularity significantly modulate behavioral performance. Finally, we found that model-free learning algorithms describe human performance better than model-based algorithms.
| S-EPMC5339299 | biostudies-literature
Project description:Optimal decisions across extended time horizons require value calculations over multiple probabilistic future states. Humans may circumvent such complex computations by resorting to easy-to-compute heuristics that approximate optimal solutions. To probe the potential interplay between heuristic and optimal computations, we develop a novel sequential decision-making task, framed as virtual foraging in which participants have to avoid virtual starvation. Rewards depend only on final outcomes over five-trial blocks, necessitating planning over five sequential decisions and probabilistic outcomes. Here, we report model comparisons demonstrating that participants primarily rely on the best available heuristic but also use the normatively optimal policy. FMRI signals in medial prefrontal cortex (MPFC) relate to heuristic and optimal policies and associated choice uncertainties. Crucially, reaction times and dorsal MPFC activity scale with discrepancies between heuristic and optimal policies. Thus, sequential decision-making in humans may emerge from integration between heuristic and optimal policies, implemented by controllers in MPFC.
| S-EPMC5780427 | biostudies-literature
Project description:Speed-accuracy tradeoff (SAT) is an adaptive process balancing urgency and caution when making decisions. Computational cognitive theories, known as "evidence accumulation models", have explained SATs via a manipulation of the amount of evidence necessary to trigger response selection. New light has been shed on these processes by single-cell recordings from monkeys who were adjusting their SAT settings. Those data have been interpreted as inconsistent with existing evidence accumulation theories, prompting the addition of new mechanisms to the models. We show that this interpretation was wrong, by demonstrating that the neural spiking data, and the behavioural data are consistent with existing evidence accumulation theories, without positing additional mechanisms. Our approach succeeds by using the neural data to provide constraints on the cognitive model. Open questions remain about the locus of the link between certain elements of the cognitive models and the neurophysiology, and about the relationship between activity in cortical neurons identified with decision-making vs. activity in downstream areas more closely linked with motor effectors.
| S-EPMC4081035 | biostudies-literature
Project description:Managers face risk in explorative decision-making and those who are better at such decisions can achieve future viability. To understand what makes a manager effective at explorative decision-making requires an analysis of the manager's motivational characteristics. The behavioral activation/inhibition system (BAS/BIS), fitting the motivational orientation of "approach" or "avoidance," can affect individual decision-making. However, very little is known about the neural correlates of BAS/BIS orientation and their interrelationship with the mental activity during explorative decision-making. We conducted an fMRI study on 111 potential managers to investigate how the brain responses of explorative decision-making interact with BAS/BIS. Participants were separated into high- and low-performance groups based on the median exploration-score. The low-performance group showed significantly higher BAS than that of the high-performance group, and its BAS had significant negative association with neural networks related to reward-seeking during explorative decision-making. Moreover, the BIS of the low-performance group was negatively correlated with the activation of cerebral regions responding to risk-choice during explorative decision-making. Our finding showed that BAS/BIS was associated with the brain activation during explorative decision-making only in the low-performance group. This study contributed to the understanding of the micro-foundations of strategically relevant decision-making and has an implication for management development.
| S-EPMC8339076 | biostudies-literature
Project description:In many perceptual and cognitive decision-making problems, humans sample multiple noisy information sources serially, and integrate the sampled information to make an overall decision. We derive the optimal decision procedure for two-alternative choice tasks in which the different options are sampled one at a time, sources vary in the quality of the information they provide, and the available time is fixed. To maximize accuracy, the optimal observer allocates time to sampling different information sources in proportion to their noise levels. We tested human observers in a corresponding perceptual decision-making task. Observers compared the direction of two random dot motion patterns that were triggered only when fixated. Observers allocated more time to the noisier pattern, in a manner that correlated with their sensory uncertainty about the direction of the patterns. There were several differences between the optimal observer predictions and human behaviour. These differences point to a number of other factors, beyond the quality of the currently available sources of information, that influences the sampling strategy.
| S-EPMC3842256 | biostudies-literature
Project description:Prior research illustrates that memory can guide value-based decision-making. For example, previous work has implicated both working memory and procedural memory (i.e., reinforcement learning) in guiding choice. However, other types of memories, such as episodic memory, may also influence decision-making. Here we test the role for episodic memory-specifically item versus associative memory-in supporting value-based choice. Participants completed a task where they first learned the value associated with trial unique lotteries. After a short delay, they completed a decision-making task where they could choose to reengage with previously encountered lotteries, or new never before seen lotteries. Finally, participants completed a surprise memory test for the lotteries and their associated values. Results indicate that participants chose to reengage more often with lotteries that resulted in high versus low rewards. Critically, participants not only formed detailed, associative memories for the reward values coupled with individual lotteries, but also exhibited adaptive decision-making only when they had intact associative memory. We further found that the relationship between adaptive choice and associative memory generalized to more complex, ecologically valid choice behavior, such as social decision-making. However, individuals more strongly encode experiences of social violations-such as being treated unfairly, suggesting a bias for how individuals form associative memories within social contexts. Together, these findings provide an important integration of episodic memory and decision-making literatures to better understand key mechanisms supporting adaptive behavior.
| S-EPMC4833575 | biostudies-literature
Project description:We propose a class of phase II clinical trial designs with sequential stopping and adaptive treatment allocation to evaluate treatment efficacy. Our work is based on two-arm (control and experimental treatment) designs with binary endpoints. Our overall goal is to construct more efficient and ethical randomized phase II trials by reducing the average sample sizes and increasing the percentage of patients assigned to the better treatment arms of the trials. The designs combine the Bayesian decision-theoretic sequential approach with adaptive randomization procedures in order to achieve simultaneous goals of improved efficiency and ethics. The design parameters represent the costs of different decisions, for example, the decisions for stopping or continuing the trials. The parameters enable us to incorporate the actual costs of the decisions in practice. The proposed designs allow the clinical trials to stop early for either efficacy or futility. Furthermore, the designs assign more patients to better treatment arms by applying adaptive randomization procedures. We develop an algorithm based on the constrained backward induction and forward simulation to implement the designs. The algorithm overcomes the computational difficulty of the backward induction method, thereby making our approach practicable. The designs result in trials with desirable operating characteristics under the simulated settings. Moreover, the designs are robust with respect to the response rate of the control group.
| S-EPMC3873748 | biostudies-literature