Evidence for surprise minimization over value maximization in choice behavior.
Ontology highlight
ABSTRACT:
SUBMITTER: Schwartenbeck P
PROVIDER: S-EPMC4643240 | biostudies-other | 2015 Nov
REPOSITORIES: biostudies-other
Similar Datasets
Project description:Pigeons have shown suboptimal gambling-like behavior when preferring a stimulus that infrequently signals reliable reinforcement over alternatives that provide greater reinforcement overall. As a mechanism for this behavior, recent research proposed that the stimulus value of alternatives with more reliable signals for reinforcement will be preferred relatively independently of their frequencies. The present study tested this hypothesis using a simplified design of a Discriminative alternative that, 50% of the time, led to either a signal for 100% reinforcement or a blackout period indicative of 0% reinforcement against a Nondiscriminative alternative that always led to a signal that predicted 50% reinforcement. Pigeons showed a strong preference for the Discriminative alternative that remained despite reducing the frequency of the signal for reinforcement in subsequent phases to 25% and then 12.5%. In Experiment 2, using the original design of Experiment 1, the stimulus following choice of the Nondiscriminative alternative was increased to 75% and then to 100%. Results showed that preference for the Discriminative alternative decreased only when the signals for reinforcement for the two alternatives predicted the same probability of reinforcement. The ability of several models to predict this behavior are discussed, but the terminal link stimulus value offers the most parsimonious account of this suboptimal behavior.
| S-EPMC4956316 | biostudies-literature
Project description:Decision making is often considered to arise out of contributions from a model-free habitual system and a model-based goal-directed system. Here, we investigated the effect of a dopamine manipulation on the degree to which either system contributes to instrumental behavior in a two-stage Markov decision task, which has been shown to discriminate model-free from model-based control. We found increased dopamine levels promote model-based over model-free choice.
| S-EPMC3417237 | biostudies-literature
Project description:The structure and quantitative composition of the cerebral cortex are interrelated with its computational capacity. Empirical data analyzed here indicate a certain hierarchy in local cortical composition. Specifically, neural wire, i.e., axons and dendrites take each about 1/3 of cortical space, spines and glia/astrocytes occupy each about (1/3)(2), and capillaries around (1/3)(4). Moreover, data analysis across species reveals that these fractions are roughly brain size independent, which suggests that they could be in some sense optimal and thus important for brain function. Is there any principle that sets them in this invariant way? This study first builds a model of local circuit in which neural wire, spines, astrocytes, and capillaries are mutually coupled elements and are treated within a single mathematical framework. Next, various forms of wire minimization rule (wire length, surface area, volume, or conduction delays) are analyzed, of which, only minimization of wire volume provides realistic results that are very close to the empirical cortical fractions. As an alternative, a new principle called "spine economy maximization" is proposed and investigated, which is associated with maximization of spine proportion in the cortex per spine size that yields equally good but more robust results. Additionally, a combination of wire cost and spine economy notions is considered as a meta-principle, and it is found that this proposition gives only marginally better results than either pure wire volume minimization or pure spine economy maximization, but only if spine economy component dominates. However, such a combined meta-principle yields much better results than the constraints related solely to minimization of wire length, wire surface area, and conduction delays. Interestingly, the type of spine size distribution also plays a role, and better agreement with the data is achieved for distributions with long tails. In sum, these results suggest that for the efficiency of local circuits wire volume may be more primary variable than wire length or temporal delays, and moreover, the new spine economy principle may be important for brain evolutionary design in a broader context.
| S-EPMC4593638 | biostudies-literature
Project description:Adaptation is a fundamental process crucial for the efficient coding of sensory information. Recent evidence suggests that similar coding principles operate in decision-related brain areas, where neural value coding adapts to recent reward history. However, the circuit mechanism for value adaptation is unknown, and the link between changes in adaptive value coding and choice behavior is unclear. Here we show that choice behavior in nonhuman primates varies with the statistics of recent rewards. Consistent with efficient coding theory, decision-making shows increased choice sensitivity in lower variance reward environments. Both the average adaptation effect and across-session variability are explained by a novel multiple timescale dynamical model of value representation implementing divisive normalization. The model predicts empirical variance-driven changes in behavior despite having no explicit knowledge of environmental statistics, suggesting that distributional characteristics can be captured by dynamic model architectures. These findings highlight the importance of treating decision-making as a dynamic process and the role of normalization as a unifying computation for contextual phenomena in choice.
| S-EPMC6086888 | biostudies-literature
Project description:The rationality assumption that underlies mainstream economic theory has proved to be a useful approximation, despite the fact that systematic violations to its predictions can be found. That is, the assumption of rational behavior is useful in understanding the ways in which many successful economic institutions function, although it is also true that actual human behavior falls systematically short of perfect rationality. We consider a possible explanation of this apparent inconsistency, suggesting that mechanisms that rest on the rationality assumption are likely to be successful when they create an environment in which the behavior they try to facilitate leads to the best payoff for all agents on average, and most of the time. Review of basic learning research suggests that, under these conditions, people quickly learn to maximize expected return. This review also shows that there are many situations in which experience does not increase maximization. In many cases, experience leads people to underweight rare events. In addition, the current paper suggests that it is convenient to distinguish between two behavioral approaches to improve economic analyses. The first, and more conventional approach among behavioral economists and psychologists interested in judgment and decision making, highlights violations of the rational model and proposes descriptive models that capture these violations. The second approach studies human learning to clarify the conditions under which people quickly learn to maximize expected return. The current review highlights one set of conditions of this type and shows how the understanding of these conditions can facilitate market design.
| S-EPMC4113920 | biostudies-other
Project description:Following a key developmental task of childhood-building a foundation of self-knowledge in the form of domain-specific self-concepts-adolescents begin to explore their emerging identities in ways that foster autonomy and connectedness. Neuroimaging studies of self-related processes demonstrate enhanced engagement of the ventromedial prefrontal cortex in adolescence, which may facilitate and reflect the development of identity by integrating the value of potential actions and choices. Drawing from neuroeconomic and social cognitive accounts, we propose that motivated behavior during adolescence can be modeled by a general value-based decision-making process centered around value accumulation in the ventromedial prefrontal cortex. This approach advances models of adolescent neurodevelopment that focus on reward sensitivity and cognitive control by considering more diverse value inputs, including contributions of developing self- and identity-related processes. It also considers adolescent decision making and behavior from adolescents' point of view rather than adults' perspectives on what adolescents should value or how they should behave.
| S-EPMC6667174 | biostudies-literature
Project description:Humans prolifically engage in mental time travel. We dwell on past actions and experience satisfaction or regret. More than storytelling, these recollections change how we act in the future and endow us with a computationally important ability to link actions and consequences across spans of time, which helps address the problem of long-term credit assignment: the question of how to evaluate the utility of actions within a long-duration behavioral sequence. Existing approaches to credit assignment in AI cannot solve tasks with long delays between actions and consequences. Here, we introduce a paradigm where agents use recall of specific memories to credit past actions, allowing them to solve problems that are intractable for existing algorithms. This paradigm broadens the scope of problems that can be investigated in AI and offers a mechanistic account of behaviors that may inspire models in neuroscience, psychology, and behavioral economics.
| S-EPMC6864102 | biostudies-literature
Project description:Food safety remains a major issue to many consumers. Previous studies examining the economic impact of food safety recalls have focused on Class I recalls. Antibiotic residue in meat products, a Class II recall, has increased in consumer importance yet little is known about how much research and development expenditure should be allocated to reduce antibiotic residue pre- and post-harvest. This study compares demand elasticities and the decrease in willingness to pay in response to either an E. coli (Class I) or antibiotic residue (Class II) recall. We compare and contrast two competing behavioral frameworks, Random Utility and Regret Minimizing. Modeling behavior using the random regret framework is found to be more powerful for assessing consumer responses. In addition, we explore if different groups of consumers exist that either maximize utility or minimize regret. Consumer devaluations of E. coli (Class I) are 40-65% larger than antibiotic residue (Class II). Approximately 60% of consumers are identified as regret minimizers and 40% were identified as utility maximizers. While industry response and government policy recommendations differed conditional on modeling framework, the regret minimizing framework required smaller price discounts than regret minimizing to maintain the same level of market share.
| S-EPMC7494847 | biostudies-literature
Project description:BACKGROUND:Suicidal behavior is associated with impaired decision making in contexts of uncertainty. Existing studies, however, do not definitively address whether suidice attempers have 1) impairment in learning from experience or 2) impairment in choice based on comparison of estimated option values. Our reinforcement learning model-based behavioral study tested these hypotheses directly in middle-aged and older suicide attempters representative of those who die by suicide. METHODS:Two samples (sample 1, n = 135; sample 2, n = 125) of suicide attempters with depression (nattempters = 54 and 39, respectively), suicide ideators, nonsuicidal patients with depression, and healthy control participants completed a probabilistic three-choice decision-making task. A second experiment in sample 2 experimentally dissociated long-term learned value from reward magnitude. Analyses combined computational reinforcement learning and mixed-effects models of decision times and choices. RESULTS:With regard to learning, suicide attempters (vs. all comparison groups) were less sensitive to one-back reinforcement, as indicated by a reduced effect on both choices and decision times. Learning deficits scaled with attempt lethality and were partially explained by poor cognitive control. With regard to value-based choice, suicide attempters (vs. all comparison groups) displayed abnormally long decision times when choosing between similarly valued options and were less able to distinguish between the best and second-best options. Group differences in value-based choice were robust to controlling for cognitive performance, comorbidities, impulsivity, psychotropic exposure, and possible brain damage from attempts. CONCLUSIONS:Serious suicidal behavior is associated with impaired reward learning, likely undermining the search for alternative solutions. Attempted suicide is associated with impaired value comparison during the choice process, potentially interfering with the consideration of deterrents and alternatives in a crisis.
| S-EPMC6380943 | biostudies-literature
Project description:Understanding the cognitive and neural processes that underlie human decision making requires the successful prediction of how, but also of when, people choose. Sequential sampling models (SSMs) have greatly advanced the decision sciences by assuming decisions to emerge from a bounded evidence accumulation process so that response times (RTs) become predictable. Here, we demonstrate a difficulty of SSMs that occurs when people are not forced to respond at once but are allowed to sample information sequentially: The decision maker might decide to delay the choice and terminate the accumulation process temporarily, a scenario not accounted for by the standard SSM approach. We developed several SSMs for predicting RTs from two independent samples of an electroencephalography (EEG) and a functional magnetic resonance imaging (fMRI) study. In these studies, participants bought or rejected fictitious stocks based on sequentially presented cues and were free to respond at any time. Standard SSM implementations did not describe RT distributions adequately. However, by adding a mechanism for postponing decisions to the model we obtained an accurate fit to the data. Time-frequency analysis of EEG data revealed alternating states of de- and increasing oscillatory power in beta-band frequencies (14-30 Hz), indicating that responses were repeatedly prepared and inhibited and thus lending further support for the existence of a decision not to decide. Finally, the extended model accounted for the results of an adapted version of our paradigm in which participants had to press a button for sampling more information. Our results show how computational modeling of decisions and RTs support a deeper understanding of the hidden dynamics in cognition.
| S-EPMC3814623 | biostudies-literature