A Neuro-computational Account of Arbitration between Choice Imitation and Goal Emulation during Human Observational Learning.
Ontology highlight
ABSTRACT:
SUBMITTER: Charpentier CJ
PROVIDER: S-EPMC7244377 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:A state of pathological uncertainty about environmental regularities might represent a key step in the pathway to psychotic illness. Early psychosis can be investigated in healthy volunteers under ketamine, an NMDA receptor antagonist. Here, we explored the effects of ketamine on contingency learning using a placebo-controlled, double-blind, crossover design. During functional magnetic resonance imaging, participants performed an instrumental learning task, in which cue-outcome contingencies were probabilistic and reversed between blocks. Bayesian model comparison indicated that in such an unstable environment, reinforcement learning parameters are downregulated depending on confidence level, an adaptive mechanism that was specifically disrupted by ketamine administration. Drug effects were underpinned by altered neural activity in a fronto-parietal network, which reflected the confidence-based shift to exploitation of learned contingencies. Our findings suggest that an early characteristic of psychosis lies in a persistent doubt that undermines the stabilization of behavioral policy resulting in a failure to exploit regularities in the environment.
| S-EPMC5414075 | biostudies-literature
Project description:The influence of mood on choices is a well-established but poorly understood phenomenon. Here, we suggest a three-fold neuro-computational account: (1) the integration of positive and negative events over time induce mood fluctuations, (2) which are underpinned by variations in the baseline activities of critical brain valuation regions, (3) which in turn modulate the relative weights assigned to key dimensions of choice options. We validate this model in healthy participants, using feedback in a quiz task to induce mood fluctuations, and a choice task (accepting vs. declining a motor challenge) to reveal their effects. Using fMRI, we demonstrate the pivotal role of the ventromedial prefrontal cortex and anterior insula, in which baseline activities respectively increase and decrease with theoretical mood level and respectively enhance the weighting of potential gains and losses during decision making. The same mechanisms might explain how decisions are biased in mood disorders at longer timescales.
| S-EPMC5919935 | biostudies-literature
Project description:A cornerstone of human intelligence is the ability to flexibly adjust our cognition and behavior as our goals change. For instance, achieving some goals requires efficiency, while others require caution. Adapting to these changing goals require corresponding adjustments in cognitive control (e.g., levels of attention, response thresholds). However, adjusting our control to meet new goals comes at a cost: we are better at achieving a goal in isolation than when transitioning between goals. The source of these control adjustment costs remains poorly understood, and the bulk of our understanding of such costs comes from settings in which participants transition between discrete task sets, rather than performance goals. Across four experiments, we show that adjustments in continuous control states incur a performance cost, and that a dynamical systems model can explain the source of these costs. Participants performed a single cognitively demanding task under varying performance goals (e.g., to be fast or to be accurate). We modeled control allocation to include a dynamic process of adjusting from one's current control state to a target state for a given performance goal. By incorporating inertia into this adjustment process, our model accounts for our empirical findings that people under-shoot their target control state more (i.e., exhibit larger adjustment costs) when (a) goals switch rather than remain fixed over a block (Study 1); (b) target control states are more distant from one another (Study 2); (c) less time is given to adjust to the new goal (Study 3); and (d) when anticipating having to switch goals more frequently (Study 4). Our findings characterize the costs of adjusting control to meet changing goals, and show that these costs can emerge directly from cognitive control dynamics. In so doing, they shed new light on the sources of and constraints on flexibility in human goal-directed behavior.
| S-EPMC10473589 | biostudies-literature
Project description:Social media has become a modern arena for human life, with billions of daily users worldwide. The intense popularity of social media is often attributed to a psychological need for social rewards (likes), portraying the online world as a Skinner Box for the modern human. Yet despite such portrayals, empirical evidence for social media engagement as reward-based behavior remains scant. Here, we apply a computational approach to directly test whether reward learning mechanisms contribute to social media behavior. We analyze over one million posts from over 4000 individuals on multiple social media platforms, using computational models based on reinforcement learning theory. Our results consistently show that human behavior on social media conforms qualitatively and quantitatively to the principles of reward learning. Specifically, social media users spaced their posts to maximize the average rate of accrued social rewards, in a manner subject to both the effort cost of posting and the opportunity cost of inaction. Results further reveal meaningful individual difference profiles in social reward learning on social media. Finally, an online experiment (n = 176), mimicking key aspects of social media, verifies that social rewards causally influence behavior as posited by our computational account. Together, these findings support a reward learning account of social media engagement and offer new insights into this emergent mode of modern human behavior.
| S-EPMC7910435 | biostudies-literature
Project description:In nature, insects show impressive adaptation and learning capabilities. The proposed computational model takes inspiration from specific structures of the insect brain: after proposing key hypotheses on the direct involvement of the mushroom bodies (MBs) and on their neural organization, we developed a new architecture for motor learning to be applied in insect-like walking robots. The proposed model is a nonlinear control system based on spiking neurons. MBs are modeled as a nonlinear recurrent spiking neural network (SNN) with novel characteristics, able to memorize time evolutions of key parameters of the neural motor controller, so that existing motor primitives can be improved. The adopted control scheme enables the structure to efficiently cope with goal-oriented behavioral motor tasks. Here, a six-legged structure, showing a steady-state exponentially stable locomotion pattern, is exposed to the need of learning new motor skills: moving through the environment, the structure is able to modulate motor commands and implements an obstacle climbing procedure. Experimental results on a simulated hexapod robot are reported; they are obtained in a dynamic simulation environment and the robot mimicks the structures of Drosophila melanogaster.
| S-EPMC5340754 | biostudies-literature
Project description:Decision making requires integrating knowledge gathered from personal experiences with advice from others. The neural underpinnings of the process of arbitrating between information sources has not been fully elucidated. In this study, we formalized arbitration as the relative precision of predictions, afforded by each learning system, using hierarchical Bayesian modeling. In a probabilistic learning task, participants predicted the outcome of a lottery using recommendations from a more informed advisor and/or self-sampled outcomes. Decision confidence, as measured by the number of points participants wagered on their predictions, varied with our definition of arbitration as a ratio of precisions. Functional neuroimaging demonstrated that arbitration signals were independent of decision confidence and involved modality-specific brain regions. Arbitrating in favor of self-gathered information activated the dorsolateral prefrontal cortex and the midbrain, whereas arbitrating in favor of social information engaged the ventromedial prefrontal cortex and the amygdala. These findings indicate that relative precision captures arbitration between social and individual learning systems at both behavioral and neural levels.
| S-EPMC7476763 | biostudies-literature
Project description:In the Stroop task, participants identify the print color of color words. The congruency effect is the observation that response times and errors are increased when the word and color are incongruent (e.g., the word "red" in green ink) relative to when they are congruent (e.g., "red" in red). The proportion congruent (PC) effect is the finding that congruency effects are reduced when trials are mostly incongruent rather than mostly congruent. This PC effect can be context-specific. For instance, if trials are mostly incongruent when presented in one location and mostly congruent when presented in another location, the congruency effect is smaller for the former location. Typically, PC effects are interpreted in terms of strategic control of attention in response to conflict, termed conflict adaptation or conflict monitoring. In the present manuscript, however, an episodic learning account is presented for context-specific proportion congruent (CSPC) effects. In particular, it is argued that context-specific contingency learning can explain part of the effect, and context-specific rhythmic responding can explain the rest. Both contingency-based and temporal-based learning can parsimoniously be conceptualized within an episodic learning framework. An adaptation of the Parallel Episodic Processing model is presented. This model successfully simulates CSPC effects, both for contingency-biased and contingency-unbiased (transfer) items. The same fixed-parameter model can explain a range of other findings from the learning, timing, binding, practice, and attentional control domains.
| S-EPMC5110540 | biostudies-literature
Project description:Healthy adults flexibly adapt their learning strategies to ongoing changes in uncertainty, a key feature of adaptive behaviour. However, the developmental trajectory of this ability is yet unknown, as developmental studies have not incorporated trial-to-trial variation in uncertainty in their analyses or models. To address this issue, we compared adolescents' and adults' trial-to-trial dynamics of uncertainty, learning rate, and exploration in two tasks that assess learning in noisy but otherwise stable environments. In an estimation task-which provides direct indices of trial-specific learning rate-both age groups reduced their learning rate over time, as self-reported uncertainty decreased. Accordingly, the estimation data in both groups was better explained by a Bayesian model with dynamic learning rate (Kalman filter) than by conventional reinforcement-learning models. Furthermore, adolescents' learning rates asymptoted at a higher level, reflecting an over-weighting of the most recent outcome, and the estimated Kalman-filter parameters suggested that this was due to an overestimation of environmental volatility. In a choice task, both age groups became more likely to choose the higher-valued option over time, but this increase in choice accuracy was smaller in the adolescents. In contrast to the estimation task, we found no evidence for a Bayesian expectation-updating process in the choice task, suggesting that estimation and choice tasks engage different learning processes. However, our modeling results of the choice task suggested that both age groups reduced their degree of exploration over time, and that the adolescents explored overall more than the adults. Finally, age-related differences in exploration parameters from fits to the choice data were mediated by participants' volatility parameter from fits to the estimation data. Together, these results suggest that adolescents overestimate the rate of environmental change, resulting in elevated learning rates and increased exploration, which may help understand developmental changes in learning and decision-making.
| S-EPMC7549782 | biostudies-literature
Project description:The current study harnessed the variability in infants' neural and behavioral responses as a novel method for evaluating the potential relations between motor system activation and social behavior. We used electroencephalography (EEG) to record neural activity as 7-month-old infants observed and responded to the actions of an experimenter. To determine whether motor system activation predicted subsequent imitation behavior, we assessed event-related desynchronization (ERD) at central sites during action observation as a function of subsequent behavior. Greater mu desynchronization over central sites was observed when infants subsequently reproduced the experimenter's goal than when they did not reproduce the goal and instead selected the nongoal object. We also found that mu desynchronization during action execution predicted the infants' later propensity to reproduce the experimenter's goal-directed behavior. These results provide the first evidence that motor system activation predicts the imitation of other individuals' goals during infancy.
| S-EPMC4875827 | biostudies-literature
Project description:Learning to predict action outcomes in morally conflicting situations is essential for social decision-making but poorly understood. Here we tested which forms of Reinforcement Learning Theory capture how participants learn to choose between self-money and other-shocks, and how they adapt to changes in contingencies. We find choices were better described by a reinforcement learning model based on the current value of separately expected outcomes than by one based on the combined historical values of past outcomes. Participants track expected values of self-money and other-shocks separately, with the substantial individual difference in preference reflected in a valuation parameter balancing their relative weight. This valuation parameter also predicted choices in an independent costly helping task. The expectations of self-money and other-shocks were biased toward the favored outcome but fMRI revealed this bias to be reflected in the ventromedial prefrontal cortex while the pain-observation network represented pain prediction errors independently of individual preferences.
| S-EPMC9988878 | biostudies-literature