Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Bayesian cue integration as a developmental outcome of reward mediated learning.

ABSTRACT: Average human behavior in cue combination tasks is well predicted by bayesian inference models. As this capability is acquired over developmental timescales, the question arises, how it is learned. Here we investigated whether reward dependent learning, that is well established at the computational, behavioral, and neuronal levels, could contribute to this development. It is shown that a model free reinforcement learning algorithm can indeed learn to do cue integration, i.e. weight uncertain cues according to their respective reliabilities and even do so if reliabilities are changing. We also consider the case of causal inference where multimodal signals can originate from one or multiple separate objects and should not always be integrated. In this case, the learner is shown to develop a behavior that is closest to bayesian model averaging. We conclude that reward mediated learning could be a driving force for the development of cue integration and causal inference.

SUBMITTER: Weisswange TH

PROVIDER: S-EPMC3130032 | biostudies-literature | 2011

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Bayesian cue integration as a developmental outcome of reward mediated learning.

Weisswange Thomas H TH Rothkopf Constantin A CA Rodemann Tobias T Triesch Jochen J

PloS one 20110705 7

Average human behavior in cue combination tasks is well predicted by bayesian inference models. As this capability is acquired over developmental timescales, the question arises, how it is learned. Here we investigated whether reward dependent learning, that is well established at the computational, behavioral, and neuronal levels, could contribute to this development. It is shown that a model free reinforcement learning algorithm can indeed learn to do cue integration, i.e. weight uncertain cue ...[more]

PMID: 21750717

Similar Datasets

Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning.

Project description:What neural changes underlie individual differences in goal-directed learning? The lateral amygdala (LA) is important for assigning emotional and motivational significance to discrete environmental cues, including those that signal rewarding events. Recognizing that a cue predicts a reward enhances an animal's ability to acquire that reward; however, the cellular and synaptic mechanisms that underlie cue-reward learning are unclear. Here we show that marked changes in both cue-induced neuronal firing and input-specific synaptic strength occur with the successful acquisition of a cue-reward association within a single training session. We performed both in vivo and ex vivo electrophysiological recordings in the LA of rats trained to self-administer sucrose. We observed that reward-learning success increased in proportion to the number of amygdala neurons that responded phasically to a reward-predictive cue. Furthermore, cue-reward learning induced an AMPA (alpha-amino-3-hydroxy-5-methyl-isoxazole propionic acid)-receptor-mediated increase in the strength of thalamic, but not cortical, synapses in the LA that was apparent immediately after the first training session. The level of learning attained by individual subjects was highly correlated with the degree of synaptic strength enhancement. Importantly, intra-LA NMDA (N-methyl-d-aspartate)-receptor blockade impaired reward-learning performance and attenuated the associated increase in synaptic strength. These findings provide evidence of a connection between LA synaptic plasticity and cue-reward learning, potentially representing a key mechanism underlying goal-directed behaviour.

| S-EPMC2759353 | biostudies-literature

Structure Learning in Bayesian Sensorimotor Integration.

Project description:Previous studies have shown that sensorimotor processing can often be described by Bayesian learning, in particular the integration of prior and feedback information depending on its degree of reliability. Here we test the hypothesis that the integration process itself can be tuned to the statistical structure of the environment. We exposed human participants to a reaching task in a three-dimensional virtual reality environment where we could displace the visual feedback of their hand position in a two dimensional plane. When introducing statistical structure between the two dimensions of the displacement, we found that over the course of several days participants adapted their feedback integration process in order to exploit this structure for performance improvement. In control experiments we found that this adaptation process critically depended on performance feedback and could not be induced by verbal instructions. Our results suggest that structural learning is an important meta-learning component of Bayesian sensorimotor integration.

| S-EPMC4549275 | biostudies-literature

Effects of subconscious and conscious emotions on human cue-reward association learning.

Project description:Life demands that we adapt our behaviour continuously in situations in which much of our incoming information is emotional and unrelated to our immediate behavioural goals. Such information is often processed without our consciousness. This poses an intriguing question of whether subconscious exposure to irrelevant emotional information (e.g. the surrounding social atmosphere) affects the way we learn. Here, we addressed this issue by examining whether the learning of cue-reward associations changes when an emotional facial expression is shown subconsciously or consciously prior to the presentation of a reward-predicting cue. We found that both subconscious (0.027?s and 0.033?s) and conscious (0.047?s) emotional signals increased the rate of learning, and this increase was smallest at the border of conscious duration (0.040?s). These data suggest not only that the subconscious and conscious processing of emotional signals enhances value-updating in cue-reward association learning, but also that the computational processes underlying the subconscious enhancement is at least partially dissociable from its conscious counterpart.

| S-EPMC4329552 | biostudies-literature

A self-initiated cue-reward learning procedure for neural recording in rodents.

Project description:BackgroundSingle-unit recording in Pavlovian conditioning tasks requires the use of within-subject designs as well as sampling a considerable number of trials per trial type and session, which increases the total trial count. Pavlovian conditioning, on the other hand, requires a long average intertrial interval (ITI) relative to cue duration for cue-specific learning to occur. These requirements combined can make the session duration unfeasibly long.New methodTo circumvent this issue, we developed a self-initiated variant of the Pavlovian magazine-approach procedure in rodents. Unlike the standard procedure, where the animals passively receive the trials, the self-initiated procedure grants animals agency to self-administer and self-pace trials from a predetermined, pseudorandomized list. Critically, whereas in the standard procedure the typical ITI is in the order of minutes, our procedure uses a much shorter ITI (10 s).ResultsDespite such a short ITI, discrimination learning in the self-initiated procedure is comparable to that observed in the standard procedure with a typical ITI, and superior to that observed in the standard procedure with an equally short ITI.Comparison with existing method(s)The self-initiated procedure permits delivering 100 trials in a ∼1-h session, almost doubling the number of trials safely attainable over that period with the standard procedure.ConclusionsThe self-initiated procedure enhances the collection of neural correlates of cue-reward learning while producing good discrimination performance. Other advantages for neural recording studies include ensuring that at the start of each trial the animal is engaged, attentive and in the same location within the conditioning chamber.

| S-EPMC7214222 | biostudies-literature

Licking-induced synchrony in the taste-reward circuit improves cue discrimination during learning.

Project description:Animals learn which foods to ingest and which to avoid. Despite many studies, the electrophysiological correlates underlying this behavior at the gustatory-reward circuit level remain poorly understood. For this reason, we measured the simultaneous electrical activity of neuronal ensembles in the orbitofrontal cortex, insular cortex, amygdala, and nucleus accumbens while rats licked for taste cues and learned to perform a taste discrimination go/no-go task. This study revealed that rhythmic licking entrains the activity in all these brain regions, suggesting that the animal's licking acts as an "internal clock signal" against which single spikes can be synchronized. That is, as animals learned a go/no-go task, there were increases in the number of licking coherent neurons as well as synchronous spiking between neuron pairs from different brain regions. Moreover, a subpopulation of gustatory cue-selective neurons that fired in synchrony with licking exhibited a greater ability to discriminate among tastants than nonsynchronized neurons. This effect was seen in all four recorded areas and increased markedly after learning, particularly after the cue was delivered and before the animals made a movement to obtain an appetitive or aversive tastant. Overall, these results show that, throughout a large segment of the taste-reward circuit, appetitive and aversive associative learning improves spike-timing precision, suggesting that proficiency in solving a taste discrimination go/no-go task requires licking-induced neural ensemble synchronous activity.

| S-EPMC2831544 | biostudies-literature

Acetylcholine is released in the basolateral amygdala in response to predictors of reward and enhances the learning of cue-reward contingency.

Project description:The basolateral amygdala (BLA) is critical for associating initially neutral cues with appetitive and aversive stimuli and receives dense neuromodulatory acetylcholine (ACh) projections. We measured BLA ACh signaling and activity of neurons expressing CaMKIIα (a marker for glutamatergic principal cells) in mice during cue-reward learning using a fluorescent ACh sensor and calcium indicators. We found that ACh levels and nucleus basalis of Meynert (NBM) cholinergic terminal activity in the BLA (NBM-BLA) increased sharply in response to reward-related events and shifted as mice learned the cue-reward contingency. BLA CaMKIIα neuron activity followed reward retrieval and moved to the reward-predictive cue after task acquisition. Optical stimulation of cholinergic NBM-BLA terminal fibers led to a quicker acquisition of the cue-reward contingency. These results indicate BLA ACh signaling carries important information about salient events in cue-reward learning and provides a framework for understanding how ACh signaling contributes to shaping BLA responses to emotional stimuli.

| S-EPMC7529459 | biostudies-literature

Learning multisensory cue integration: A computational model of crossmodal synaptic plasticity enables reliability-based cue weighting by capturing stimulus statistics.

Project description:The brain forms unified, coherent, and accurate percepts of events occurring in the environment by integrating information from multiple senses through the process of multisensory integration. The neural mechanisms underlying this process, its development and its maturation in a multisensory environment are yet to be properly understood. Numerous psychophysical studies suggest that the multisensory cue integration process follows the principle of Bayesian estimation, where the contributions of individual sensory modalities are proportional to the relative reliabilities of the different sensory stimuli. In this article I hypothesize that experience dependent crossmodal synaptic plasticity may be a plausible mechanism underlying development of multisensory cue integration. I test this hypothesis via a computational model that implements Bayesian multisensory cue integration using reliability-based cue weighting. The model uses crossmodal synaptic plasticity to capture stimulus statistics within synaptic weights that are adapted to reflect the relative reliabilities of the participating stimuli. The model is embodied in a simulated robotic agent that learns to localize an audio-visual target by integrating spatial location cues extracted from of auditory and visual sensory modalities. Results of multiple randomized target localization trials in simulation indicate that the model is able to learn modality-specific synaptic weights proportional to the relative reliabilities of the auditory and visual stimuli. The proposed model with learned synaptic weights is also compared with a maximum-likelihood estimation model for cue integration via regression analysis. Results indicate that the proposed model reflects maximum-likelihood estimation.

| S-EPMC9393257 | biostudies-literature

Optimal cue integration in ants.

Project description:In situations with redundant or competing sensory information, humans have been shown to perform cue integration, weighting different cues according to their certainty in a quantifiably optimal manner. Ants have been shown to merge the directional information available from their path integration (PI) and visual memory, but as yet it is not clear that they do so in a way that reflects the relative certainty of the cues. In this study, we manipulate the variance of the PI home vector by allowing ants (Cataglyphis velox) to run different distances and testing their directional choice when the PI vector direction is put in competition with visual memory. Ants show progressively stronger weighting of their PI direction as PI length increases. The weighting is quantitatively predicted by modelling the expected directional variance of home vectors of different lengths and assuming optimal cue integration. However, a subsequent experiment suggests ants may not actually compute an internal estimate of the PI certainty, but are using the PI home vector length as a proxy.

| S-EPMC4614770 | biostudies-literature

Global Cue Inconsistency Diminishes Learning of Cue Validity.

Project description:In daily life, we make decisions that are associated with probabilistic outcomes (e.g., the chance of rain today). People search for and utilize information that validly predicts an outcome (i.e., we look for dark clouds to indicate the possibility of rain). In the current study (N = 107), we present a two-stage learning task that examines how participants learn and utilize predictive information within a probabilistic learning environment. In the first stage, participants select one of three cues that gives predictive information about the outcome of the second stage. Participants then use this information to predict the outcome in stage two, for which they receive feedback. Critically, only one of the three cues in stage one gives valid predictive information about the outcome in stage two. Participants must differentiate the valid from non-valid cues and select this cue on subsequent trials in order to inform their prediction of the outcome in stage two. The validity of this predictive information, however, is reinforced with varying levels of probabilistic feedback (i.e., 75, 85, 95, 100%). A second manipulation involved changing the consistency of the predictive information in stage one and the outcome in stage two. The results show that participants, with higher levels of probabilistic feedback, learned to utilize the valid cue. In inconsistent task conditions, however, participants were significantly less successful in utilizing higher validity cues. We interpret this result as implying that learning in probabilistic categorization is based on developing a representation of the task that allows for goal-directed action.

| S-EPMC5104735 | biostudies-literature

Mesolimbic dopamine projections mediate cue-motivated reward seeking but not reward retrieval in rats.

Project description:Efficient foraging requires an ability to coordinate discrete reward-seeking and reward-retrieval behaviors. We used pathway-specific chemogenetic inhibition to investigate how rats' mesolimbic and mesocortical dopamine circuits contribute to the expression and modulation of reward seeking and retrieval. Inhibiting ventral tegmental area dopamine neurons disrupted the tendency for reward-paired cues to motivate reward seeking, but spared their ability to increase attempts to retrieve reward. Similar effects were produced by inhibiting dopamine inputs to nucleus accumbens, but not medial prefrontal cortex. Inhibiting dopamine neurons spared the suppressive effect of reward devaluation on reward seeking, an assay of goal-directed behavior. Attempts to retrieve reward persisted after devaluation, indicating they were habitually performed as part of a fixed action sequence. Our findings show that complete bouts of reward seeking and retrieval are behaviorally and neurally dissociable from bouts of reward seeking without retrieval. This dichotomy may prove useful for uncovering mechanisms of maladaptive behavior.

| S-EPMC6548499 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data