Browse
Submit Data
Databases
API
Help

Dataset Information

20 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Learning with reinforcement prediction errors in a model of the Drosophila mushroom body.

ABSTRACT: Effective decision making in a changing environment demands that accurate predictions are learned about decision outcomes. In Drosophila, such learning is orchestrated in part by the mushroom body, where dopamine neurons signal reinforcing stimuli to modulate plasticity presynaptic to mushroom body output neurons. Building on previous mushroom body models, in which dopamine neurons signal absolute reinforcement, we propose instead that dopamine neurons signal reinforcement prediction errors by utilising feedback reinforcement predictions from output neurons. We formulate plasticity rules that minimise prediction errors, verify that output neurons learn accurate reinforcement predictions in simulations, and postulate connectivity that explains more physiological observations than an experimentally constrained model. The constrained and augmented models reproduce a broad range of conditioning and blocking experiments, and we demonstrate that the absence of blocking does not imply the absence of prediction error dependent learning. Our results provide five predictions that can be tested using established experimental methods.

SUBMITTER: Bennett JEM

PROVIDER: S-EPMC8105414 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Frontal theta links prediction errors to behavioral adaptation in reinforcement learning.

Project description:Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single-trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Mediofrontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single-trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations, with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.

| S-EPMC2818688 | biostudies-other

Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning.

Project description:Decision-making in the presence of other competitive intelligent agents is fundamental for social and economic behavior. Such decisions require agents to behave strategically, where in addition to learning about the rewards and punishments available in the environment, they also need to anticipate and respond to actions of others competing for the same rewards. However, whereas we know much about strategic learning at both theoretical and behavioral levels, we know relatively little about the underlying neural mechanisms. Here, we show using a multi-strategy competitive learning paradigm that strategic choices can be characterized by extending the reinforcement learning (RL) framework to incorporate agents' beliefs about the actions of their opponents. Furthermore, using this characterization to generate putative internal values, we used model-based functional magnetic resonance imaging to investigate neural computations underlying strategic learning. We found that the distinct notions of prediction errors derived from our computational model are processed in a partially overlapping but distinct set of brain regions. Specifically, we found that the RL prediction error was correlated with activity in the ventral striatum. In contrast, activity in the ventral striatum, as well as the rostral anterior cingulate (rACC), was correlated with a previously uncharacterized belief-based prediction error. Furthermore, activity in rACC reflected individual differences in degree of engagement in belief learning. These results suggest a model of strategic behavior where learning arises from interaction of dissociable reinforcement and belief-based inputs.

| S-EPMC3277161 | biostudies-literature

Project description:Humans can acquire knowledge of new motor behavior via different forms of learning. The two forms most commonly studied have been the development of internal models based on sensory-prediction errors (error-based learning) and success-based feedback (reinforcement learning). Human behavioral studies suggest these are distinct learning processes, though the neurophysiological mechanisms that are involved have not been characterized. Here, we evaluated physiological markers from the cerebellum and the primary motor cortex (M1) using noninvasive brain stimulations while healthy participants trained finger-reaching tasks. We manipulated the extent to which subjects rely on error-based or reinforcement by providing either vector or binary feedback about task performance. Our results demonstrated a double dissociation where learning the task mainly via error-based mechanisms leads to cerebellar plasticity modifications but not long-term potentiation (LTP)-like plasticity changes in M1; while learning a similar action via reinforcement mechanisms elicited M1 LTP-like plasticity but not cerebellar plasticity changes. Our findings indicate that learning complex motor behavior is mediated by the interplay of different forms of learning, weighing distinct neural mechanisms in M1 and the cerebellum. Our study provides insights for designing effective interventions to enhance human motor learning.

| S-EPMC6887949 | biostudies-literature

Drosophila Netrin-B controls mushroom body axon extension and regulates courtship-associated learning and memory of a Drosophila fragile X syndrome model.

Project description:Mushroom body (MB) is a prominent structure essential for olfactory learning and memory in the Drosophila brain. The development of the MB involves the appropriate guidance of axon lobes and sister axon branches. Appropriate guidance that accurately shapes MB development requires the integration of various guidance cues provided by a series of cell types, which guide axons to reach their final positions within the MB neuropils. Netrins are axonal guidance molecules that are conserved regulators of embryonic nerve cord patterning. However, whether they contribute to MB morphogenesis has not yet been evaluated. Here, we find that Netrin-B (NetB) is highly expressed in the MB lobes, regulating lobe length through genetic interactions with the receptors Frazzled and Uncoordinated-5 from 24 h after pupal formation onwards. We observe that overexpression of NetB causes severe β lobe fusion in the MB, which is similar to the MB defects seen in the Drosophila model of fragile X syndrome (FXS). Our results further show that fragile-X mental retardation protein FMRP inhibits the translational activity of human ortholog Netrin-1 (NTN1). Knock-down of NetB significantly rescues the MB defects and ameliorates deficits in the learning and memory in FXS model Drosophila. These results indicate a critical role for NetB in MB lobe extension and identify NetB as a novel target of FMRP which contributes to learning and memory.

| S-EPMC6540430 | biostudies-literature

Functional architecture of reward learning in mushroom body extrinsic neurons of larval Drosophila.

Project description:The brain adaptively integrates present sensory input, past experience, and options for future action. The insect mushroom body exemplifies how a central brain structure brings about such integration. Here we use a combination of systematic single-cell labeling, connectomics, transgenic silencing, and activation experiments to study the mushroom body at single-cell resolution, focusing on the behavioral architecture of its input and output neurons (MBINs and MBONs), and of the mushroom body intrinsic APL neuron. Our results reveal the identity and morphology of almost all of these 44 neurons in stage 3 Drosophila larvae. Upon an initial screen, functional analyses focusing on the mushroom body medial lobe uncover sparse and specific functions of its dopaminergic MBINs, its MBONs, and of the GABAergic APL neuron across three behavioral tasks, namely odor preference, taste preference, and associative learning between odor and taste. Our results thus provide a cellular-resolution study case of how brains organize behavior.

| S-EPMC5856778 | biostudies-literature

Aversive Learning and Appetitive Motivation Toggle Feed-Forward Inhibition in the Drosophila Mushroom Body.

Project description:In Drosophila, negatively reinforcing dopaminergic neurons also provide the inhibitory control of satiety over appetitive memory expression. Here we show that aversive learning causes a persistent depression of the conditioned odor drive to two downstream feed-forward inhibitory GABAergic interneurons of the mushroom body, called MVP2, or mushroom body output neuron (MBON)-?1pedc>?/?. However, MVP2 neuron output is only essential for expression of short-term aversive memory. Stimulating MVP2 neurons preferentially inhibits the odor-evoked activity of avoidance-directing MBONs and odor-driven avoidance behavior, whereas their inhibition enhances odor avoidance. In contrast, odor-evoked activity of MVP2 neurons is elevated in hungry flies, and their feed-forward inhibition is required for expression of appetitive memory at all times. Moreover, imposing MVP2 activity promotes inappropriate appetitive memory expression in food-satiated flies. Aversive learning and appetitive motivation therefore toggle alternate modes of a common feed-forward inhibitory MVP2 pathway to promote conditioned odor avoidance or approach.

| S-EPMC4893166 | biostudies-literature

Serotonin receptor 5-HT7 in Drosophila mushroom body neurons mediates larval appetitive olfactory learning.

Project description:Serotonin (5-HT) and dopamine are critical neuromodulators known to regulate a range of behaviors in invertebrates and mammals, such as learning and memory. Effects of both serotonin and dopamine are mediated largely through their downstream G-protein coupled receptors through cAMP-PKA signaling. While the role of dopamine in olfactory learning in Drosophila is well described, the function of serotonin and its downstream receptors on Drosophila olfactory learning remain largely unexplored. In this study we show that the output of serotonergic neurons, possibly through points of synaptic contacts on the mushroom body (MB), is essential for training during olfactory associative learning in Drosophila larvae. Additionally, we demonstrate that the regulation of olfactory associative learning by serotonin is mediated by its downstream receptor (d5-HT7) in a cAMP-dependent manner. We show that d5-HT7 expression specifically in the MB, an anatomical structure essential for olfactory learning in Drosophila, is critical for olfactory associative learning. Importantly our work shows that spatio-temporal restriction of d5-HT7 expression to the MB is sufficient to rescue olfactory learning deficits in a d5-HT7 null larvae. In summary, our results establish a critical, and previously unknown, role of d5-HT7 in olfactory learning.

| S-EPMC7718245 | biostudies-literature

Localized inhibition in the Drosophila mushroom body.

Project description:Many neurons show compartmentalized activity, in which activity does not spread readily across the cell, allowing input and output to occur locally. However, the functional implications of compartmentalized activity for the wider neural circuit are often unclear. We addressed this problem in the Drosophila mushroom body, whose principal neurons, Kenyon cells, receive feedback inhibition from a non-spiking interneuron called the anterior paired lateral (APL) neuron. We used local stimulation and volumetric calcium imaging to show that APL inhibits Kenyon cells' dendrites and axons, and that both activity in APL and APL's inhibitory effect on Kenyon cells are spatially localized (the latter somewhat less so), allowing APL to differentially inhibit different mushroom body compartments. Applying these results to the Drosophila hemibrain connectome predicts that individual Kenyon cells inhibit themselves via APL more strongly than they inhibit other individual Kenyon cells. These findings reveal how cellular physiology and detailed network anatomy can combine to influence circuit function.

| S-EPMC7541083 | biostudies-literature

States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.

Project description:Reinforcement learning (RL) uses sequential experience with situations ("states") and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model. A state prediction error (SPE) plays a central role, reporting discrepancies between the current model and the observed state transitions. Using functional magnetic resonance imaging in humans solving a probabilistic Markov decision task, we found the neural signature of an SPE in the intraparietal sulcus and lateral prefrontal cortex, in addition to the previously well-characterized RPE in the ventral striatum. This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.

| S-EPMC2895323 | biostudies-other

Transcription profiling of Drosophila mushroom body neurons

Project description:Genomic analysis of axon pruning in Drosophila mushroom body neurons identifies the RNA-binding protein Boule as a negative regulator; This SuperSeries is composed of the following subset Series:; GSE10012: Timecourse: MB neurons at the onset and early steps of axon pruning; GSE10013: EcR-dependent gene expression in MB neurons at the onset of axon pruning Experiment Overall Design: Refer to individual Series

2008-06-18 | E-GEOD-10014 | biostudies-arrayexpress

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data