Dissociating Representations of Time and Number in Reinforcement-Rate Learning by Deletion of the GluA1 AMPA Receptor Subunit in Mice.
Ontology highlight
ABSTRACT:
SUBMITTER: Austen JM
PROVIDER: S-EPMC7883001 | biostudies-literature | 2021 Feb
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Dysfunction of the glutamate α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptor GluA1 subunit and deficits in synaptic plasticity are implicated in schizophrenia and sleep and circadian rhythm disruption. To investigate the role of GluA1 in circadian and sleep behaviour, we used wheel-running, passive-infrared, and video-based home-cage activity monitoring to assess daily rest-activity profiles of GluA1-knockout mice (Gria1-/-). We showed that these mice displayed various circadian abnormalities, including misaligned, fragmented, and more variable rest-activity patterns. In addition, they showed heightened, but transient, behavioural arousal to light→dark and dark→light transitions, as well as attenuated nocturnal-light-induced activity suppression (negative masking). In the hypothalamic suprachiasmatic nuclei (SCN), nocturnal-light-induced cFos signals (a molecular marker of neuronal activity in the preceding ~1-2 h) were attenuated, indicating reduced light sensitivity in the SCN. However, there was no change in the neuroanatomical distribution of expression levels of two neuropeptides-vasoactive intestinal peptide (VIP) and arginine vasopressin (AVP)-differentially expressed in the core (ventromedial) vs. shell (dorsolateral) SCN subregions and both are known to be important for neuronal synchronisation within the SCN and circadian rhythmicity. In the motor cortex (area M1/M2), there was increased inter-individual variability in cFos levels during the evening period, mirroring the increased inter-individual variability in locomotor activity under nocturnal light. Finally, in the spontaneous odour recognition task GluA1 knockouts' short-term memory was impaired due to enhanced attention to the recently encountered familiar odour. These abnormalities due to altered AMPA-receptor-mediated signalling resemble and may contribute to sleep and circadian rhythm disruption and attentional deficits in different modalities in schizophrenia.
| S-EPMC8593011 | biostudies-literature
Project description:From both functional and biological considerations, it is widely believed that action production, planning, and goal-oriented behaviors supported by the frontal cortex are organized hierarchically [Fuster (1991); Koechlin, E., Ody, C., & Kouneiher, F. (2003). Neuroscience: The architecture of cognitive control in the human prefrontal cortex. Science, 424, 1181-1184; Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the structure of behavior. New York: Holt]. However, the nature of the different levels of the hierarchy remains unclear, and little attention has been paid to the origins of such a hierarchy. We address these issues through biologically-inspired computational models that develop representations through reinforcement learning. We explore several different factors in these models that might plausibly give rise to a hierarchical organization of representations within the PFC, including an initial connectivity hierarchy within PFC, a hierarchical set of connections between PFC and subcortical structures controlling it, and differential synaptic plasticity schedules. Simulation results indicate that architectural constraints contribute to the segregation of different types of representations, and that this segregation facilitates learning. These findings are consistent with the idea that there is a functional hierarchy in PFC, as captured in our earlier computational models of PFC function and a growing body of empirical data.
| S-EPMC2783795 | biostudies-literature
Project description:Memory capacity (MC) refers to the number of elements one can maintain for a short retention interval. The molecular mechanisms underlying MC are unexplored. We have recently reported that mice as well as humans have a limited MC, which is reduced by hippocampal lesions. Here, we addressed the molecular mechanisms supporting MC. GluA1 AMPA-receptors (AMPA-R) mediate the majority of fast excitatory synaptic transmission in the brain and are critically involved in memory. Phosphorylation of GluA1 at serine residues S831 and S845 is promoted by CaMKII and PKA, respectively, and regulates AMPA-R function in memory duration. We hypothesized that AMPA-R phosphorylation may also be a key plastic process for supporting MC because it occurs in a few minutes, and potentiates AMPA-R ion channel function. Here, we show that knock-in mutant mice that specifically lack both of S845 and S831 phosphorylation sites on the GluA1 subunit had reduced MC in two different behavioral tasks specifically designed to assess MC in mice. This demonstrated a causal link between AMPA-R phosphorylation and MC. We then showed that information load regulates AMPA-R phosphorylation within the hippocampus, and that an overload condition associated with impaired memory is paralleled by a lack of AMPA-R phosphorylation. Accordingly, we showed that in conditions of high load, but not of low load, the pharmacological inhibition of the NMDA-CaMKII-PKA pathways within the hippocampus prevents memory as well as associated AMPA-R phosphorylation. These data provide the first identified molecular mechanism that regulates MC.
| S-EPMC4425615 | biostudies-literature
Project description:Temporal-difference (TD) algorithms have been proposed as models of reinforcement learning (RL). We examine two issues of distributed representation in these TD algorithms: distributed representations of belief and distributed discounting factors. Distributed representation of belief allows the believed state of the world to distribute across sets of equivalent states. Distributed exponential discounting factors produce hyperbolic discounting in the behavior of the agent itself. We examine these issues in the context of a TD RL model in which state-belief is distributed over a set of exponentially-discounting "micro-Agents", each of which has a separate discounting factor (gamma). Each microAgent maintains an independent hypothesis about the state of the world, and a separate value-estimate of taking actions within that hypothesized state. The overall agent thus instantiates a flexible representation of an evolving world-state. As with other TD models, the value-error (delta) signal within the model matches dopamine signals recorded from animals in standard conditioning reward-paradigms. The distributed representation of belief provides an explanation for the decrease in dopamine at the conditioned stimulus seen in overtrained animals, for the differences between trace and delay conditioning, and for transient bursts of dopamine seen at movement initiation. Because each microAgent also includes its own exponential discounting factor, the overall agent shows hyperbolic discounting, consistent with behavioral experiments.
| S-EPMC2760757 | biostudies-literature
Project description:It has been proposed that the sensorimotor system uses a loss (cost) function to evaluate potential movements in the presence of random noise. Here we test this idea in the context of both error-based and reinforcement-based learning. In a reaching task, we laterally shifted a cursor relative to true hand position using a skewed probability distribution. This skewed probability distribution had its mean and mode separated, allowing us to dissociate the optimal predictions of an error-based loss function (corresponding to the mean of the lateral shifts) and a reinforcement-based loss function (corresponding to the mode). We then examined how the sensorimotor system uses error feedback and reinforcement feedback, in isolation and combination, when deciding where to aim the hand during a reach. We found that participants compensated differently to the same skewed lateral shift distribution depending on the form of feedback they received. When provided with error feedback, participants compensated based on the mean of the skewed noise. When provided with reinforcement feedback, participants compensated based on the mode. Participants receiving both error and reinforcement feedback continued to compensate based on the mean while repeatedly missing the target, despite receiving auditory, visual and monetary reinforcement feedback that rewarded hitting the target. Our work shows that reinforcement-based and error-based learning are separable and can occur independently. Further, when error and reinforcement feedback are in conflict, the sensorimotor system heavily weights error feedback over reinforcement feedback.
| S-EPMC5550011 | biostudies-other
Project description:In computer science, reinforcement learning is a powerful framework with which artificial agents can learn to maximize their performance for any given Markov decision process (MDP). Advances over the last decade, in combination with deep neural networks, have enjoyed performance advantages over humans in many difficult task settings. However, such frameworks perform far less favorably when evaluated in their ability to generalize or transfer representations across different tasks. Existing algorithms that facilitate transfer typically are limited to cases in which the transition function or the optimal policy is portable to new contexts, but achieving "deep transfer" characteristic of human behavior has been elusive. Such transfer typically requires discovery of abstractions that permit analogical reuse of previously learned representations to superficially distinct tasks. Here, we demonstrate that abstractions that minimize error in predictions of reward outcomes generalize across tasks with different transition and reward functions. Such reward-predictive representations compress the state space of a task into a lower dimensional representation by combining states that are equivalent in terms of both the transition and reward functions. Because only state equivalences are considered, the resulting state representation is not tied to the transition and reward functions themselves and thus generalizes across tasks with different reward and transition functions. These results contrast with those using abstractions that myopically maximize reward in any given MDP and motivate further experiments in humans and animals to investigate if neural and cognitive systems involved in state representation perform abstractions that facilitate such equivalence relations.
| S-EPMC7591094 | biostudies-literature
Project description:The dynamic trafficking of AMPA receptors (AMPARs), which enables the endocytosis, recycling, and exocytosis of AMPARs, is crucial for synaptic plasticity. Endophilin2, which directly interacts with the GluA1 subunit of AMPARs, plays an important role in AMPAR endocytosis. Collapsin response mediator protein 2 (CRMP2) promotes the maturation of the dendritic spine and can transfer AMPARs to the membrane. Although the mechanisms of AMPAR endocytosis and exocytosis are well known, the exact molecular mechanisms underlying AMPAR recycling remain unclear. Here, we report a unique interaction between CRMP2 and endophilin2. Our results showed that overexpression of CRMP2 and endophilin2 increased the amplitude and frequency of miniature excitatory synaptic currents (mEPSCs) and modestly enhanced AMPAR levels in hippocampal neurons. Furthermore, the CRMP2 and endophilin2 overexpression phenotype failed to occur when the interaction between these two proteins was inhibited. Further analysis revealed that this interaction was regulated by CRMP2 phosphorylation. The phosphorylation of CRMP2 inhibited its interaction with endophilin2; this was mainly affected by the phosphorylation of Thr514 and Ser518 by glycogen synthase kinase (GSK) 3?. CRMP2 phosphorylation increased degradation and inhibited the surface expression of AMPAR GluA1 subunits in cultured hippocampal neurons. However, the dephosphorylation of CRMP2 inhibited degradation and promoted the surface expression of AMPAR GluA1 subunits in cultured hippocampal neurons. Taken together, our data demonstrated that the interaction between CRMP2 and endophilin2 was conductive to the recycling of AMPA receptor GluA1 subunits in hippocampal neurons.
| S-EPMC7417516 | biostudies-literature
Project description:We tested the proposal that the C-terminal domain (CTD) of the AMPAR subunit GluA1 is required for LTP. We found that a knock-in mouse lacking the CTD of GluA1 expresses normal LTP and spatial memory, assayed by the Morris water maze. Our results support a model in which LTP generates synaptic slots, which capture passively diffusing AMPARs.
| S-EPMC7500950 | biostudies-literature
Project description:The acute neurotoxicity of oligomeric forms of amyloid-β 1-42 (Aβ) is implicated in the pathogenesis of Alzheimer's disease (AD). However, how these oligomers might first impair neuronal function at the onset of pathology is poorly understood. Here we have examined the underlying toxic effects caused by an increase in levels of intracellular Aβ, an event that could be important during the early stages of the disease. We show that oligomerised Aβ induces a rapid enhancement of AMPA receptor-mediated synaptic transmission (EPSC(A)) when applied intracellularly. This effect is dependent on postsynaptic Ca(2+) and PKA. Knockdown of GluA1, but not GluA2, prevents the effect, as does expression of a S845-phosphomutant of GluA1. Significantly, an inhibitor of Ca(2+)-permeable AMPARs (CP-AMPARs), IEM 1460, reverses the increase in the amplitude of EPSC(A). These results suggest that a primary neuronal response to intracellular Aβ oligomers is the rapid synaptic insertion of CP-AMPARs.
| S-EPMC4460729 | biostudies-other
Project description:The number of ?-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptors (AMPARs) in synapses determines synaptic strength. AMPAR expression can be regulated locally in dendrites by synaptic activity. The mechanisms of activity-dependent local regulation of AMPAR expression, however, remain unclear. Here, we tested whether microRNAs (miRNAs) are involved in N-methyl-D-aspartate (NMDA) receptor (NMDAR)-dependent AMPAR expression. We used the 3' untranslated region of Gria1, which encodes the AMPA receptor subunit GluA1, to pull down miRNAs binding to it and analyzed these miRNAs using next-generation deep sequencing. Among the identified miRNAs, miR-501-3p is also a computationally predicted Gria1-targeting miRNA. We confirmed that miR-501-3p targets Gria1 and regulates its expression under physiological conditions. The expression of miR-501-3p and GluA1, moreover, is inversely correlated during postnatal brain development. miR-501-3p expression is up-regulated locally in dendrites through the NMDAR subunit GluN2A, and this regulation is required for NMDA-induced suppression of GluA1 expression and long-lasting remodeling of dendritic spines. These findings elucidate a miRNA-mediated mechanism for activity-dependent, local regulation of AMPAR expression in dendrites.
| S-EPMC4384731 | biostudies-literature