Browse
Submit Data
Databases
API
Help

Dataset Information

29 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

ABSTRACT: Although the cerebellum has been implicated in simple reward-based learning recently, the role of complex spikes (CS) and simple spikes (SS), their interaction and their relationship to complex reinforcement learning and decision making is still unclear. Here we show that in a context where a non-human primate learned to make novel visuomotor associations, classifying CS responses based on their SS properties revealed distinct cell-type specific encoding of the probability of failure after the stimulus onset and the non-human primate’s decision. In a different context, CS from the same cerebellar area also responded in a cell-type and learning independent manner to the stimulus that signaled the beginning of the trial. Both types of CS signals were independent of changes in any motor kinematics and were unlikely to instruct the concurrent SS activity through an error based mechanism, suggesting the presence of context dependent, flexible, multiple independent channels of neural encoding by CS and SS. This diversity in neural information encoding in the mid-lateral cerebellum, depending on the context and learning state, is well suited to promote exploration and acquisition of wide range of cognitive behaviors that entail flexible stimulus-action-reward relationships but not necessarily motor learning. The role of complex spikes in reinforcement learning is still unclear. Here, the authors show that complex spikes carry multiple context based, cell type specific and learning dependent signals that are independent of changes in any motor kinematics and unlikely to instruct the concurrent simple spike activity during reinforcement learning.

SUBMITTER: Sendhilnathan N

PROVIDER: S-EPMC8578621 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Neural Correlates of Reinforcement Learning in Mid-lateral Cerebellum.

Project description:The role of the cerebellum in non-motor learning is poorly understood. Here, we investigated the activity of Purkinje cells (P-cells) in the mid-lateral cerebellum as the monkey learned to associate one arbitrary symbol with the movement of the left hand and another with the movement of the right hand. During learning, but not when the monkey had learned the association, the simple spike responses of P-cells reported the outcome of the animal's most recent decision without concomitant changes in other sensorimotor parameters such as hand movement, licking, or eye movement. At the population level, P-cells collectively maintained a memory of the most recent decision throughout the entire trial. As the monkeys learned the association, the magnitude of this reward-related error signal approached zero. Our results provide a major departure from the current understanding of cerebellar processing and have critical implications for cerebellum's role in cognitive control.

| S-EPMC8015782 | biostudies-literature

Separate mesocortical and mesolimbic pathways encode effort and reward learning signals.

Project description:Optimal decision making mandates organisms learn the relevant features of choice options. Likewise, knowing how much effort we should expend can assume paramount importance. A mesolimbic network supports reward learning, but it is unclear whether other choice features, such as effort learning, rely on this same network. Using computational fMRI, we show parallel encoding of effort and reward prediction errors (PEs) within distinct brain regions, with effort PEs expressed in dorsomedial prefrontal cortex and reward PEs in ventral striatum. We show a common mesencephalic origin for these signals evident in overlapping, but spatially dissociable, dopaminergic midbrain regions expressing both types of PE. During action anticipation, reward and effort expectations were integrated in ventral striatum, consistent with a computation of an overall net benefit of a stimulus. Thus, we show that motivationally relevant stimulus features are learned in parallel dopaminergic pathways, with formation of an integrated utility signal at choice.

| S-EPMC5584432 | biostudies-literature

Cerebellar climbing fibers encode expected reward size.

Project description:Climbing fiber inputs to the cerebellum encode error signals that instruct learning. Recently, evidence has accumulated to suggest that the cerebellum is also involved in the processing of reward. To study how rewarding events are encoded, we recorded the activity of climbing fibers when monkeys were engaged in an eye movement task. At the beginning of each trial, the monkeys were cued to the size of the reward that would be delivered upon successful completion of the trial. Climbing fiber activity increased when the monkeys were presented with a cue indicating a large reward, but not a small reward. Reward size did not modulate activity at reward delivery or during eye movements. Comparison between climbing fiber and simple spike activity indicated different interactions for coding of movement and reward. These results indicate that climbing fibers encode the expected reward size and suggest a general role of the cerebellum in associative learning beyond error correction.

| S-EPMC6844644 | biostudies-literature

Cerebellar granule cells encode the expectation of reward.

Project description:The human brain contains approximately 60 billion cerebellar granule cells, which outnumber all other brain neurons combined. Classical theories posit that a large, diverse population of granule cells allows for highly detailed representations of sensorimotor context, enabling downstream Purkinje cells to sense fine contextual changes. Although evidence suggests a role for the cerebellum in cognition, granule cells are known to encode only sensory and motor context. Here, using two-photon calcium imaging in behaving mice, we show that granule cells convey information about the expectation of reward. Mice initiated voluntary forelimb movements for delayed sugar-water reward. Some granule cells responded preferentially to reward or reward omission, whereas others selectively encoded reward anticipation. Reward responses were not restricted to forelimb movement, as a Pavlovian task evoked similar responses. Compared to predictable rewards, unexpected rewards elicited markedly different granule cell activity despite identical stimuli and licking responses. In both tasks, reward signals were widespread throughout multiple cerebellar lobules. Tracking the same granule cells over several days of learning revealed that cells with reward-anticipating responses emerged from those that responded at the start of learning to reward delivery, whereas reward-omission responses grew stronger as learning progressed. The discovery of predictive, non-sensorimotor encoding in granule cells is a major departure from the current understanding of these neurons and markedly enriches the contextual information available to postsynaptic Purkinje cells, with important implications for cognitive processing in the cerebellum.

| S-EPMC5532014 | biostudies-literature

Serotonin neurons in the dorsal raphe nucleus encode reward signals.

Project description:The dorsal raphe nucleus (DRN) is involved in organizing reward-related behaviours; however, it remains unclear how genetically defined neurons in the DRN of a freely behaving animal respond to various natural rewards. Here we addressed this question using fibre photometry and single-unit recording from serotonin (5-HT) neurons and GABA neurons in the DRN of behaving mice. Rewards including sucrose, food, sex and social interaction rapidly activate 5-HT neurons, but aversive stimuli including quinine and footshock do not. Both expected and unexpected rewards activate 5-HT neurons. After mice learn to wait for sucrose delivery, most 5-HT neurons fire tonically during waiting and then phasically on reward acquisition. Finally, GABA neurons are activated by aversive stimuli but inhibited when mice seek rewards. Thus, DRN 5-HT neurons positively encode a wide range of reward signals during anticipatory and consummatory phases of reward responses. Moreover, GABA neurons play a complementary role in reward processing.

| S-EPMC4738365 | biostudies-literature

Lateral Hypothalamic GABAergic Neurons Encode Reward Predictions that Are Relayed to the Ventral Tegmental Area to Regulate Learning.

Project description:Eating is a learned process. Our desires for specific foods arise through experience. Both electrical stimulation and optogenetic studies have shown that increased activity in the lateral hypothalamus (LH) promotes feeding. Current dogma is that these effects reflect a role for LH neurons in the control of the core motivation to feed, and their activity comes under control of forebrain regions to elicit learned food-motivated behaviors. However, these effects could also reflect the storage of associative information about the cues leading to food in LH itself. Here, we present data from several studies that are consistent with a role for LH in learning. In the first experiment, we use a novel GAD-Cre rat to show that optogenetic inhibition of LH γ-aminobutyric acid (GABA) neurons restricted to cue presentation disrupts the rats' ability to learn that a cue predicts food without affecting subsequent food consumption. In the second experiment, we show that this manipulation also disrupts the ability of a cue to promote food seeking after learning. Finally, we show that inhibition of the terminals of the LH GABA neurons in ventral-tegmental area (VTA) facilitates learning about reward-paired cues. These results suggest that the LH GABA neurons are critical for storing and later disseminating information about reward-predictive cues.

| S-EPMC5564224 | biostudies-literature

Dentate granule cells encode auditory decisions after reinforcement learning in rats.

Project description:Auditory-cued goal-oriented behaviors requires the participation of cortical and subcortical brain areas, but how neural circuits associate sensory-based decisions with goal locations through learning remains poorly understood. The hippocampus is critical for spatial coding, suggesting its possible involvement in transforming sensory inputs to the goal-oriented decisions. Here, we developed an auditory discrimination task in which rats learned to navigate to goal locations based on the frequencies of auditory stimuli. Using in vivo calcium imaging in freely behaving rats over the course of learning, we found that dentate granule cells became more active, spatially tuned, and responsive to task-related variables as learning progressed. Furthermore, only after task learning, the activity of dentate granule cell ensembles represented the navigation path and predicts auditory decisions as early as when rats began to approach the goals. Finally, chemogenetic silencing of dentate gyrus suppressed task learning. Our results demonstrate that dentate granule cells gain task-relevant firing pattern through reinforcement learning and could be a potential link of sensory decisions to spatial navigation.

| S-EPMC8277790 | biostudies-literature

Reward-predictive representations generalize across tasks in reinforcement learning.

Project description:In computer science, reinforcement learning is a powerful framework with which artificial agents can learn to maximize their performance for any given Markov decision process (MDP). Advances over the last decade, in combination with deep neural networks, have enjoyed performance advantages over humans in many difficult task settings. However, such frameworks perform far less favorably when evaluated in their ability to generalize or transfer representations across different tasks. Existing algorithms that facilitate transfer typically are limited to cases in which the transition function or the optimal policy is portable to new contexts, but achieving "deep transfer" characteristic of human behavior has been elusive. Such transfer typically requires discovery of abstractions that permit analogical reuse of previously learned representations to superficially distinct tasks. Here, we demonstrate that abstractions that minimize error in predictions of reward outcomes generalize across tasks with different transition and reward functions. Such reward-predictive representations compress the state space of a task into a lower dimensional representation by combining states that are equivalent in terms of both the transition and reward functions. Because only state equivalences are considered, the resulting state representation is not tied to the transition and reward functions themselves and thus generalizes across tasks with different reward and transition functions. These results contrast with those using abstractions that myopically maximize reward in any given MDP and motivate further experiments in humans and animals to investigate if neural and cognitive systems involved in state representation perform abstractions that facilitate such equivalence relations.

| S-EPMC7591094 | biostudies-literature

Stress transforms lateral habenula reward responses into punishment signals.

Project description:Neuronal activity in the lateral habenula (LHb), a brain region implicated in depression [C. D. Proulx, O. Hikosaka, R. Malinow, Nat. Neurosci. 17, 1146-1152 (2014)], decreases during reward and increases during punishment or reward omission [M. Matsumoto, O. Hikosaka, Nature 447, 1111-1115 (2007)]. While stress is a major risk factor for depression and strongly impacts the LHb, its effect on LHb reward signals is unknown. Here we image LHb neuronal activity in behaving mice and find that acute stress transforms LHb reward responses into punishment-like neural signals; punishment-like responses to reward omission also increase. These neural changes matched the onset of anhedonic behavior and were specific to LHb neurons that distinguished reward and its omission. Thus, stress distorts LHb responsivity to positive and negative feedback, which could bias individuals toward negative expectations, a key aspect of the proposed pathogenesis of depression [A. T. Beck, Depression: Clinical, Experimental, and Theoretical Aspects, sixth Ed (1967)].

| S-EPMC6589650 | biostudies-literature

Improving Robot Motor Learning with Negatively Valenced Reinforcement Signals.

Project description:Both nociception and punishment signals have been used in robotics. However, the potential for using these negatively valenced types of reinforcement learning signals for robot learning has not been exploited in detail yet. Nociceptive signals are primarily used as triggers of preprogrammed action sequences. Punishment signals are typically disembodied, i.e., with no or little relation to the agent-intrinsic limitations, and they are often used to impose behavioral constraints. Here, we provide an alternative approach for nociceptive signals as drivers of learning rather than simple triggers of preprogrammed behavior. Explicitly, we use nociception to expand the state space while we use punishment as a negative reinforcement learning signal. We compare the performance-in terms of task error, the amount of perceived nociception, and length of learned action sequences-of different neural networks imbued with punishment-based reinforcement signals for inverse kinematic learning. We contrast the performance of a version of the neural network that receives nociceptive inputs to that without such a process. Furthermore, we provide evidence that nociception can improve learning-making the algorithm more robust against network initializations-as well as behavioral performance by reducing the task error, perceived nociception, and length of learned action sequences. Moreover, we provide evidence that punishment, at least as typically used within reinforcement learning applications, may be detrimental in all relevant metrics.

| S-EPMC5376586 | biostudies-other

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data