Browse
Submit Data
Databases
API
Help

Dataset Information

14 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Neural reactivations during sleep determine network credit assignment.

ABSTRACT: A fundamental goal of motor learning is to establish the neural patterns that produce a desired behavioral outcome. It remains unclear how and when the nervous system solves this 'credit assignment' problem. Using neuroprosthetic learning, in which we could control the causal relationship between neurons and behavior, we found that sleep-dependent processing was required for credit assignment and the establishment of task-related functional connectivity reflecting the casual neuron-behavior relationship. Notably, we observed a strong link between the microstructure of sleep reactivations and credit assignment, with downscaling of non-causal activity. Decoupling of spiking to slow oscillations using optogenetic methods eliminated rescaling. Thus, our results suggest that coordinated firing during sleep is essential for establishing sparse activation patterns that reflect the causal neuron-behavior relationship.

SUBMITTER: Gulati T

PROVIDER: S-EPMC5808917 | biostudies-literature | 2017 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Neural reactivations during sleep determine network credit assignment.

Gulati Tanuj T Guo Ling L Ramanathan Dhakshin S DS Bodepudi Anitha A Ganguly Karunesh K

Nature neuroscience 20170710 9

A fundamental goal of motor learning is to establish the neural patterns that produce a desired behavioral outcome. It remains unclear how and when the nervous system solves this 'credit assignment' problem. Using neuroprosthetic learning, in which we could control the causal relationship between neurons and behavior, we found that sleep-dependent processing was required for credit assignment and the establishment of task-related functional connectivity reflecting the casual neuron-behavior rela ...[more]

PMID: 28692062

Similar Datasets

Cell-type-specific neuromodulation guides synaptic credit assignment in a spiking neural network.

Project description:Brains learn tasks via experience-driven differential adjustment of their myriad individual synaptic connections, but the mechanisms that target appropriate adjustment to particular connections remain deeply enigmatic. While Hebbian synaptic plasticity, synaptic eligibility traces, and top-down feedback signals surely contribute to solving this synaptic credit-assignment problem, alone, they appear to be insufficient. Inspired by new genetic perspectives on neuronal signaling architectures, here, we present a normative theory for synaptic learning, where we predict that neurons communicate their contribution to the learning outcome to nearby neurons via cell-type-specific local neuromodulation. Computational tests suggest that neuron-type diversity and neuron-type-specific local neuromodulation may be critical pieces of the biological credit-assignment puzzle. They also suggest algorithms for improved artificial neural network learning efficiency.

| S-EPMC8713766 | biostudies-literature

Neural correlates of temporal credit assignment in the parietal lobe.

Project description:Empirical studies of decision making have typically assumed that value learning is governed by time, such that a reward prediction error arising at a specific time triggers temporally-discounted learning for all preceding actions. However, in natural behavior, goals must be acquired through multiple actions, and each action can have different significance for the final outcome. As is recognized in computational research, carrying out multi-step actions requires the use of credit assignment mechanisms that focus learning on specific steps, but little is known about the neural correlates of these mechanisms. To investigate this question we recorded neurons in the monkey lateral intraparietal area (LIP) during a serial decision task where two consecutive eye movement decisions led to a final reward. The underlying decision trees were structured such that the two decisions had different relationships with the final reward, and the optimal strategy was to learn based on the final reward at one of the steps (the "F" step) but ignore changes in this reward at the remaining step (the "I" step). In two distinct contexts, the F step was either the first or the second in the sequence, controlling for effects of temporal discounting. We show that LIP neurons had the strongest value learning and strongest post-decision responses during the transition after the F step regardless of the serial position of this step. Thus, the neurons encode correlates of temporal credit assignment mechanisms that allocate learning to specific steps independently of temporal discounting.

| S-EPMC3921206 | biostudies-literature

Cortical reactivations of recent sensory experiences predict bidirectional network changes during learning.

Project description:Salient experiences are often relived in the mind. Human neuroimaging studies suggest that such experiences drive activity patterns in visual association cortex that are subsequently reactivated during quiet waking. Nevertheless, the circuit-level consequences of such reactivations remain unclear. Here, we imaged hundreds of neurons in visual association cortex across days as mice learned a visual discrimination task. Distinct patterns of neurons were activated by different visual cues. These same patterns were subsequently reactivated during quiet waking in darkness, with higher reactivation rates during early learning and for food-predicting versus neutral cues. Reactivations involving ensembles of neurons encoding both the food cue and the reward predicted strengthening of next-day functional connectivity of participating neurons, while the converse was observed for reactivations involving ensembles encoding only the food cue. We propose that task-relevant neurons strengthen while task-irrelevant neurons weaken their dialog with the network via participation in distinct flavors of reactivation.

| S-EPMC7392804 | biostudies-literature

Spatio-temporal credit assignment in neuronal population learning.

Project description:In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

| S-EPMC3127803 | biostudies-literature

Efficiency and prioritization of inference-based credit assignment.

Project description:Organisms adapt to their environments by learning to approach states that predict rewards and avoid states associated with punishments. Knowledge about the affective value of states often relies on credit assignment (CA), whereby state values are updated on the basis of reward feedback. Remarkably, humans assign credit to states that are not observed but are instead inferred based on a cognitive map that represents structural knowledge of an environment. A pertinent example is authors attempting to infer the identity of anonymous reviewers to assign them credit or blame and, on this basis, inform future referee recommendations. Although inference is cognitively costly, it is unknown how it influences CA or how it is apportioned between hidden and observable states (for example, both anonymous and revealed reviewers). We addressed these questions in a task that provided choices between lotteries where each led to a unique pair of occasionally rewarding outcome states. On some trials, both states were observable (rendering inference nugatory), whereas on others, the identity of one of the states was concealed. Importantly, by exploiting knowledge of choice-state associations, subjects could infer the identity of this hidden state. We show that having to perform inference reduces state-value updates. Strikingly, and in violation of normative theories, this reduction in CA was selective for the observed outcome alone. These findings have implications for the operation of putative cognitive maps.

| S-EPMC8279739 | biostudies-literature

Retrospective model-based inference guides model-free credit assignment.

Project description:An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants' momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions.

| S-EPMC6375980 | biostudies-literature

Human subjects exploit a cognitive map for credit assignment.

Project description:An influential reinforcement learning framework proposes that behavior is jointly governed by model-free (MF) and model-based (MB) controllers. The former learns the values of actions directly from past encounters, and the latter exploits a cognitive map of the task to calculate these prospectively. Considerable attention has been paid to how these systems interact during choice, but how and whether knowledge of a cognitive map contributes to the way MF and MB controllers assign credit (i.e., to how they revaluate actions and states following the receipt of an outcome) remains underexplored. Here, we examine such sophisticated credit assignment using a dual-outcome bandit task. We provide evidence that knowledge of a cognitive map influences credit assignment in both MF and MB systems, mediating subtly different aspects of apparent relevance. Specifically, we show MF credit assignment is enhanced for those rewards that are related to a choice, and this contrasted with choice-unrelated rewards that reinforced subsequent choices negatively. This modulation is only possible based on knowledge of task structure. On the other hand, MB credit assignment was boosted for outcomes that impacted on differences in values between offered bandits. We consider mechanistic accounts and the normative status of these findings. We suggest the findings extend the scope and sophistication of cognitive map-based credit assignment during reinforcement learning, with implications for understanding behavioral control.

| S-EPMC7848688 | biostudies-literature

Predicting corporate credit risk: Network contagion via trade credit.

Project description:Trade credit is a payment extension granted by a selling firm to its customer. Companies typically respond to late payments from their customers by delaying payments to suppliers, thus generating a ripple through the transaction network. Therefore, trade credit is as a potential vehicle of propagation of losses in case of default events. The goal of this work is to leverage information on the trade credit among connected firms to predict imminent defaults of firms. We use a unique dataset of client firms of a major Italian bank to investigate firm bankruptcy between October 2016 to March 2018. We develop a model to capture network spillover effects originating from the supply chain on the probability of default of each firm via a sequential approach: the output of a first model component on single firm features is used in a subsequent model which captures network spillovers. While the first component is the standard econometrics way to predict such dynamics, the network module represents an innovative way to look into the effect of trade credit on default probability. This module looks at the transaction network of the firm, as inferred from the payments transiting via the bank, in order to identify the trade partners of the firm. By using several features extracted from the network of transactions, this model is able to predict a large fraction of the defaults, thus showing the value hidden in the network information. Finally, we merge firm and network features with a machine learning model to create a 'hybrid' model, which improves the recall for the task by almost 20 percentage points over the baseline.

| S-EPMC8084139 | biostudies-literature

Neural network detects errors in the assignment of mRNA splice sites.

Project description:The use of databanks in genetic research assumes reliability of the information they contain. Currently, error-detection in the manually or electronically entered data contained in the nucleotide sequence databanks at EMBL, Heidelberg and GenBank at Los Alamos is limited. We have used a subset of sequences from these databanks to train neural networks to recognize pre-mRNA splicing signals in human genes. During the training on 33 human genes from the EMBL databank seven genes appeared to disturb the learning process. Subsequent investigation revealed discrepancies from the original published papers, for three genes. In four genes, we found wrongly assigned splicing frames of introns. We believe this to be a reflection of the fact that splicing frames cannot always be unambiguously assigned on the basis of experimental data. Thus incorrect assignment appear both due to mere typographical misprints as well as erroneous interpretation of experiments. Training on 241 human sequences from GenBank revealed nine new errors. We propose that such errors could be detected by computer algorithms designed to check the consistency of data prior to their incorporation in databanks.

| S-EPMC331948 | biostudies-other

Neural mechanisms of credit card spending.

Project description:Credit cards have often been blamed for consumer overspending and for the growth in household debt. Indeed, laboratory studies of purchase behavior have shown that credit cards can facilitate spending in ways that are difficult to justify on purely financial grounds. However, the psychological mechanisms behind this spending facilitation effect remain conjectural. A leading hypothesis is that credit cards reduce the pain of payment and so 'release the brakes' that hold expenditures in check. Alternatively, credit cards could provide a 'step on the gas,' increasing motivation to spend. Here we present the first evidence of differences in brain activation in the presence of real credit and cash purchase opportunities. In an fMRI shopping task, participants purchased items tailored to their interests, either by using a personal credit card or their own cash. Credit card purchases were associated with strong activation in the striatum, which coincided with onset of the credit card cue and was not related to product price. In contrast, reward network activation weakly predicted cash purchases, and only among relatively cheaper items. The presence of reward network activation differences highlights the potential neural impact of novel payment instruments in stimulating spending-these fundamental reward mechanisms could be exploited by new payment methods as we transition to a purely cashless society.

| S-EPMC7892835 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data