Dataset Information

RM-SORN: a reward-modulated self-organizing recurrent neural network.

ABSTRACT: Neural plasticity plays an important role in learning and memory. Reward-modulation of plasticity offers an explanation for the ability of the brain to adapt its neural activity to achieve a rewarded goal. Here, we define a neural network model that learns through the interaction of Intrinsic Plasticity (IP) and reward-modulated Spike-Timing-Dependent Plasticity (STDP). IP enables the network to explore possible output sequences and STDP, modulated by reward, reinforces the creation of the rewarded output sequences. The model is tested on tasks for prediction, recall, non-linear computation, pattern recognition, and sequence generation. It achieves performance comparable to networks trained with supervised learning, while using simple, biologically motivated plasticity rules, and rewarding strategies. The results confirm the importance of investigating the interaction of several plasticity rules in the context of reward-modulated learning and whether reward-modulated self-organization can explain the amazing capabilities of the brain.

SUBMITTER: Aswolinskiy W

PROVIDER: S-EPMC4371712 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

RM-SORN: a reward-modulated self-organizing recurrent neural network.

Aswolinskiy Witali W Pipa Gordon G

Frontiers in computational neuroscience 20150324

Neural plasticity plays an important role in learning and memory. Reward-modulation of plasticity offers an explanation for the ability of the brain to adapt its neural activity to achieve a rewarded goal. Here, we define a neural network model that learns through the interaction of Intrinsic Plasticity (IP) and reward-modulated Spike-Timing-Dependent Plasticity (STDP). IP enables the network to explore possible output sequences and STDP, modulated by reward, reinforces the creation of the rewar ...[more]

PMID: 25852533

Similar Datasets

Project description:A food's reward value is dependent on its caloric content. Furthermore, a food's acute reward value also depends on hunger state. The drive to obtain rewards (reward sensitivity), however, differs between individuals. Here, we assessed the association between brain responses to calories in the mouth and trait reward sensitivity in different hunger states. Firstly, we assessed this in data from a functional neuroimaging study (van Rijn et al., 2015), in which participants (n = 30) tasted simple solutions of a non-caloric sweetener with or without a non-sweet carbohydrate (maltodextrin) during hunger and satiety. Secondly, we expanded these analyses to regular drinks by assessing the same relationship in data from a study in which soft drinks sweetened with either sucrose or a non-caloric sweetener were administered during hunger (n = 18) (Griffioen-Roose et al., 2013). First, taste activation by the non-caloric solution/soft drink was subtracted from that by the caloric solution/soft drink to eliminate sweetness effects and retain activation induced by calories. Subsequently, this difference in taste activation was correlated with reward sensitivity as measured with the BAS drive subscale of the Behavioral Activation System (BAS) questionnaire. When participants were hungry and tasted calories from the simple solution, brain activation in the right ventral striatum (caudate), right amygdala and anterior cingulate cortex (bilaterally) correlated negatively with BAS drive scores. In contrast, when participants were satiated, taste responses correlated positively with BAS drive scores in the left caudate. These results were not replicated for soft drinks. Thus, neural responses to oral calories from maltodextrin were modulated by reward sensitivity in reward-related brain areas. This was not the case for sucrose. This may be due to the direct detection of maltodextrin, but not sucrose in the oral cavity. Also, in a familiar beverage, detection of calories per se may be overruled by a conditioned response to its flavor. In conclusion, the brain reward response to calories from a long chain starch sugar (maltodextrin) varies with trait reward sensitivity. The absence of this effect in a familiar beverage warrants further research into its relevance for real life ingestive behavior.

Project description:At around 7 months of age, human infants begin to reliably produce well-formed syllables containing both consonants and vowels, a behavior called canonical babbling. Over subsequent months, the frequency of canonical babbling continues to increase. How the infant's nervous system supports the acquisition of this ability is unknown. Here we present a computational model that combines a spiking neural network, reinforcement-modulated spike-timing-dependent plasticity, and a human-like vocal tract to simulate the acquisition of canonical babbling. Like human infants, the model's frequency of canonical babbling gradually increases. The model is rewarded when it produces a sound that is more auditorily salient than sounds it has previously produced. This is consistent with data from human infants indicating that contingent adult responses shape infant behavior and with data from deaf and tracheostomized infants indicating that hearing, including hearing one's own vocalizations, is critical for canonical babbling development. Reward receipt increases the level of dopamine in the neural network. The neural network contains a reservoir with recurrent connections and two motor neuron groups, one agonist and one antagonist, which control the masseter and orbicularis oris muscles, promoting or inhibiting mouth closure. The model learns to increase the number of salient, syllabic sounds it produces by adjusting the base level of muscle activation and increasing their range of activity. Our results support the possibility that through dopamine-modulated spike-timing-dependent plasticity, the motor cortex learns to harness its natural oscillations in activity in order to produce syllabic sounds. It thus suggests that learning to produce rhythmic mouth movements for speech production may be supported by general cortical learning mechanisms. The model makes several testable predictions and has implications for our understanding not only of how syllabic vocalizations develop in infancy but also for our understanding of how they may have evolved.

Dataset Information

RM-SORN: a reward-modulated self-organizing recurrent neural network.

Publications

RM-SORN: a reward-modulated self-organizing recurrent neural network.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets