RM-SORN: a reward-modulated self-organizing recurrent neural network.
Ontology highlight
ABSTRACT:
SUBMITTER: Aswolinskiy W
PROVIDER: S-EPMC4371712 | biostudies-literature | 2015
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Many experiments have suggested that the brain operates close to a critical state, based on signatures of criticality such as power-law distributed neuronal avalanches. In neural network models, criticality is a dynamical state that maximizes information processing capacities, e.g. sensitivity to input, dynamical range and storage capacity, which makes it a favorable candidate state for brain function. Although models that self-organize towards a critical state have been proposed, the relation between criticality signatures and learning is still unclear. Here, we investigate signatures of criticality in a self-organizing recurrent neural network (SORN). Investigating criticality in the SORN is of particular interest because it has not been developed to show criticality. Instead, the SORN has been shown to exhibit spatio-temporal pattern learning through a combination of neural plasticity mechanisms and it reproduces a number of biological findings on neural variability and the statistics and fluctuations of synaptic efficacies. We show that, after a transient, the SORN spontaneously self-organizes into a dynamical state that shows criticality signatures comparable to those found in experiments. The plasticity mechanisms are necessary to attain that dynamical state, but not to maintain it. Furthermore, onset of external input transiently changes the slope of the avalanche distributions - matching recent experimental findings. Interestingly, the membrane noise level necessary for the occurrence of the criticality signatures reduces the model's performance in simple learning tasks. Overall, our work shows that the biologically inspired plasticity and homeostasis mechanisms responsible for the SORN's spatio-temporal learning abilities can give rise to criticality signatures in its activity when driven by random input, but these break down under the structured input of short repeating sequences.
| S-EPMC5446191 | biostudies-literature
Project description:The proposed architecture applies the principle of predictive coding and deep learning in a brain-inspired approach to robotic sensorimotor control. It is composed of many layers each of which is a recurrent network. The component networks can be spontaneously active due to the homeokinetic learning rule, a principle that has been studied previously for the purpose of self-organized generation of behavior. We present robotic simulations that illustrate the function of the network and show evidence that deeper networks enable more complex exploratory behavior.
| S-EPMC7523181 | biostudies-literature
Project description:A self-organized phenomenon in postural coordination is essential for understanding the auto-switching mechanism of in-phase and anti-phase postural coordination modes during standing and related supra-postural activities. Previously, a model-based approach was proposed to reproduce such self-organized phenomenon. However, if we set this problem including the process of how we establish the internal predictive model in our central nervous system, the learning process is critical to be considered for establishing a neural network for managing adaptive postural control. Particularly when body characteristics may change due to growth or aging or are initially unknown for infants, a learning capability can improve the hyper-adaptivity of human motor control for maintaining postural stability and saving energy in daily living. This study attempted to generate a self-organizing neural network that can adaptively coordinate the postural mode without assuming a prior body model regarding body dynamics and kinematics. Postural coordination modes are reproduced in head-target tracking tasks through a deep reinforcement learning algorithm. The transitions between the postural coordination types, i.e. in-phase and anti-phase coordination modes, could be reproduced by changing the task condition of the head tracking target, by changing the frequencies of the moving target. These modes are considered emergent phenomena existing in human head tracking tasks. Various evaluation indices, such as correlation, and relative phase of hip and ankle joint, are analyzed to verify the self-organizing neural network performance to produce the postural coordination transition between the in-phase and anti-phase modes. In addition, after learning, the neural network can also adapt to continuous task condition changes and even to unlearned body mass conditions keeping consistent in-phase and anti-phase mode alternation.
| S-EPMC10238493 | biostudies-literature
Project description:Place cells in the mammalian hippocampus signal self-location with sparse spatially stable firing fields. Based on observation of place cell activity it is possible to accurately decode an animal's location. The precision of this decoding sets a lower bound for the amount of information that the hippocampal population conveys about the location of the animal. In this work we use a novel recurrent neural network (RNN) decoder to infer the location of freely moving rats from single unit hippocampal recordings. RNNs are biologically plausible models of neural circuits that learn to incorporate relevant temporal context without the need to make complicated assumptions about the use of prior information to predict the current state. When decoding animal position from spike counts in 1D and 2D-environments, we show that the RNN consistently outperforms a standard Bayesian approach with either flat priors or with memory. In addition, we also conducted a set of sensitivity analysis on the RNN decoder to determine which neurons and sections of firing fields were the most influential. We found that the application of RNNs to neural data allowed flexible integration of temporal context, yielding improved accuracy relative to the more commonly used Bayesian approaches and opens new avenues for exploration of the neural code.
| S-EPMC6407788 | biostudies-literature
Project description:A food's reward value is dependent on its caloric content. Furthermore, a food's acute reward value also depends on hunger state. The drive to obtain rewards (reward sensitivity), however, differs between individuals. Here, we assessed the association between brain responses to calories in the mouth and trait reward sensitivity in different hunger states. Firstly, we assessed this in data from a functional neuroimaging study (van Rijn et al., 2015), in which participants (n = 30) tasted simple solutions of a non-caloric sweetener with or without a non-sweet carbohydrate (maltodextrin) during hunger and satiety. Secondly, we expanded these analyses to regular drinks by assessing the same relationship in data from a study in which soft drinks sweetened with either sucrose or a non-caloric sweetener were administered during hunger (n = 18) (Griffioen-Roose et al., 2013). First, taste activation by the non-caloric solution/soft drink was subtracted from that by the caloric solution/soft drink to eliminate sweetness effects and retain activation induced by calories. Subsequently, this difference in taste activation was correlated with reward sensitivity as measured with the BAS drive subscale of the Behavioral Activation System (BAS) questionnaire. When participants were hungry and tasted calories from the simple solution, brain activation in the right ventral striatum (caudate), right amygdala and anterior cingulate cortex (bilaterally) correlated negatively with BAS drive scores. In contrast, when participants were satiated, taste responses correlated positively with BAS drive scores in the left caudate. These results were not replicated for soft drinks. Thus, neural responses to oral calories from maltodextrin were modulated by reward sensitivity in reward-related brain areas. This was not the case for sucrose. This may be due to the direct detection of maltodextrin, but not sucrose in the oral cavity. Also, in a familiar beverage, detection of calories per se may be overruled by a conditioned response to its flavor. In conclusion, the brain reward response to calories from a long chain starch sugar (maltodextrin) varies with trait reward sensitivity. The absence of this effect in a familiar beverage warrants further research into its relevance for real life ingestive behavior.
| S-EPMC4712268 | biostudies-literature
Project description:Activity prediction plays an essential role in drug discovery by directing search of drug candidates in the relevant chemical space. Despite being applied successfully to image recognition and semantic similarity, the Siamese neural network has rarely been explored in drug discovery where modelling faces challenges such as insufficient data and class imbalance. Here, we present a Siamese recurrent neural network model (SiameseCHEM) based on bidirectional long short-term memory architecture with a self-attention mechanism, which can automatically learn discriminative features from the SMILES representations of small molecules. Subsequently, it is used to categorize bioactivity of small molecules via N-shot learning. Trained on random SMILES strings, it proves robust across five different datasets for the task of binary or categorical classification of bioactivity. Benchmarking against two baseline machine learning models which use the chemistry-rich ECFP fingerprints as the input, the deep learning model outperforms on three datasets and achieves comparable performance on the other two. The failure of both baseline methods on SMILES strings highlights that the deep learning model may learn task-specific chemistry features encoded in SMILES strings.
| S-EPMC8153912 | biostudies-literature
Project description:ObjectiveThe regularity of scale-free patterns in rank-size relations has been observed in word frequency, city size distribution, firm size distribution, and gene expression. Because of the common emergence of this regularity, understanding its mechanisms has been of great interest. For obtaining the scale-free pattern regularity, various models based on the rich-get-richer mechanism have been proposed; however, the overarching procedure of searching for the "rich" is in disagreement with the locally interacting behaviors seen in the aforementioned natural and social phenomena.ResultsWe implemented a computational model of a resource distribution system inspired by observations of word connectivity, which is created by local constraints with periodic and phase modulatory features. Here, we empirically demonstrated that a phase-modulated periodic connecting system can reach a dynamic equilibrium state as the most probable case, with the self-organizing scale-free patterns. The regularity could be a result of the configurational balance in spatiotemporal inequity during the resource distribution process with an adaptive constrained connectivity. Our results suggest that investigations of interferences of oscillating fluctuations in the system will elucidate the autoregulatory dynamic behavior.
| S-EPMC6402156 | biostudies-literature
Project description:To address the problems of high overflow rate of pipe network inspection well and low drainage efficiency, a rainwater control optimization design approach based on a self-organizing feature map neural network model (SOFM) was proposed in this paper. These problems are caused by low precision parameter design in various rainwater control measures such as the diameter of the rainwater pipe network and the green roof area ratio. This system is to be combined with the newly built rainwater pipe control optimization design project of China International Airport in Daxing District of Beijing, China. Through the optimization adjustment of the pipe network parameters such as the diameter of the rainwater pipe network, the slope of the pipeline, and the green infrastructure (GI) parameters such as the sinking green area and the green roof area, reasonable control of airport rainfall and the construction of sustainable drainage systems can be achieved. This research indicates that compared with the result of the drainage design under the initial value of the parameter, the green roof model and the conceptual model of the mesoscale sustainable drainage system, in the case of a hundred-year torrential rainstorm, the overflow rate of pipe network inspection wells has reduced by 36% to 67.5%, the efficiency of drainage has increased by 26.3% to 61.7%, which achieves the requirements for reasonable control of airport rainwater and building a sponge airport and a sustainable drainage system.
| S-EPMC6974174 | biostudies-literature
Project description:At around 7 months of age, human infants begin to reliably produce well-formed syllables containing both consonants and vowels, a behavior called canonical babbling. Over subsequent months, the frequency of canonical babbling continues to increase. How the infant's nervous system supports the acquisition of this ability is unknown. Here we present a computational model that combines a spiking neural network, reinforcement-modulated spike-timing-dependent plasticity, and a human-like vocal tract to simulate the acquisition of canonical babbling. Like human infants, the model's frequency of canonical babbling gradually increases. The model is rewarded when it produces a sound that is more auditorily salient than sounds it has previously produced. This is consistent with data from human infants indicating that contingent adult responses shape infant behavior and with data from deaf and tracheostomized infants indicating that hearing, including hearing one's own vocalizations, is critical for canonical babbling development. Reward receipt increases the level of dopamine in the neural network. The neural network contains a reservoir with recurrent connections and two motor neuron groups, one agonist and one antagonist, which control the masseter and orbicularis oris muscles, promoting or inhibiting mouth closure. The model learns to increase the number of salient, syllabic sounds it produces by adjusting the base level of muscle activation and increasing their range of activity. Our results support the possibility that through dopamine-modulated spike-timing-dependent plasticity, the motor cortex learns to harness its natural oscillations in activity in order to produce syllabic sounds. It thus suggests that learning to produce rhythmic mouth movements for speech production may be supported by general cortical learning mechanisms. The model makes several testable predictions and has implications for our understanding not only of how syllabic vocalizations develop in infancy but also for our understanding of how they may have evolved.
| S-EPMC4726623 | biostudies-literature
Project description:Visual attention appears to modulate cortical neurodynamics and synchronization through various cholinergic mechanisms. In order to study these mechanisms, we have developed a neural network model of visual cortex area V4, based on psychophysical, anatomical and physiological data. With this model, we want to link selective visual information processing to neural circuits within V4, bottom-up sensory input pathways, top-down attention input pathways, and to cholinergic modulation from the prefrontal lobe. We investigate cellular and network mechanisms underlying some recent analytical results from visual attention experimental data. Our model can reproduce the experimental findings that attention to a stimulus causes increased gamma-frequency synchronization in the superficial layers. Computer simulations and STA power analysis also demonstrate different effects of the different cholinergic attention modulation action mechanisms.
| S-EPMC2289046 | biostudies-other