Synaptic metaplasticity in binarized neural networks.
Ontology highlight
ABSTRACT:
SUBMITTER: Laborieux A
PROVIDER: S-EPMC8100137 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:In the biological neural network, the learning process is achieved through massively parallel synaptic connections between neurons that can be adjusted in an analog manner. Recent developments in emerging synaptic devices and their networks can emulate the functionality of a biological neural network, which will be the fundamental building block for a neuromorphic computing architecture. However, on-chip implementation of a large-scale artificial neural network is still very challenging due to unreliable analog weight modulation in current synaptic device technology. Here, we demonstrate a binarized neural network (BNN) based on a gate-all-around silicon nanosheet synaptic transistor, where reliable digital-type weight modulation can contribute to improve the sustainability of the entire network. BNN is applied to three proof-of-concept examples: (1) handwritten digit classification (MNIST dataset), (2) face image classification (Yale dataset), and (3) experimental 3 × 3 binary pattern classifications using an integrated synaptic transistor network (total 9 × 9 × 2 162 cells) through a supervised online training procedure. The results consolidate the feasibility of binarized neural networks and pave the way toward building a reliable and large-scale artificial neural network by using more advanced conventional digital device technologies.
| S-EPMC6690903 | biostudies-literature
Project description:Brain-inspired neuromorphic systems (hardware neural networks) are expected to be an energy-efficient computing architecture for solving cognitive tasks, which critically depend on the development of reliable synaptic weight storage (i.e., synaptic device). Although various nanoelectronic devices have successfully reproduced the learning rules of biological synapses through their internal analog conductance states, the sustainability of such devices is still in doubt due to the variability common to all nanoelectronic devices. Alternatively, a neuromorphic system based on a relatively more reliable digital-type switching device has been recently demonstrated, i.e., a binarized neural network (BNN). The synaptic device is a more mature digital-type switching device, and the training/recognition algorithm developed for the BNN enables the task of facial image classification with a supervised training scheme. Here, we quantitatively investigate the effects of device parameter variations on the classification accuracy; the parameters include the number of weight states (Nstate), the weight update margin (?G), and the weight update variation (Gvar). This analysis demonstrates the feasibility of the BNN and introduces a practical neuromorphic system based on mature, conventional digital device technologies.
| S-EPMC6811618 | biostudies-literature
Project description:The rapid development of artificial neural networks and applied artificial intelligence has led to many applications. However, current software implementation of neural networks is severely limited in terms of performance and energy efficiency. It is believed that further progress requires the development of neuromorphic systems, in which hardware directly mimics the neuronal network structure of a human brain. Here, we propose theoretically and realize experimentally an optical network of nodes performing binary operations. The nonlinearity required for efficient computation is provided by semiconductor microcavities in the strong quantum light-matter coupling regime, which exhibit exciton-polariton interactions. We demonstrate the system performance against a pattern recognition task, obtaining accuracy on a par with state-of-the-art hardware implementations. Our work opens the way to ultrafast and energy-efficient neuromorphic systems taking advantage of ultrastrong optical nonlinearity of polaritons.
| S-EPMC8155323 | biostudies-literature
Project description:The development of a persistent depressive affective state has for some time been thought to result from persistent alterations in neurotransmitter-mediated synaptic transmission. While the identity of those transmitters has changed over the years, the literature has lacked mechanistic connections between the neurophysiological mechanisms they regulate, and how these mechanisms alter neuronal function, and, hence, affective homeostasis. This review will examine recent work that suggests that both long-term activity-dependent changes in synaptic strength ("plasticity"), and shifting set points for the ease of induction of future long-term changes ("metaplasticity"), may be critical to establishing and reversing a depressive behavioral state. Activitydependent long-term synaptic plasticity involves both strengthening and weakening of synaptic connections associated with a dizzying array of neurochemical alterations that include synaptic insertion and removal of a number of subtypes of AMPA, NMDA and metabotropic glutamate receptors, changes in presynaptic glutamate release, and structural changes in dendritic spines. Cellular mechanisms of metaplasticity are far less well understood. Here, we will review the growing evidence that long-term synaptic changes in glutamatergic transmission, in brain regions that regulate mood, are key determinants of affective homeostasis and therapeutic targets with immense potential for drug development.
| S-EPMC5327460 | biostudies-literature
Project description:The brain performs intelligent tasks with extremely low energy consumption. This work takes its inspiration from two strategies used by the brain to achieve this energy efficiency: the absence of separation between computing and memory functions and reliance on low-precision computation. The emergence of resistive memory technologies indeed provides an opportunity to tightly co-integrate logic and memory in hardware. In parallel, the recently proposed concept of a Binarized Neural Network, where multiplications are replaced by exclusive NOR (XNOR) logic gates, offers a way to implement artificial intelligence using very low precision computation. In this work, we therefore propose a strategy for implementing low-energy Binarized Neural Networks that employs brain-inspired concepts while retaining the energy benefits of digital electronics. We design, fabricate, and test a memory array, including periphery and sensing circuits, that is optimized for this in-memory computing scheme. Our circuit employs hafnium oxide resistive memory integrated in the back end of line of a 130-nm CMOS process, in a two-transistor, two-resistor cell, which allows the exclusive NOR operations of the neural network to be performed directly within the sense amplifiers. We show, based on extensive electrical measurements, that our design allows a reduction in the number of bit errors on the synaptic weights without the use of formal error-correcting codes. We design a whole system using this memory array. We show on standard machine learning tasks (MNIST, CIFAR-10, ImageNet, and an ECG task) that the system has inherent resilience to bit errors. We evidence that its energy consumption is attractive compared to more standard approaches and that it can use memory devices in regimes where they exhibit particularly low programming energy and high endurance. We conclude the work by discussing how it associates biologically plausible ideas with more traditional digital electronics concepts.
| S-EPMC6962102 | biostudies-literature
Project description:Prediction is the ability of the brain to quickly activate a target concept in response to a related stimulus (prime). Experiments point to the existence of an overlap between the populations of the neurons coding for different stimuli, and other experiments show that prime-target relations arise in the process of long term memory formation. The classical modelling paradigm is that long term memories correspond to stable steady states of a Hopfield network with Hebbian connectivity. Experiments show that short term synaptic depression plays an important role in the processing of memories. This leads naturally to a computational model of priming, called latching dynamics; a stable state (prime) can become unstable and the system may converge to another transiently stable steady state (target). Hopfield network models of latching dynamics have been studied by means of numerical simulation, however the conditions for the existence of this dynamics have not been elucidated. In this work we use a combination of analytic and numerical approaches to confirm that latching dynamics can exist in the context of a symmetric Hebbian learning rule, however lacks robustness and imposes a number of biologically unrealistic restrictions on the model. In particular our work shows that the symmetry of the Hebbian rule is not an obstruction to the existence of latching dynamics, however fine tuning of the parameters of the model is needed.
| S-EPMC5573234 | biostudies-literature
Project description:BackgroundNeural circuit function is highly sensitive to energetic limitations. Much like mammals, brain activity in American bullfrogs quickly fails in hypoxia. However, after emergence from overwintering, circuits transform to function for approximately 30-fold longer without oxygen using only anaerobic glycolysis for fuel, a unique trait among vertebrates considering the high cost of network activity. Here, we assessed neuronal functions that normally limit network output and identified components that undergo energetic plasticity to increase robustness in hypoxia.ResultsIn control animals, oxygen deprivation depressed excitatory synaptic drive within native circuits, which decreased postsynaptic firing to cause network failure within minutes. Assessments of evoked and spontaneous synaptic transmission showed that hypoxia impairs synaptic communication at pre- and postsynaptic loci. However, control neurons maintained membrane potentials and a capacity for firing during hypoxia, indicating that those processes do not limit network activity. After overwintering, synaptic transmission persisted in hypoxia to sustain motor function for at least 2 h.ConclusionsAlterations that allow anaerobic metabolism to fuel synapses are critical for transforming a circuit to function without oxygen. Data from many vertebrate species indicate that anaerobic glycolysis cannot fuel active synapses due to the low ATP yield of this pathway. Thus, our results point to a unique strategy whereby synapses switch from oxidative to exclusively anaerobic glycolytic metabolism to preserve circuit function during prolonged energy limitations.
| S-EPMC10022038 | biostudies-literature
Project description:Events that are important to an individual's life trigger neuromodulator release in brain areas responsible for cognitive and behavioral function. While it is well known that the presence of neuromodulators such as dopamine and norepinephrine is required for memory consolidation, the impact of neuromodulator concentration is, however, less understood. In a recurrent spiking neural network model featuring neuromodulator-dependent synaptic tagging and capture, we study how synaptic memory consolidation depends on the amount of neuromodulator present in the minutes to hours after learning. We find that the storage of rate-based and spike timing-based information is controlled by the level of neuromodulation. Specifically, we find better recall of temporal information for high levels of neuromodulation, while we find better recall of rate-coded spatial patterns for lower neuromodulation, mediated by the selection of different groups of synapses for consolidation. Hence, our results indicate that in minutes to hours after learning, the level of neuromodulation may alter the process of synaptic consolidation to ultimately control which type of information becomes consolidated in the recurrent neural network.
| S-EPMC9588040 | biostudies-literature
Project description:The dynamics of neural networks is often characterized by collective behavior and quasi-synchronous events, where a large fraction of neurons fire in short time intervals, separated by uncorrelated firing activity. These global temporal signals are crucial for brain functioning. They strongly depend on the topology of the network and on the fluctuations of the connectivity. We propose a heterogeneous mean-field approach to neural dynamics on random networks, that explicitly preserves the disorder in the topology at growing network sizes, and leads to a set of self-consistent equations. Within this approach, we provide an effective description of microscopic and large scale temporal signals in a leaky integrate-and-fire model with short term plasticity, where quasi-synchronous events arise. Our equations provide a clear analytical picture of the dynamics, evidencing the contributions of both periodic (locked) and aperiodic (unlocked) neurons to the measurable average signal. In particular, we formulate and solve a global inverse problem of reconstructing the in-degree distribution from the knowledge of the average activity field. Our method is very general and applies to a large class of dynamical models on dense random networks.
| S-EPMC3949294 | biostudies-other
Project description:Correlated neural activities such as synchronizations can significantly alter the characteristics of spike transfer between neural layers. However, it is not clear how this synchronization-dependent spike transfer can be affected by the structure of convergent feedforward wiring. To address this question, we implemented computer simulations of model neural networks: a source and a target layer connected with different types of convergent wiring rules. In the Gaussian-Gaussian (GG) model, both the connection probability and the strength are given as Gaussian distribution as a function of spatial distance. In the Uniform-Constant (UC) and Uniform-Exponential (UE) models, the connection probability density is a uniform constant within a certain range, but the connection strength is set as a constant value or an exponentially decaying function, respectively. Then we examined how the spike transfer function is modulated under these conditions, while static or synchronized input patterns were introduced to simulate different levels of feedforward spike synchronization. We observed that the synchronization-dependent modulation of the transfer function appeared noticeably different for each convergence condition. The modulation of the spike transfer function was largest in the UC model, and smallest in the UE model. Our analysis showed that this difference was induced by the different spike weight distributions that was generated from convergent synapses in each model. Our results suggest that, the structure of the feedforward convergence is a crucial factor for correlation-dependent spike control, thus must be considered important to understand the mechanism of information transfer in the brain.
| S-EPMC5691111 | biostudies-literature