Neuromorphic Binarized Polariton Networks.
Ontology highlight
ABSTRACT:
SUBMITTER: Mirek R
PROVIDER: S-EPMC8155323 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:While deep neural networks have surpassed human performance in multiple situations, they are prone to catastrophic forgetting: upon training a new task, they rapidly forget previously learned ones. Neuroscience studies, based on idealized tasks, suggest that in the brain, synapses overcome this issue by adjusting their plasticity depending on their past history. However, such "metaplastic" behaviors do not transfer directly to mitigate catastrophic forgetting in deep neural networks. In this work, we interpret the hidden weights used by binarized neural networks, a low-precision version of deep neural networks, as metaplastic variables, and modify their training technique to alleviate forgetting. Building on this idea, we propose and demonstrate experimentally, in situations of multitask and stream learning, a training technique that reduces catastrophic forgetting without needing previously presented data, nor formal boundaries between datasets and with performance approaching more mainstream techniques with task boundaries. We support our approach with a theoretical analysis on a tractable task. This work bridges computational neuroscience and deep learning, and presents significant assets for future embedded and neuromorphic systems, especially when using novel nanodevices featuring physics analogous to metaplasticity.
| S-EPMC8100137 | biostudies-literature
Project description:Efforts to emulate the formidable information processing capabilities of the brain through neuromorphic engineering have been bolstered by recent progress in the fabrication of nonlinear, nanoscale circuit elements that exhibit synapse-like operational characteristics. However, conventional fabrication techniques are unable to efficiently generate structures with the highly complex interconnectivity found in biological neuronal networks. Here we demonstrate the physical realization of a self-assembled neuromorphic device which implements basic concepts of systems neuroscience through a hardware-based platform comprised of over a billion interconnected atomic-switch inorganic synapses embedded in a complex network of silver nanowires. Observations of network activation and passive harmonic generation demonstrate a collective response to input stimulus in agreement with recent theoretical predictions. Further, emergent behaviors unique to the complex network of atomic switches and akin to brain function are observed, namely spatially distributed memory, recurrent dynamics and the activation of feedforward subnetworks. These devices display the functional characteristics required for implementing unconventional, biologically and neurally inspired computational methodologies in a synthetic experimental system.
| S-EPMC3412809 | biostudies-literature
Project description:Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamics arising from the interplay of their synapse-like electrical junctions and their complex network topology. Additionally, various information processing tasks have been demonstrated with neuromorphic nanowire networks. Here, we investigate the dynamics of how these unique systems process information through information-theoretic metrics. In particular, Transfer Entropy (TE) and Active Information Storage (AIS) are employed to investigate dynamical information flow and short-term memory in nanowire networks. In addition to finding that the topologically central parts of networks contribute the most to the information flow, our results also reveal TE and AIS are maximized when the networks transitions from a quiescent to an active state. The performance of neuromorphic networks in memory and learning tasks is demonstrated to be dependent on their internal dynamical states as well as topological structure. Optimal performance is found when these networks are pre-initialised to the transition state where TE and AIS are maximal. Furthermore, an optimal range of information processing resources (i.e. connectivity density) is identified for performance. Overall, our results demonstrate information dynamics is a valuable tool to study and benchmark neuromorphic systems.
| S-EPMC8219687 | biostudies-literature
Project description:Neuromorphic networks are formed by random self-assembly of silver nanowires. Silver nanowires are coated with a polymer layer after synthesis in which junctions between two nanowires act as resistive switches, often compared with neurosynapses. We analyze the role of single junction switching in the dynamical properties of the neuromorphic network. Network transitions to a high-conductance state under the application of a voltage bias higher than a threshold value. The stability and permanence of this state is studied by shifting the voltage bias in order to activate or deactivate the network. A model of the electrical network with atomic switches reproduces the relation between individual nanowire junctions switching events with current pathway formation or destruction. This relation is further manifested in changes in 1/f power-law scaling of the spectral distribution of current. The current fluctuations involved in this scaling shift are considered to arise from an essential equilibrium between formation, stochastic-mediated breakdown of individual nanowire-nanowire junctions and the onset of different current pathways that optimize power dissipation. This emergent dynamics shown by polymer-coated Ag nanowire networks places this system in the class of optimal transport networks, from which new fundamental parallels with neural dynamics and natural computing problem-solving can be drawn.
| S-EPMC6797708 | biostudies-literature
Project description:Graph theory has been extensively applied to the topological mapping of complex networks, ranging from social networks to biological systems. Graph theory has increasingly been applied to neuroscience as a method to explore the fundamental structural and functional properties of human neural networks. Here, we apply graph theory to a model of a novel neuromorphic system constructed from self-assembled nanowires, whose structure and function may mimic that of human neural networks. Simulations of neuromorphic nanowire networks allow us to directly examine their topology at the individual nanowire-node scale. This type of investigation is currently extremely difficult experimentally. We then apply network cartographic approaches to compare neuromorphic nanowire networks with: random networks (including an untrained artificial neural network); grid-like networks and the structural network of C. elegans. Our results demonstrate that neuromorphic nanowire networks exhibit a small-world architecture similar to the biological system of C. elegans, and significantly different from random and grid-like networks. Furthermore, neuromorphic nanowire networks appear more segregated and modular than random, grid-like and simple biological networks and more clustered than artificial neural networks. Given the inextricable link between structure and function in neural networks, these results may have important implications for mimicking cognitive functions in neuromorphic nanowire networks.
| S-EPMC7069063 | biostudies-literature
Project description:Photonic systems for high-performance information processing have attracted renewed interest. Neuromorphic silicon photonics has the potential to integrate processing functions that vastly exceed the capabilities of electronics. We report first observations of a recurrent silicon photonic neural network, in which connections are configured by microring weight banks. A mathematical isomorphism between the silicon photonic circuit and a continuous neural network model is demonstrated through dynamical bifurcation analysis. Exploiting this isomorphism, a simulated 24-node silicon photonic neural network is programmed using "neural compiler" to solve a differential system emulation task. A 294-fold acceleration against a conventional benchmark is predicted. We also propose and derive power consumption analysis for modulator-class neurons that, as opposed to laser-class neurons, are compatible with silicon photonic platforms. At increased scale, Neuromorphic silicon photonics could access new regimes of ultrafast information processing for radio, control, and scientific computing.
| S-EPMC5547135 | biostudies-literature
Project description:Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that (i) approach state-of-the-art classification accuracy across eight standard datasets encompassing vision and speech, (ii) perform inference while preserving the hardware's underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1,200 and 2,600 frames/s and using between 25 and 275 mW (effectively >6,000 frames/s per Watt), and (iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. This approach allows the algorithmic power of deep learning to be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.
| S-EPMC5068316 | biostudies-literature
Project description:The brain performs intelligent tasks with extremely low energy consumption. This work takes its inspiration from two strategies used by the brain to achieve this energy efficiency: the absence of separation between computing and memory functions and reliance on low-precision computation. The emergence of resistive memory technologies indeed provides an opportunity to tightly co-integrate logic and memory in hardware. In parallel, the recently proposed concept of a Binarized Neural Network, where multiplications are replaced by exclusive NOR (XNOR) logic gates, offers a way to implement artificial intelligence using very low precision computation. In this work, we therefore propose a strategy for implementing low-energy Binarized Neural Networks that employs brain-inspired concepts while retaining the energy benefits of digital electronics. We design, fabricate, and test a memory array, including periphery and sensing circuits, that is optimized for this in-memory computing scheme. Our circuit employs hafnium oxide resistive memory integrated in the back end of line of a 130-nm CMOS process, in a two-transistor, two-resistor cell, which allows the exclusive NOR operations of the neural network to be performed directly within the sense amplifiers. We show, based on extensive electrical measurements, that our design allows a reduction in the number of bit errors on the synaptic weights without the use of formal error-correcting codes. We design a whole system using this memory array. We show on standard machine learning tasks (MNIST, CIFAR-10, ImageNet, and an ECG task) that the system has inherent resilience to bit errors. We evidence that its energy consumption is attractive compared to more standard approaches and that it can use memory devices in regimes where they exhibit particularly low programming energy and high endurance. We conclude the work by discussing how it associates biologically plausible ideas with more traditional digital electronics concepts.
| S-EPMC6962102 | biostudies-literature
Project description:Recent advances in bioelectronics and neural engineering allowed the development of brain machine interfaces and neuroprostheses, capable of facilitating or recovering functionality in people with neurological disability. To realize energy-efficient and real-time capable devices, neuromorphic computing systems are envisaged as the core of next-generation systems for brain repair. We demonstrate here a real-time hardware neuromorphic prosthesis to restore bidirectional interactions between two neuronal populations, even when one is damaged or missing. We used in vitro modular cell cultures to mimic the mutual interaction between neuronal assemblies and created a focal lesion to functionally disconnect the two populations. Then, we employed our neuromorphic prosthesis for bidirectional bridging to artificially reconnect two disconnected neuronal modules and for hybrid bidirectional bridging to replace the activity of one module with a real-time hardware neuromorphic Spiking Neural Network. Our neuroprosthetic system opens avenues for the exploitation of neuromorphic-based devices in bioelectrical therapeutics for health care.
| S-EPMC6706626 | biostudies-literature
Project description:Biological neuronal networks are the computing engines of the mammalian brain. These networks exhibit structural characteristics such as hierarchical architectures, small-world attributes, and scale-free topologies, providing the basis for the emergence of rich temporal characteristics such as scale-free dynamics and long-range temporal correlations. Devices that have both the topological and the temporal features of a neuronal network would be a significant step toward constructing a neuromorphic system that can emulate the computational ability and energy efficiency of the human brain. Here we use numerical simulations to show that percolating networks of nanoparticles exhibit structural properties that are reminiscent of biological neuronal networks, and then show experimentally that stimulation of percolating networks by an external voltage stimulus produces temporal dynamics that are self-similar, follow power-law scaling, and exhibit long-range temporal correlations. These results are expected to have important implications for the development of neuromorphic devices, especially for those based on the concept of reservoir computing.
| S-EPMC7286302 | biostudies-literature