Implementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy
Ontology highlight
ABSTRACT:
SUBMITTER: Huang H
PROVIDER: S-EPMC7509653 | biostudies-literature | 2020 Jul
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:We describe the memristive properties of cluster-assembled gold films. We show that resistive switching is observed in pure metallic nanostructured films at room temperature and atmospheric pressure, in response to applied voltage inputs. In particular, we observe resistance changes up to 400% and archetypal switching events that have remarkable symmetry with the applied voltage. We associated this symmetry with 'potentiation' and 'anti-potentiation' processes involving the activation of synapses and of pathways comprising multiple synapses. The stability and reproducibility of the resistance switching, which lasted over many hours, make these devices ideal test-beds for exploration of the basic mechanisms of the switching processes, and allow convenient fabrication of devices that may have neuromorphic properties.
| S-EPMC5554187 | biostudies-literature
Project description:While there is an abundance of research on neural networks that are "inspired" by the brain, few mimic the critical temporal compute features that allow the brain to efficiently perform complex computations. Even fewer methods emulate the heterogeneity of learning produced by biological neurons. Memory devices, such as memristors, are also investigated for their potential to implement neuronal functions in electronic hardware. However, memristors in computing architectures typically operate as non-volatile memories, either as storage or as the weights in a multiply-and-accumulate function that requires direct access to manipulate memristance via a costly learning algorithm. Hence, the integration of memristors into architectures as time-dependent computational units is studied, starting with the development of a compact and versatile mathematical model that is capable of emulating flux-linkage controlled analog (FLCA) memristors and their unique temporal characteristics. The proposed model, which is validated against experimental FLCA LixNbO2 intercalation devices, is used to create memristive circuits that mimic neuronal behavior such as desensitization, paired-pulse facilitation, and spike-timing-dependent plasticity. The model is used to demonstrate building blocks of biomimetic learning via dynamical memristive circuits that implement biomimetic learning rules in a self-training neural network, with dynamical memristive weights that are capable of associative lifelong learning. Successful training of the dynamical memristive neural network to perform image classification of handwritten digits is shown, including lifelong learning by having the dynamical memristive network relearn different characters in succession. An analog computing architecture that learns to associate input-to-input correlations is also introduced, with examples demonstrating image classification and pattern recognition without convolution. The biomimetic functions shown in this paper result from fully ion-driven memristive circuits devoid of integrating capacitors and thus are instructive for exploiting the immense potential of memristive technology for neuromorphic computation in hardware and allowing a common architecture to be applied to a wide range of learning rules, including STDP, magnitude, frequency, and pulse shape among others, to enable an inorganic implementation of the complex heterogeneity of biological neural systems.
| S-EPMC10157062 | biostudies-literature
Project description:Bayesian networks are powerful statistical models to understand causal relationships in real-world probabilistic problems such as diagnosis, forecasting, computer vision, etc. For systems that involve complex causal dependencies among many variables, the complexity of the associated Bayesian networks become computationally intractable. As a result, direct hardware implementation of these networks is one promising approach to reducing power consumption and execution time. However, the few hardware implementations of Bayesian networks presented in literature rely on deterministic CMOS devices that are not efficient in representing the stochastic variables in a Bayesian network that encode the probability of occurrence of the associated event. This work presents an experimental demonstration of a Bayesian network building block implemented with inherently stochastic spintronic devices based on the natural physics of nanomagnets. These devices are based on nanomagnets with perpendicular magnetic anisotropy, initialized to their hard axes by the spin orbit torque from a heavy metal under-layer utilizing the giant spin Hall effect, enabling stochastic behavior. We construct an electrically interconnected network of two stochastic devices and manipulate the correlations between their states by changing connection weights and biases. By mapping given conditional probability tables to the circuit hardware, we demonstrate that any two node Bayesian networks can be implemented by our stochastic network. We then present the stochastic simulation of an example case of a four node Bayesian network using our proposed device, with parameters taken from the experiment. We view this work as a first step towards the large scale hardware implementation of Bayesian networks.
| S-EPMC7524796 | biostudies-literature
Project description:Neuromorphic computing has emerged as one of the most promising paradigms to overcome the limitations of von Neumann architecture of conventional digital processors. The aim of neuromorphic computing is to faithfully reproduce the computing processes in the human brain, thus paralleling its outstanding energy efficiency and compactness. Toward this goal, however, some major challenges have to be faced. Since the brain processes information by high-density neural networks with ultra-low power consumption, novel device concepts combining high scalability, low-power operation, and advanced computing functionality must be developed. This work provides an overview of the most promising device concepts in neuromorphic computing including complementary metal-oxide semiconductor (CMOS) and memristive technologies. First, the physics and operation of CMOS-based floating-gate memory devices in artificial neural networks will be addressed. Then, several memristive concepts will be reviewed and discussed for applications in deep neural network and spiking neural network architectures. Finally, the main technology challenges and perspectives of neuromorphic computing will be discussed.
| S-EPMC6981548 | biostudies-literature
Project description:Neuromorphic architectures offer great promise for achieving computation capacities beyond conventional Von Neumann machines. The essential elements for achieving this vision are highly scalable synaptic mimics that do not undermine biological fidelity. Here we demonstrate that single solid-state TiO2 memristors can exhibit non-associative plasticity phenomena observed in biological synapses, supported by their metastable memory state transition properties. We show that, contrary to conventional uses of solid-state memory, the existence of rate-limiting volatility is a key feature for capturing short-term synaptic dynamics. We also show how the temporal dynamics of our prototypes can be exploited to implement spatio-temporal computation, demonstrating the memristors full potential for building biophysically realistic neural processing systems.
| S-EPMC4698662 | biostudies-literature
Project description:Random telegraph noise is a widely investigated phenomenon affecting the reliability of the reading operation of the class of memristive devices whose operation relies on formation and dissolution of conductive filaments. The trap and the release of electrons into and from defects surrounding the filament produce current fluctuations at low read voltages. In this work, telegraphic resistance variations are intentionally stimulated through pulse trains in HfO2-based memristive devices. The stimulated noise results from the re-arrangement of ionic defects constituting the filament responsible for the switching. Therefore, the stimulated noise has an ionic origin in contrast to the electronic nature of conventional telegraph noise. The stimulated noise is interpreted as raising from a dynamic equilibrium establishing from the tendencies of ionic drift and diffusion acting on the edges of conductive filament. We present a model that accounts for the observed increase of noise amplitude with the average device resistance. This work provides the demonstration and the physical foundation for the intentional stimulation of ionic telegraph noise which, on one hand, affects the programming operations performed with trains of identical pulses, as for neuromorphic computing, and on the other hand, it can open opportunities for applications relying on stochastic processes in nanoscaled devices.
| S-EPMC6465356 | biostudies-literature
Project description:Biomolecular materials offer tremendous potential for the development of memristive devices due to their low cost of production, environmental friendliness, and, most notably, biocompatibility. Herein, biocompatible memristive devices based on amyloid-gold nanoparticle hybrids have been investigated. These memristors demonstrate excellent electrical performance, featuring an ultrahigh Roff/Ron ratio (>107), a low switching voltage (<0.8 V), and reliable reproducibility. Additionally, the reversible transition from threshold switching to resistive switching mode was achieved in this work. The arrangement of peptides in amyloid fibrils endows the surface polarity and phenylalanine packing, which provides channels for the migration of Ag ions in the memristors. By modulating voltage pulse signals, the study successfully imitates the synaptic behavior of excitatory postsynaptic current (EPSC), paired-pulse facilitation (PPF), and the transition from short-term plasticity (STP) to long-term plasticity (LTP). More interestingly, Boolean logic standard cells were designed and simulated using the memristive devices. The fundamental and experimental results of this study thus offer insights into the utilization of biomolecular materials for advanced memristive devices.
| S-EPMC10004345 | biostudies-literature
Project description:Redox-based resistive switching random access memory (ReRAM) offers excellent properties to implement future non-volatile memory arrays. Recently, the capability of two-state ReRAMs to implement Boolean logic functionality gained wide interest. Here, we report on seven-states Tantalum Oxide Devices, which enable the realization of an intrinsic modular arithmetic using a ternary number system. Modular arithmetic, a fundamental system for operating on numbers within the limit of a modulus, is known to mathematicians since the days of Euclid and finds applications in diverse areas ranging from e-commerce to musical notations. We demonstrate that multistate devices not only reduce the storage area consumption drastically, but also enable novel in-memory operations, such as computing using high-radix number systems, which could not be implemented using two-state devices. The use of high radix number system reduces the computational complexity by reducing the number of needed digits. Thus the number of calculation operations in an addition and the number of logic devices can be reduced.
| S-EPMC5105152 | biostudies-literature
Project description:This research explores a novel two-terminal heterostructure of the Pt/Cu2Se/Sb2Se3/FTO memristor, which exhibited essential biological synaptic functions. These synaptic functions play a critical role in emulating biological neural systems and overcoming the limitations of traditional computing architectures. By repeating a fixed pulse train, in this study, we realized a few crucial neural functions toward weight modulation, such as nonlinear conductance changes and potentiation/depression characteristics, which aid the transition of short-term memory to long-term memory. However, we also employed multilevel switching, which provides easily accessible multilevel (4-states, 2-bit) states, for high-density data storage capability along with endurance (102 pulse cycles for each state) in our proposed device. In terms of synaptic plasticity, the device performed well by controlling the pulse voltage and pulse width during excitatory post-synaptic current (EPSC) measurements. The spike-time-dependent plasticity (STDP) highlights their outstanding functional properties, indicating that the device can be used in artificial biological synapse applications. The artificial neural network with Pt/Cu2Se/Sb2Se3/FTO achieved a significant accuracy of 73% in the simulated Modified National Institute of Standards and Technology database (MNIST) pattern. The conduction mechanism of resistive switching and the artificial synaptic phenomena could be attributed to the transfer of Se2- ions and selenium vacancies. The neuromorphic characteristics of the Pt/Cu2Se/Sb2Se3/FTO devices demonstrate their potential as futuristic synaptic devices.
| S-EPMC10142452 | biostudies-literature
Project description:To reach the energy efficiency and the computing capability of biological neural networks, novel hardware systems and paradigms are required where the information needs to be processed in both spatial and temporal domains. Resistive switching memory (RRAM) devices appear as key enablers for the implementation of large-scale neuromorphic computing systems with high energy efficiency and extended scalability. Demonstrating a full set of spatiotemporal primitives with RRAM-based circuits remains an open challenge. By taking inspiration from the neurobiological processes in the human auditory systems, we develop neuromorphic circuits for memristive tonotopic mapping via volatile RRAM devices. Based on a generalized stochastic device-level approach, we demonstrate the main features of signal processing of cochlea, namely logarithmic integration and tonotopic mapping of signals. We also show that our tonotopic classification is suitable for speech recognition. These results support memristive devices for physical processing of temporal signals, thus paving the way for energy efficient, high density neuromorphic systems.
| S-EPMC10985068 | biostudies-literature