Browse
Submit Data
Databases
API
Help

Dataset Information

21 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Trainable hardware for dynamical computing using error backpropagation through physical media.

ABSTRACT: Neural networks are currently implemented on digital Von Neumann machines, which do not fully leverage their intrinsic parallelism. We demonstrate how to use a novel class of reconfigurable dynamical systems for analogue information processing, mitigating this problem. Our generic hardware platform for dynamic, analogue computing consists of a reciprocal linear dynamical system with nonlinear feedback. Thanks to reciprocity, a ubiquitous property of many physical phenomena like the propagation of light and sound, the error backpropagation-a crucial step for tuning such systems towards a specific task-can happen in hardware. This can potentially speed up the optimization process significantly, offering important benefits for the scalability of neuro-inspired hardware. In this paper, we show, using one experimentally validated and one conceptual example, that such systems may provide a straightforward mechanism for constructing highly scalable, fully dynamical analogue computers.

SUBMITTER: Hermans M

PROVIDER: S-EPMC4382991 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Hardware implementation of backpropagation using progressive gradient descent for in situ training of multilayer neural networks.

Project description:Neural network training can be slow and energy-expensive due to the frequent transfer of weight data between digital memory and processing units. Neuromorphic systems can accelerate neural networks by performing multiply-accumulate operations in parallel using nonvolatile analog memory. However, executing the widely used backpropagation training algorithm in multilayer neural networks requires information-and therefore storage-of the partial derivatives of the weight values preventing suitable and scalable implementation in hardware. Here, we propose a hardware implementation of the backpropagation algorithm that progressively updates each layer using in situ stochastic gradient descent, avoiding this storage requirement. We experimentally demonstrate the in situ error calculation and the proposed progressive backpropagation method in a multilayer hardware-implemented neural network. We confirm identical learning characteristics and classification performance compared to conventional backpropagation in software. We show that our approach can be scaled to large and deep neural networks, enabling highly efficient training of advanced artificial intelligence computing systems.

| S-EPMC11244533 | biostudies-literature

Exponential rise of dynamical complexity in quantum computing through projections.

Project description:The ability of quantum systems to host exponentially complex dynamics has the potential to revolutionize science and technology. Therefore, much effort has been devoted to developing of protocols for computation, communication and metrology, which exploit this scaling, despite formidable technical difficulties. Here we show that the mere frequent observation of a small part of a quantum system can turn its dynamics from a very simple one into an exponentially complex one, capable of universal quantum computation. After discussing examples, we go on to show that this effect is generally to be expected: almost any quantum dynamics becomes universal once 'observed' as outlined above. Conversely, we show that any complex quantum dynamics can be 'purified' into a simpler one in larger dimensions. We conclude by demonstrating that even local noise can lead to an exponentially complex dynamics.

| S-EPMC4214416 | biostudies-other

Today's computing challenges: opportunities for computer hardware design.

Project description:Due to the explosive increase of digital data creation, demand on advancement of computing capability is ever increasing. However, the legacy approaches that we have used for continuous improvement of three elements of computer (process, memory, and interconnect) have started facing their limits, and therefore are not as effective as they used to be and are also expected to reach the end in the near future. Evidently, it is a large challenge for computer hardware industry. However, at the same time it also provides great opportunities for the hardware design industry to develop novel technologies and to take leadership away from incumbents. This paper reviews the technical challenges that today's computing systems are facing and introduces potential directions for continuous advancement of computing capability, and discusses where computer hardware designers find good opportunities to contribute.

| S-EPMC8022507 | biostudies-literature

Somato-dendritic Synaptic Plasticity and Error-backpropagation in Active Dendrites.

Project description:In the last decade dendrites of cortical neurons have been shown to nonlinearly combine synaptic inputs by evoking local dendritic spikes. It has been suggested that these nonlinearities raise the computational power of a single neuron, making it comparable to a 2-layer network of point neurons. But how these nonlinearities can be incorporated into the synaptic plasticity to optimally support learning remains unclear. We present a theoretically derived synaptic plasticity rule for supervised and reinforcement learning that depends on the timing of the presynaptic, the dendritic and the postsynaptic spikes. For supervised learning, the rule can be seen as a biological version of the classical error-backpropagation algorithm applied to the dendritic case. When modulated by a delayed reward signal, the same plasticity is shown to maximize the expected reward in reinforcement learning for various coding scenarios. Our framework makes specific experimental predictions and highlights the unique advantage of active dendrites for implementing powerful synaptic plasticity rules that have access to downstream information via backpropagation of action potentials.

| S-EPMC4739747 | biostudies-literature

Random synaptic feedback weights support error backpropagation for deep learning.

Project description:The brain processes information through multiple layers of neurons. This deep architecture is representationally powerful, but complicates learning because it is difficult to identify the responsible neurons when a mistake is made. In machine learning, the backpropagation algorithm assigns blame by multiplying error signals with all the synaptic weights on each neuron's axon and further downstream. However, this involves a precise, symmetric backward connectivity pattern, which is thought to be impossible in the brain. Here we demonstrate that this strong architectural constraint is not required for effective error propagation. We present a surprisingly simple mechanism that assigns blame by multiplying errors by even random synaptic weights. This mechanism can transmit teaching signals across multiple layers of neurons and performs as effectively as backpropagation on a variety of tasks. Our results help reopen questions about how the brain could use error signals and dispel long-held assumptions about algorithmic constraints on learning.

| S-EPMC5105169 | biostudies-literature

Physical Implementation of Reservoir Computing through Electrochemical Reaction.

Project description:Nonlinear dynamical systems serving reservoir computing enrich the physical implementation of computing systems. A method for building physical reservoirs from electrochemical reactions is provided, and the potential of chemical dynamics as computing resources is shown. The essence of signal processing in such systems includes various degrees of ionic currents which pass through the solution as well as the electrochemical current detected based on a multiway data acquisition system to achieve switchable and parallel testing. The results show that they have respective advantages in periodic signals and temporal dynamic signals. Polyoxometalate molecule in the solution increases the diversity of the response current and thus improves their abilities to predict periodic signals. Conversely, distilled water exhibits great computing power in solving a second-order nonlinear problem. It is expected that these results will lead to further exploration of ionic conductance as a nonlinear dynamical system and provide more support for novel devices as computing resources.

| S-EPMC8867144 | biostudies-literature

Deep physical neural networks trained with backpropagation.

Project description:Deep-learning models have become pervasive tools in science and engineering. However, their energy requirements now increasingly limit their scalability1. Deep-learning accelerators2-9 aim to perform deep learning energy-efficiently, usually targeting the inference phase and often by exploiting physical substrates beyond conventional electronics. Approaches so far10-22 have been unable to apply the backpropagation algorithm to train unconventional novel hardware in situ. The advantages of backpropagation have made it the de facto training method for large-scale neural networks, so this deficiency constitutes a major impediment. Here we introduce a hybrid in situ-in silico algorithm, called physics-aware training, that applies backpropagation to train controllable physical systems. Just as deep learning realizes computations with deep neural networks made from layers of mathematical functions, our approach allows us to train deep physical neural networks made from layers of controllable physical systems, even when the physical layers lack any mathematical isomorphism to conventional artificial neural network layers. To demonstrate the universality of our approach, we train diverse physical neural networks based on optics, mechanics and electronics to experimentally perform audio and image classification tasks. Physics-aware training combines the scalability of backpropagation with the automatic mitigation of imperfections and noise achievable with in situ algorithms. Physical neural networks have the potential to perform machine learning faster and more energy-efficiently than conventional electronic processors and, more broadly, can endow physical systems with automatically designed physical functionalities, for example, for robotics23-26, materials27-29 and smart sensors30-32.

| S-EPMC8791835 | biostudies-literature

Ultra-fast data-mining hardware architecture based on stochastic computing.

Project description:Minimal hardware implementations able to cope with the processing of large amounts of data in reasonable times are highly desired in our information-driven society. In this work we review the application of stochastic computing to probabilistic-based pattern-recognition analysis of huge database sets. The proposed technique consists in the hardware implementation of a parallel architecture implementing a similarity search of data with respect to different pre-stored categories. We design pulse-based stochastic-logic blocks to obtain an efficient pattern recognition system. The proposed architecture speeds up the screening process of huge databases by a factor of 7 when compared to a conventional digital implementation using the same hardware area.

| S-EPMC4425430 | biostudies-literature

Molecular Energy Landscapes of Hardware-Efficient Ansatze in Quantum Computing.

Project description:Rapid advances in quantum computing have opened up new opportunities for solving the central electronic structure problem in computational chemistry. In the noisy intermediate-scale quantum (NISQ) era, where qubit coherence times are limited, it is essential to exploit quantum algorithms with sufficiently short quantum circuits to maximize qubit efficiency. The procedural construction of hardware-efficient ansätze provides one approach to design such circuits. However, refining the accuracy of the global minimum by increasing circuit depth may lead to a proliferation of local minima that hinders global optimization. To investigate this phenomenon, we explore the energy landscapes of hardware-efficient circuits to identify ground-state energies of the hydrogen, lithium hydride, and beryllium hydride molecules. We also propose a simple dimensionality reduction procedure that reduces quantum gate depth while retaining high accuracy for the global minimum, simplifying the energy landscape, and hence speeding up optimization from both software and hardware perspectives.

| S-EPMC9979602 | biostudies-literature

Leveraging plant physiological dynamics using physical reservoir computing.

Project description:Plants are complex organisms subject to variable environmental conditions, which influence their physiology and phenotype dynamically. We propose to interpret plants as reservoirs in physical reservoir computing. The physical reservoir computing paradigm originates from computer science; instead of relying on Boolean circuits to perform computations, any substrate that exhibits complex non-linear and temporal dynamics can serve as a computing element. Here, we present the first application of physical reservoir computing with plants. In addition to investigating classical benchmark tasks, we show that Fragaria × ananassa (strawberry) plants can solve environmental and eco-physiological tasks using only eight leaf thickness sensors. Although the results indicate that plants are not suitable for general-purpose computation but are well-suited for eco-physiological tasks such as photosynthetic rate and transpiration rate. Having the means to investigate the information processing by plants improves quantification and understanding of integrative plant responses to dynamic changes in their environment. This first demonstration of physical reservoir computing with plants is key for transitioning towards a holistic view of phenotyping and early stress detection in precision agriculture applications since physical reservoir computing enables us to analyse plant responses in a general way: environmental changes are processed by plants to optimise their phenotype.

| S-EPMC9307625 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data