Constructing optimized binary masks for reservoir computing with delay systems.
Ontology highlight
ABSTRACT:
SUBMITTER: Appeltant L
PROVIDER: S-EPMC3887384 | biostudies-other | 2014
REPOSITORIES: biostudies-other
Similar Datasets
Project description:Delays are ubiquitous in biological systems, ranging from genetic regulatory networks and synaptic conductances, to predator/pray population interactions. The evidence is mounting, not only to the presence of delays as physical constraints in signal propagation speed, but also to their functional role in providing dynamical diversity to the systems that comprise them. The latter observation in biological systems inspired the recent development of a computational architecture that harnesses this dynamical diversity, by delay-coupling a single nonlinear element to itself. This architecture is a particular realization of Reservoir Computing, where stimuli are injected into the system in time rather than in space as is the case with classical recurrent neural network realizations. This architecture also exhibits an internal memory which fades in time, an important prerequisite to the functioning of any reservoir computing device. However, fading memory is also a limitation to any computation that requires persistent storage. In order to overcome this limitation, the current work introduces an extended version to the single node Delay-Coupled Reservoir, that is based on trained linear feedback. We show by numerical simulations that adding task-specific linear feedback to the single node Delay-Coupled Reservoir extends the class of solvable tasks to those that require nonfading memory. We demonstrate, through several case studies, the ability of the extended system to carry out complex nonlinear computations that depend on past information, whereas the computational power of the system with fading memory alone quickly deteriorates. Our findings provide the theoretical basis for future physical realizations of a biologically-inspired ultrafast computing device with extended functionality.
| S-EPMC5081200 | biostudies-literature
Project description:Reservoir computing is a recently introduced, highly efficient bio-inspired approach for processing time dependent data. The basic scheme of reservoir computing consists of a non linear recurrent dynamical system coupled to a single input layer and a single output layer. Within these constraints many implementations are possible. Here we report an optoelectronic implementation of reservoir computing based on a recently proposed architecture consisting of a single non linear node and a delay line. Our implementation is sufficiently fast for real time information processing. We illustrate its performance on tasks of practical importance such as nonlinear channel equalization and speech recognition, and obtain results comparable to state of the art digital implementations.
| S-EPMC3286854 | biostudies-other
Project description:Photonic implementations of reservoir computing (RC) promise to reach ultra-high bandwidth of operation with moderate training efforts. Several optoelectronic demonstrations reported state of the art performances for hard tasks as speech recognition, object classification and time series prediction. Scaling these systems in space and time faces challenges in control complexity, size and power demand, which can be relieved by integrated optical solutions. Silicon photonics can be the disruptive technology to achieve this goal. However, the experimental demonstrations have been so far focused on spatially distributed reservoirs, where the massive use of splitters/combiners and the interconnection loss limits the number of nodes. Here, we propose and validate an all optical RC scheme based on a silicon microring (MR) and time multiplexing. The input layer is encoded in the intensity of a pump beam, which is nonlinearly transferred to the free carrier concentration in the MR and imprinted on a secondary probe. We harness the free carrier dynamics to create a chain-like reservoir topology with 50 virtual nodes. We give proof of concept demonstrations of RC by solving two nontrivial tasks: the delayed XOR and the classification of Iris flowers. This forms the basic building block from which larger hybrid spatio-temporal reservoirs with thousands of nodes can be realized with a limited set of resources.
| S-EPMC8329232 | biostudies-literature
Project description:Reservoir computing is a best-in-class machine learning algorithm for processing information generated by dynamical systems using observed time-series data. Importantly, it requires very small training data sets, uses linear optimization, and thus requires minimal computing resources. However, the algorithm uses randomly sampled matrices to define the underlying recurrent neural network and has a multitude of metaparameters that must be optimized. Recent results demonstrate the equivalence of reservoir computing to nonlinear vector autoregression, which requires no random matrices, fewer metaparameters, and provides interpretable results. Here, we demonstrate that nonlinear vector autoregression excels at reservoir computing benchmark tasks and requires even shorter training data sets and training time, heralding the next generation of reservoir computing.
| S-EPMC8455577 | biostudies-literature
Project description:Reservoir computing is a brain-inspired machine learning framework that employs a signal-driven dynamical system, in particular harnessing common-signal-induced synchronization which is a widely observed nonlinear phenomenon. Basic understanding of a working principle in reservoir computing can be expected to shed light on how information is stored and processed in nonlinear dynamical systems, potentially leading to progress in a broad range of nonlinear sciences. As a first step toward this goal, from the viewpoint of nonlinear physics and information theory, we study the memory-nonlinearity trade-off uncovered by Dambre et al. (2012). Focusing on a variational equation, we clarify a dynamical mechanism behind the trade-off, which illustrates why nonlinear dynamics degrades memory stored in dynamical system in general. Moreover, based on the trade-off, we propose a mixture reservoir endowed with both linear and nonlinear dynamics and show that it improves the performance of information processing. Interestingly, for some tasks, significant improvements are observed by adding a few linear dynamics to the nonlinear dynamical system. By employing the echo state network model, the effect of the mixture reservoir is numerically verified for a simple function approximation task and for more complex tasks.
| S-EPMC5579006 | biostudies-literature
Project description:Chunking is the process by which frequently repeated segments of temporal inputs are concatenated into single units that are easy to process. Such a process is fundamental to time-series analysis in biological and artificial information processing systems. The brain efficiently acquires chunks from various information streams in an unsupervised manner; however, the underlying mechanisms of this process remain elusive. A widely-adopted statistical method for chunking consists of predicting frequently repeated contiguous elements in an input sequence based on unequal transition probabilities over sequence elements. However, recent experimental findings suggest that the brain is unlikely to adopt this method, as human subjects can chunk sequences with uniform transition probabilities. In this study, we propose a novel conceptual framework to overcome this limitation. In this process, neural networks learn to predict dynamical response patterns to sequence input rather than to directly learn transition patterns. Using a mutually supervising pair of reservoir computing modules, we demonstrate how this mechanism works in chunking sequences of letters or visual images with variable regularity and complexity. In addition, we demonstrate that background noise plays a crucial role in correctly learning chunks in this model. In particular, the model can successfully chunk sequences that conventional statistical approaches fail to chunk due to uniform transition probabilities. In addition, the neural responses of the model exhibit an interesting similarity to those of the basal ganglia observed after motor habit formation.
| S-EPMC6193738 | biostudies-literature
Project description:Reservoir computing is a temporal information processing system that exploits artificial or physical dissipative dynamics to learn a dynamical system and generate the target time-series. This paper proposes the use of real superconducting quantum computing devices as the reservoir, where the dissipative property is served by the natural noise added to the quantum bits. The performance of this natural quantum reservoir is demonstrated in a benchmark time-series regression problem and a practical problem classifying different objects based on temporal sensor data. In both cases the proposed reservoir computer shows a higher performance than a linear regression or classification model. The results indicate that a noisy quantum device potentially functions as a reservoir computer, and notably, the quantum noise, which is undesirable in the conventional quantum computation, can be used as a rich computation resource.
| S-EPMC8789868 | biostudies-literature
Project description:Plants are complex organisms subject to variable environmental conditions, which influence their physiology and phenotype dynamically. We propose to interpret plants as reservoirs in physical reservoir computing. The physical reservoir computing paradigm originates from computer science; instead of relying on Boolean circuits to perform computations, any substrate that exhibits complex non-linear and temporal dynamics can serve as a computing element. Here, we present the first application of physical reservoir computing with plants. In addition to investigating classical benchmark tasks, we show that Fragaria × ananassa (strawberry) plants can solve environmental and eco-physiological tasks using only eight leaf thickness sensors. Although the results indicate that plants are not suitable for general-purpose computation but are well-suited for eco-physiological tasks such as photosynthetic rate and transpiration rate. Having the means to investigate the information processing by plants improves quantification and understanding of integrative plant responses to dynamic changes in their environment. This first demonstration of physical reservoir computing with plants is key for transitioning towards a holistic view of phenotyping and early stress detection in precision agriculture applications since physical reservoir computing enables us to analyse plant responses in a general way: environmental changes are processed by plants to optimise their phenotype.
| S-EPMC9307625 | biostudies-literature
Project description:Network analysis is entering fields where network structures are unknown, such as psychology and the educational sciences. A crucial step in the application of network models lies in the assessment of network structure. Current methods either have serious drawbacks or are only suitable for Gaussian data. In the present paper, we present a method for assessing network structures from binary data. Although models for binary data are infamous for their computational intractability, we present a computationally efficient model for estimating network structures. The approach, which is based on Ising models as used in physics, combines logistic regression with model selection based on a Goodness-of-Fit measure to identify relevant relationships between variables that define connections in a network. A validation study shows that this method succeeds in revealing the most relevant features of a network for realistic sample sizes. We apply our proposed method to estimate the network of depression and anxiety symptoms from symptom scores of 1108 subjects. Possible extensions of the model are discussed.
| S-EPMC4118196 | biostudies-literature
Project description:Reservoir computing systems utilize dynamic reservoirs having short-term memory to project features from the temporal inputs into a high-dimensional feature space. A readout function layer can then effectively analyze the projected features for tasks, such as classification and time-series analysis. The system can efficiently compute complex and temporal data with low-training cost, since only the readout function needs to be trained. Here we experimentally implement a reservoir computing system using a dynamic memristor array. We show that the internal ionic dynamic processes of memristors allow the memristor-based reservoir to directly process information in the temporal domain, and demonstrate that even a small hardware system with only 88 memristors can already be used for tasks, such as handwritten digit recognition. The system is also used to experimentally solve a second-order nonlinear task, and can successfully predict the expected output without knowing the form of the original dynamic transfer function.
| S-EPMC5736649 | biostudies-literature