A Unified Framework for Reservoir Computing and Extreme Learning Machines based on a Single Time-delayed Neuron.
Ontology highlight
ABSTRACT:
SUBMITTER: Ortin S
PROVIDER: S-EPMC4597340 | biostudies-other | 2015
REPOSITORIES: biostudies-other
Similar Datasets
Project description:Reservoir computing (RC) is a recently introduced bio-inspired computational framework capable of excellent performances in the temporal data processing, owing to its derivation from the recurrent neural network (RNN). It is well-known for the fast and effective training scheme, as well as the ease of the hardware implementation, but also the problematic sensitivity of its performance to the optimizable architecture parameters. In this article, a particular time-delayed RC with a single clamped-clamped silicon beam resonator that exhibits a classical Duffing nonlinearity is presented and its optimization problem is studied. Specifically, we numerically analyze the nonlinear response of the resonator and find a quasi-linear bifurcation point shift of the driving voltage with the driving frequency sweeping, which is called Bifurcation Point Frequency Modulation (BPFM). Furthermore, we first proposed that this method can be used to find the optimal driving frequency of RC with a Duffing mechanical resonator for a given task, and then put forward a comprehensive optimization process. The high performance of RC presented on four typical tasks proves the feasibility of this optimization method. Finally, we envision the potential application of the method based on the BPFM in our future work to implement the RC with other mechanical oscillators.
| S-EPMC7806606 | biostudies-literature
Project description:Reservoir computing, a new method of machine learning, has recently been used to predict the state evolution of various chaotic dynamic systems. It has significant advantages in terms of training cost and adjusted parameters; however, the prediction length is limited. For classic reservoir computing, the prediction length can only reach five to six Lyapunov times. Here, we modified the method of reservoir computing by adding feedback, continuous or discrete, to "calibrate" the input of the reservoir and then reconstruct the entire dynamic systems. The reconstruction length appreciably increased and the training length obviously decreased. The reconstructing of dynamical systems is studied in detail under this method. The reconstruction can be significantly improved both in length and accuracy. Additionally, we summarized the effect of different kinds of input feedback. The more it interacts with others in dynamical equations, the better the reconstructions. Nonlinear terms can reveal more information than linear terms once the interaction terms are equal. This method has proven effective via several classical chaotic systems. It can be superior to traditional reservoir computing in reconstruction, provides new hints in computing promotion, and may be used in some real applications.
| S-EPMC9522934 | biostudies-literature
Project description:We investigated a time-delayed optoelectronic oscillator (OEO) that displays a wide range of complex dynamic behavior under small time delay. The phase-space trajectory distributions in different dynamic regimes were compared which brings a new perspective on the underlying mechanism of the transition process. It was found that bifurcation is always possible no matter how small the time delay is even if the universal adiabatic approximation model is invalid. Hereby we proposed a versatile simple oscillator which has a potential capacity as memory carrier and high-dimensional state spatial mapping ability that brings 1000 times computing-efficiency improvements of reservoir computing over the large time delay one. Furthermore, we demonstrated a new approach for a tunable optoelectronic pulse generator (repetition rate at 0.2 MHz and 0.25 GHz) which depends critically on time-delayed input electrical pulse. The proposed oscillator is also a promising system for the applications of fast chaos-based communication.
| S-EPMC11319353 | biostudies-literature
Project description:Recent years have witnessed a rising demand for edge computing, and there is a need for methods to decrease the computational cost while maintaining a high learning performance when processing information at arbitrary edges. Reservoir computing using physical dynamics has attracted significant attention. However, currently, the timescale of the input signals that can be processed by physical reservoirs is limited by the transient characteristics inherent to the selected physical system. This study used an Sn-doped In2 O3 /Nb-doped SrTiO3 junction to fabricate a memristor that could respond to both electrical and optical stimuli. The results show that the timescale of the transient current response of the device could be controlled over several orders of magnitude simply by applying a small voltage. The computational performance of the device as a physical reservoir is evaluated in an image classification task, demonstrating that the learning accuracy could be optimized by tuning the device to exhibit appropriate transient characteristics according to the timescale of the input signals. These results are expected to provide deeper insights into the photoconductive properties of strontium titanate, as well as support the physical implementation of computing systems.
| S-EPMC10797460 | biostudies-literature
Project description:Respiration induced motion is a well-recognized challenge in many clinical practices including upper body imaging, lung tumor motion tracking and radiation therapy. In this work, we present a recurrent neural network algorithm that was implemented in a photonic delay-line reservoir computer (RC) for real-time respiratory motion prediction. The respiratory motion signals are quasi-periodic waveforms subject to a variety of non-linear distortions. In this work, we demonstrated for the first time that RC can be effective in predicting short to medium range of respiratory motions within practical timescales. A double-sliding window technology is explored to enable the real-time establishment of an individually trained model for each patient and the real-time processing of live-streamed respiratory motion data. A breathing dataset from a total of 76 patients with breathing speeds ranging from 3 to 20 breaths per minute (BPM) is studied. Motion prediction of look-ahead times of 66.6, 166.6, and 333 ms are investigated. With a 333 ms look-ahead time, the real-time RC model achieves an average normalized mean square error (NMSE) of 0.025, an average mean absolute error (MAE) of 0.34 mm, an average root mean square error (RMSE) of 0.45 mm, an average therapeutic beam efficiency (TBE) of 94.14% for an absolute error (AE) < 1 mm, and 99.89% for AE < 3 mm. This study demonstrates that real-time RC is an efficient computing framework for high precision respiratory motion prediction.
| S-EPMC10082218 | biostudies-literature
Project description:Photonic reservoir computing is an emergent technology toward beyond-Neumann computing. Although photonic reservoir computing provides superior performance in environments whose characteristics are coincident with the training datasets for the reservoir, the performance is significantly degraded if these characteristics deviate from the original knowledge used in the training phase. Here, we propose a scheme of adaptive model selection in photonic reservoir computing using reinforcement learning. In this scheme, a temporal waveform is generated by different dynamic source models that change over time. The system autonomously identifies the best source model for the task of time series prediction using photonic reservoir computing and reinforcement learning. We prepare two types of output weights for the source models, and the system adaptively selected the correct model using reinforcement learning, where the prediction errors are associated with rewards. We succeed in adaptive model selection when the source signal is temporally mixed, having originally been generated by two different dynamic system models, as well as when the signal is a mixture from the same model but with different parameter values. This study paves the way for autonomous behavior in photonic artificial intelligence and could lead to new applications in load forecasting and multi-objective control, where frequent environment changes are expected.
| S-EPMC7308406 | biostudies-literature
Project description:Reservoir computing is a recently introduced, highly efficient bio-inspired approach for processing time dependent data. The basic scheme of reservoir computing consists of a non linear recurrent dynamical system coupled to a single input layer and a single output layer. Within these constraints many implementations are possible. Here we report an optoelectronic implementation of reservoir computing based on a recently proposed architecture consisting of a single non linear node and a delay line. Our implementation is sufficiently fast for real time information processing. We illustrate its performance on tasks of practical importance such as nonlinear channel equalization and speech recognition, and obtain results comparable to state of the art digital implementations.
| S-EPMC3286854 | biostudies-literature
Project description:This paper presents a novel color face recognition algorithm by means of fusing color and local information. The proposed algorithm fuses the multiple features derived from different color spaces. Multiorientation and multiscale information relating to the color face features are extracted by applying Steerable Pyramid Transform (SPT) to the local face regions. In this paper, the new three hybrid color spaces, YSCr, ZnSCr, and BnSCr, are firstly constructed using the Cb and Cr component images of the YCbCr color space, the S color component of the HSV color spaces, and the Zn and Bn color components of the normalized XYZ color space. Secondly, the color component face images are partitioned into the local patches. Thirdly, SPT is applied to local face regions and some statistical features are extracted. Fourthly, all features are fused according to decision fusion frame and the combinations of Extreme Learning Machines classifiers are applied to achieve color face recognition with fast and high correctness. The experiments show that the proposed Local Color Steerable Pyramid Transform (LCSPT) face recognition algorithm improves seriously face recognition performance by using the new color spaces compared to the conventional and some hybrid ones. Furthermore, it achieves faster recognition compared with state-of-the-art studies.
| S-EPMC3914600 | biostudies-literature
Project description:Reservoir computing is a best-in-class machine learning algorithm for processing information generated by dynamical systems using observed time-series data. Importantly, it requires very small training data sets, uses linear optimization, and thus requires minimal computing resources. However, the algorithm uses randomly sampled matrices to define the underlying recurrent neural network and has a multitude of metaparameters that must be optimized. Recent results demonstrate the equivalence of reservoir computing to nonlinear vector autoregression, which requires no random matrices, fewer metaparameters, and provides interpretable results. Here, we demonstrate that nonlinear vector autoregression excels at reservoir computing benchmark tasks and requires even shorter training data sets and training time, heralding the next generation of reservoir computing.
| S-EPMC8455577 | biostudies-literature
Project description:Mechanical computing offers a new modality to formulate computational autonomy in intelligent matter or machines without any external powering or active elements. Transition (or solitary) waves, induced by nonreciprocity in mechanical metamaterials comprising a chain of bistable elements, have proven to be a key ingredient for dissipation-free transmission and computation of mechanical information. However, advanced processing of mechanical information in existing designs is hindered by its dissipation when interacting with networked logic gates. Here, we present a metamaterial design strategy that allows non-dispersive mechanical solitary waves to compute multi-level cascaded logic functions, termed 'integrated mechanical computing', by propagating through a network of structurally heterogeneous computing units. From a perspective of characteristic potential energy, we establish an analytical framework that helps in understanding the solitary wave-based mechanical computation, and governs the mechanical design of key determinants for realizing cascaded logic computation, such as soliton profile and logic elements. The developed integrated mechanical computing systems are shown to receive, transmit and compute mechanical information to actuate intelligent soft machine prototypes in a seamless and integrated manner. These findings would pave the way for future intelligent robots and machines that perform computational operations between various non-electrical environmental inputs.
| S-EPMC10995184 | biostudies-literature