Dataset Information

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops.

ABSTRACT: Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

SUBMITTER: Stelzer F

PROVIDER: S-EPMC8397757 | biostudies-literature | 2021 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops.

Stelzer Florian F Röhm André A Vicente Raul R Fischer Ingo I Yanchuk Serhiy S

Nature communications 20210827 1

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By ...[more]

PMID: 34453053

Similar Datasets

Project description:PurposeThis study proposes a cascaded network model for generating high-resolution doses (i.e., a 1 mm grid) from low-resolution doses (i.e., ≥3 mm grids) with reduced computation time.MethodsUsing the anisotropic analytical algorithm with three grid sizes (1, 3, and 5 mm) and the Acuros XB algorithm with two grid sizes (1 and 3 mm), dose distributions were calculated for volumetric modulated arc therapy plans for 73 prostate cancer patients. Our cascaded network model consisted of a hierarchically densely connected U-net (HD U-net) and a residual dense network (RDN), which were trained separately following a two-dimensional slice-by-slice procedure. The first network (HD U-net) predicted the downsampled high-resolution dose (generated through bicubic downsampling of the baseline high-resolution dose) using the low-resolution dose; subsequently, the second network (RDN) predicted the high-resolution dose from the output of the first network. Further, the predicted high-resolution dose was converted to its absolute value. We quantified the network performance using the spatial/dosimetric parameters (dice similarity coefficient, mean dose, maximum dose, minimum dose, homogeneity index, conformity index, and V95%, V70%, V50%, and V30%) for the low-resolution and predicted high-resolution doses relative to the baseline high-resolution dose. Gamma analysis (between the baseline dose and the low-resolution dose/predicted high-resolution dose) was performed with a 2%/2 mm criterion and 10% threshold.ResultsThe average computation time to predict a high-resolution axial dose plane was <0.02 s. The dice similarity coefficient values for the predicted doses were closer to 1 when compared to those for the low-resolution doses. Most of the dosimetric parameters for the predicted doses agreed more closely with those for the baseline than for the low-resolution doses. In most of the parameters, no significant differences (p-value of >0.05) between the baseline and predicted doses were observed. The gamma passing rates for the predicted high-resolution does were higher than those for the low-resolution doses.ConclusionThe proposed model accurately predicted high-resolution doses for the same dose calculation algorithm. Our model uses only dose data as the input without additional data, which provides advantages of convenience to user over other dose super-resolution methods.

Project description:BackgroundFeedback loops in gene regulatory networks play pivotal roles in governing functional dynamics of cells. Systems approaches demonstrated characteristic dynamical features, including multistability and oscillation, of positive and negative feedback loops. Recent experiments and theories have implicated highly interconnected feedback loops (high-feedback loops) in additional nonintuitive functions, such as controlling cell differentiation rate and multistep cell lineage progression. However, it remains challenging to identify and visualize high-feedback loops in complex gene regulatory networks due to the myriad of ways in which the loops can be combined. Furthermore, it is unclear whether the high-feedback loop structures with these potential functions are widespread in biological systems. Finally, it remains challenging to understand diverse dynamical features, such as high-order multistability and oscillation, generated by individual networks containing high-feedback loops. To address these problems, we developed HiLoop, a toolkit that enables discovery, visualization, and analysis of several types of high-feedback loops in large biological networks.ResultsHiLoop not only extracts high-feedback structures and visualize them in intuitive ways, but also quantifies the enrichment of overrepresented structures. Through random parameterization of mathematical models derived from target networks, HiLoop presents characteristic features of the underlying systems, including complex multistability and oscillations, in a unifying framework. Using HiLoop, we were able to analyze realistic gene regulatory networks containing dozens to hundreds of genes, and to identify many small high-feedback systems. We found more than a 100 human transcription factors involved in high-feedback loops that were not studied previously. In addition, HiLoop enabled the discovery of an enrichment of high feedback in pathways related to epithelial-mesenchymal transition.ConclusionsHiLoop makes the study of complex networks accessible without significant computational demands. It can serve as a hypothesis generator through identification and modeling of high-feedback subnetworks, or as a quantification method for motif enrichment analysis. As an example of discovery, we found that multistep cell lineage progression may be driven by either specific instances of high-feedback loops with sparse appearances, or generally enriched topologies in gene regulatory networks. We expect HiLoop's usefulness to increase as experimental data of regulatory networks accumulate. Code is freely available for use or extension at https://github.com/BenNordick/HiLoop .

Dataset Information

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops.

Publications

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets