Dataset Information

Engineering recurrent neural networks from task-relevant manifolds and dynamics.

ABSTRACT: Many cognitive processes involve transformations of distributed representations in neural populations, creating a need for population-level models. Recurrent neural network models fulfill this need, but there are many open questions about how their connectivity gives rise to dynamics that solve a task. Here, we present a method for finding the connectivity of networks for which the dynamics are specified to solve a task in an interpretable way. We apply our method to a working memory task by synthesizing a network that implements a drift-diffusion process over a ring-shaped manifold. We also use our method to demonstrate how inputs can be used to control network dynamics for cognitive flexibility and explore the relationship between representation geometry and network capacity. Our work fits within the broader context of understanding neural computations as dynamics over relatively low-dimensional manifolds formed by correlated patterns of neurons.

SUBMITTER: Pollock E

PROVIDER: S-EPMC7446915 | biostudies-literature | 2020 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Engineering recurrent neural networks from task-relevant manifolds and dynamics.

Pollock Eli E Jazayeri Mehrdad M

PLoS computational biology 20200812 8

Many cognitive processes involve transformations of distributed representations in neural populations, creating a need for population-level models. Recurrent neural network models fulfill this need, but there are many open questions about how their connectivity gives rise to dynamics that solve a task. Here, we present a method for finding the connectivity of networks for which the dynamics are specified to solve a task in an interpretable way. We apply our method to a working memory task by syn ...[more]

PMID: 32785228

Similar Datasets

Project description:Electroencephalogram (EEG) microstates that represent quasi-stable, global neuronal activity are considered as the building blocks of brain dynamics. Therefore, the analysis of microstate sequences is a promising approach to understand fast brain dynamics that underlie various mental processes. Recent studies suggest that EEG microstate sequences are non-Markovian and nonstationary, highlighting the importance of the sequential flow of information between different brain states. These findings inspired us to model these sequences using Recurrent Neural Networks (RNNs) consisting of long-short-term-memory (LSTM) units to capture the complex temporal dependencies. Using an LSTM-based auto encoder framework and different encoding schemes, we modeled the microstate sequences at multiple time scales (200-2,000 ms) aiming to capture stably recurring microstate patterns within and across subjects. We show that RNNs can learn underlying microstate patterns with high accuracy and that the microstate trajectories are subject invariant at shorter time scales (≤400 ms) and reproducible across sessions. Significant drop in the reconstruction accuracy was observed for longer sequence lengths of 2,000 ms. These findings indirectly corroborate earlier studies which indicated that EEG microstate sequences exhibit long-range dependencies with finite memory content. Furthermore, we find that the latent representations learned by the RNNs are sensitive to external stimulation such as stress while the conventional univariate microstate measures (e.g., occurrence, mean duration, etc.) fail to capture such changes in brain dynamics. While RNNs cannot be configured to identify the specific discriminating patterns, they have the potential for learning the underlying temporal dynamics and are sensitive to sequence aberrations characterized by changes in metal processes. Empowered with the macroscopic understanding of the temporal dynamics that extends beyond short-term interactions, RNNs offer a reliable alternative for exploring system level brain dynamics using EEG microstate sequences.

Project description:BackgroundRepetitive elements contribute a large part of eukaryotic genomes. For example, about 40 to 50% of human, mouse and rat genomes are repetitive. So identifying and classifying repeats is an important step in genome annotation. This annotation step is traditionally performed using alignment based methods, either in a de novo approach or by aligning the genome sequence to a species specific set of repetitive sequences. Recently, Li (Bioinformatics 35:4408-4410, 2019) developed a novel software tool dna-brnn to annotate repetitive sequences using a recurrent neural network trained on sample annotations of repetitive elements.ResultsWe have developed the methods of dna-brnn further and engineered a new software tool DeepGRP. This combines the basic concepts of Li (Bioinformatics 35:4408-4410, 2019) with current techniques developed for neural machine translation, the attention mechanism, for the task of nucleotide-level annotation of repetitive elements. An evaluation on the human genome shows a 20% improvement of the Matthews correlation coefficient for the predictions delivered by DeepGRP, when compared to dna-brnn. DeepGRP predicts two additional classes of repeats (compared to dna-brnn) and is able to transfer repeat annotations, using RepeatMasker-based training data to a different species (mouse). Additionally, we could show that DeepGRP predicts repeats annotated in the Dfam database, but not annotated by RepeatMasker. DeepGRP is highly scalable due to its implementation in the TensorFlow framework. For example, the GPU-accelerated version of DeepGRP is approx. 1.8 times faster than dna-brnn, approx. 8.6 times faster than RepeatMasker and over 100 times faster than HMMER searching for models of the Dfam database.ConclusionsBy incorporating methods from neural machine translation, DeepGRP achieves a consistent improvement of the quality of the predictions compared to dna-brnn. Improved running times are obtained by employing TensorFlow as implementation framework and the use of GPUs. By incorporating two additional classes of repeats, DeepGRP provides more complete annotations, which were evaluated against three state-of-the-art tools for repeat annotation.

Dataset Information

Engineering recurrent neural networks from task-relevant manifolds and dynamics.

Publications

Engineering recurrent neural networks from task-relevant manifolds and dynamics.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets