Dataset Information

A Closed-Loop Toolchain for Neural Network Simulations of Learning Autonomous Agents.

ABSTRACT: Neural network simulation is an important tool for generating and evaluating hypotheses on the structure, dynamics, and function of neural circuits. For scientific questions addressing organisms operating autonomously in their environments, in particular where learning is involved, it is crucial to be able to operate such simulations in a closed-loop fashion. In such a set-up, the neural agent continuously receives sensory stimuli from the environment and provides motor signals that manipulate the environment or move the agent within it. So far, most studies requiring such functionality have been conducted with custom simulation scripts and manually implemented tasks. This makes it difficult for other researchers to reproduce and build upon previous work and nearly impossible to compare the performance of different learning architectures. In this work, we present a novel approach to solve this problem, connecting benchmark tools from the field of machine learning and state-of-the-art neural network simulators from computational neuroscience. The resulting toolchain enables researchers in both fields to make use of well-tested high-performance simulation software supporting biologically plausible neuron, synapse and network models and allows them to evaluate and compare their approach on the basis of standardized environments with various levels of complexity. We demonstrate the functionality of the toolchain by implementing a neuronal actor-critic architecture for reinforcement learning in the NEST simulator and successfully training it on two different environments from the OpenAI Gym. We compare its performance to a previously suggested neural network model of reinforcement learning in the basal ganglia and a generic Q-learning algorithm.

SUBMITTER: Jordan J

PROVIDER: S-EPMC6687756 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Closed-Loop Toolchain for Neural Network Simulations of Learning Autonomous Agents.

Jordan Jakob J Weidel Philipp P Morrison Abigail A

Frontiers in computational neuroscience 20190802

Neural network simulation is an important tool for generating and evaluating hypotheses on the structure, dynamics, and function of neural circuits. For scientific questions addressing organisms operating autonomously in their environments, in particular where learning is involved, it is crucial to be able to operate such simulations in a closed-loop fashion. In such a set-up, the neural agent continuously receives sensory stimuli from the environment and provides motor signals that manipulate t ...[more]

PMID: 31427939

Similar Datasets

Project description:Neural stimulation can reduce the frequency of seizures in persons with epilepsy, but rates of seizure-free outcome are low. Vagus nerve stimulation prevents seizures by continuously activating noradrenergic projections from the brainstem to the cortex. Cortical norepinephrine then increases GABAergic transmission and increases seizure threshold. Another approach, responsive nervous stimulation, prevents seizures by reactively shocking the seizure onset zone in precise synchrony with seizure onset. The electrical shocks abort seizures before they can spread and manifest clinically. The goal of this study was to determine whether a hybrid platform in which brainstem activation triggered in response to impending seizure activity could prevent seizures. We chose the zebrafish as a model organism for this study because of its ability to recapitulate human disease, in conjunction with its innate capacity for tightly controlled high-throughput experimentation. We first set out to determine whether electrical stimulation of the zebrafish hindbrain could have an anticonvulsant effect. We found that pulse train electrical stimulation of the hindbrain significantly increased the latency to onset of pentylenetetrazole-induced seizures, and that this apparent anticonvulsant effect was blocked by noradrenergic antagonists, as is also the case with rodents and humans. We also found that the anticonvulsant effect of hindbrain stimulation could be potentiated by reactive triggering of single pulse electrical stimulations in response to impending seizure activity. Finally, we found that the rate of stimulation triggering was directly proportional to pentylenetetrazole concentration and that the stimulation rate was reduced by the anticonvulsant valproic acid and by larger stimulation currents. Taken as a whole, these results show that that the anticonvulsant effect of brainstem activation can be efficiently utilized by reactive triggering, which suggests that alternative stimulation paradigms for vagus nerve stimulation might be useful. Moreover, our results show that the zebrafish epilepsy model can be used to advance our understanding of neural stimulation in the treatment of epilepsy.

Dataset Information

A Closed-Loop Toolchain for Neural Network Simulations of Learning Autonomous Agents.

Publications

A Closed-Loop Toolchain for Neural Network Simulations of Learning Autonomous Agents.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets