Dataset Information

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning.

ABSTRACT: The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism's survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain's computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain.

SUBMITTER: Zhang Z

PROVIDER: S-EPMC7673505 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning.

Zhang Zhewei Z Cheng Huzi H Yang Tianming T

PLoS computational biology 20201103 11

The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism's survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurr ...[more]

PMID: 33141824

Similar Datasets

Project description:According to a prominent view of sensorimotor processing in primates, selection and specification of possible actions are not sequential operations. Rather, a decision for an action emerges from competition between different movement plans, which are specified and selected in parallel. For action choices which are based on ambiguous sensory input, the frontoparietal sensorimotor areas are considered part of the common underlying neural substrate for selection and specification of action. These areas have been shown capable of encoding alternative spatial motor goals in parallel during movement planning, and show signatures of competitive value-based selection among these goals. Since the same network is also involved in learning sensorimotor associations, competitive action selection (decision making) should not only be driven by the sensory evidence and expected reward in favor of either action, but also by the subject's learning history of different sensorimotor associations. Previous computational models of competitive neural decision making used predefined associations between sensory input and corresponding motor output. Such hard-wiring does not allow modeling of how decisions are influenced by sensorimotor learning or by changing reward contingencies. We present a dynamic neural field model which learns arbitrary sensorimotor associations with a reward-driven Hebbian learning algorithm. We show that the model accurately simulates the dynamics of action selection with different reward contingencies, as observed in monkey cortical recordings, and that it correctly predicted the pattern of choice errors in a control experiment. With our adaptive model we demonstrate how network plasticity, which is required for association learning and adaptation to new reward contingencies, can influence choice behavior. The field model provides an integrated and dynamic account for the operations of sensorimotor integration, working memory and action selection required for decision making in ambiguous choice situations.

Project description:The ability to simultaneously record from large numbers of neurons in behaving animals has ushered in a new era for the study of the neural circuit mechanisms underlying cognitive functions. One promising approach to uncovering the dynamical and computational principles governing population responses is to analyze model recurrent neural networks (RNNs) that have been optimized to perform the same tasks as behaving animals. Because the optimization of network parameters specifies the desired output but not the manner in which to achieve this output, "trained" networks serve as a source of mechanistic hypotheses and a testing ground for data analyses that link neural computation to behavior. Complete access to the activity and connectivity of the circuit, and the ability to manipulate them arbitrarily, make trained networks a convenient proxy for biological circuits and a valuable platform for theoretical investigation. However, existing RNNs lack basic biological features such as the distinction between excitatory and inhibitory units (Dale's principle), which are essential if RNNs are to provide insights into the operation of biological circuits. Moreover, trained networks can achieve the same behavioral performance but differ substantially in their structure and dynamics, highlighting the need for a simple and flexible framework for the exploratory training of RNNs. Here, we describe a framework for gradient descent-based training of excitatory-inhibitory RNNs that can incorporate a variety of biological knowledge. We provide an implementation based on the machine learning library Theano, whose automatic differentiation capabilities facilitate modifications and extensions. We validate this framework by applying it to well-known experimental paradigms such as perceptual decision-making, context-dependent integration, multisensory integration, parametric working memory, and motor sequence generation. Our results demonstrate the wide range of neural activity patterns and behavior that can be modeled, and suggest a unified setting in which diverse cognitive computations and mechanisms can be studied.

Project description:The crucial role of sustainable development and resiliency strategies is undeniable in today’s competitive market space, especially after the Coronavirus outbreak. Hence, this research develops a multistage decision-making framework to investigate the supply chain network design problem considering the sustainability and resiliency dimensions. In this way, the scores of the potential suppliers based on the sustainability and resilience dimensions were calculated using the MADM methods, and then, these scores were applied as inputs in the proposed mathematical model (the second stage), which determined which supplier should be selected. The proposed model aims to minimize the total costs, maximize the suppliers' sustainability and resiliency, and maximize the distribution centers' resiliency. Then, the proposed model is solved by the preemptive fuzzy goal programming method. Overall, the main objectives and aims of the current work are to present a comprehensive decision-making model that can incorporate the sustainability and resilience dimensions into the supplier selection and supply chain configuration processes. In general, the main contributions and advantages of this work can be summarized as follows: (i) this research simultaneously investigates the sustainability and resiliency concepts in the dairy supply chain, (ii) the current work develops an efficient multistage decision-making model that can evaluate the suppliers based on the resilience and sustainability dimensions and configure the supply chain network, simultaneously. Based on the obtained results, the responsiveness and facilities reinforcement indicators are the most important indicators for the resilient aspect. On the other hand, reliability and quality are the most important indicators of sustainability aspect. Also, the results show that a large percentage of supply chain costs are related to purchasing and production costs. Besides, according to the outputs, the total cost of supply chain increases by enhancing the demand. Graphical abstract Supplementary Information The online version contains supplementary material available at 10.1007/s10098-023-02538-8.

Dataset Information

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning.

Publications

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets