Cooperative recurrent modular neural networks for constrained optimization: a survey of models and applications.
Ontology highlight
ABSTRACT:
SUBMITTER: Kamel MS
PROVIDER: S-EPMC2645489 | biostudies-other | 2009 Mar
REPOSITORIES: biostudies-other
Similar Datasets
Project description:A major tenet in theoretical neuroscience is that cognitive and behavioral processes are ultimately implemented in terms of the neural system dynamics. Accordingly, a major aim for the analysis of neurophysiological measurements should lie in the identification of the computational dynamics underlying task processing. Here we advance a state space model (SSM) based on generative piecewise-linear recurrent neural networks (PLRNN) to assess dynamics from neuroimaging data. In contrast to many other nonlinear time series models which have been proposed for reconstructing latent dynamics, our model is easily interpretable in neural terms, amenable to systematic dynamical systems analysis of the resulting set of equations, and can straightforwardly be transformed into an equivalent continuous-time dynamical system. The major contributions of this paper are the introduction of a new observation model suitable for functional magnetic resonance imaging (fMRI) coupled to the latent PLRNN, an efficient stepwise training procedure that forces the latent model to capture the 'true' underlying dynamics rather than just fitting (or predicting) the observations, and of an empirical measure based on the Kullback-Leibler divergence to evaluate from empirical time series how well this goal of approximating the underlying dynamics has been achieved. We validate and illustrate the power of our approach on simulated 'ground-truth' dynamical systems as well as on experimental fMRI time series, and demonstrate that the learnt dynamics harbors task-related nonlinear structure that a linear dynamical model fails to capture. Given that fMRI is one of the most common techniques for measuring brain activity non-invasively in human subjects, this approach may provide a novel step toward analyzing aberrant (nonlinear) dynamics for clinical assessment or neuroscientific research.
| S-EPMC6719895 | biostudies-literature
Project description:With the rise of artificial intelligence (AI) in drug discovery, de novo molecular generation provides new ways to explore chemical space. However, because de novo molecular generation methods rely on abundant known molecules, generated molecules may have a problem of novelty. Novelty is important in highly competitive areas of medicinal chemistry, such as the discovery of kinase inhibitors. In this study, de novo molecular generation based on recurrent neural networks was applied to discover a new chemical space of kinase inhibitors. During the application, the practicality was evaluated, and new inspiration was found. With the successful discovery of one potent Pim1 inhibitor and two lead compounds that inhibit CDK4, AI-based molecular generation shows potentials in drug discovery and development.
| S-EPMC7278228 | biostudies-literature
Project description:Sensitivity analysis applied to Artificial Neural Networks (ANNs) as well as to other types of empirical ecological models allows assessing the importance of environmental predictive variables in affecting species distribution or other target variables. However, approaches that only consider values of the environmental variables that are likely to be observed in real-world conditions, given the underlying ecological relationships with other variables, have not yet been proposed. Here, a constrained sensitivity analysis procedure is presented, which evaluates the importance of the environmental variables considering only their plausible changes, thereby exploring only ecological meaningful scenarios. To demonstrate the procedure, we applied it to an ANN model predicting fish species richness, as identifying relationships between environmental variables and fish species occurrence in river ecosystems is a recurring topic in freshwater ecology. Results showed that several environmental variables played a less relevant role in driving the model output when that sensitivity analysis allowed them to vary only within an ecologically meaningful range of values, i.e. avoiding values that the model would never handle in its practical applications. By comparing percent changes in MSE between constrained and unconstrained sensitivity analysis, the relative importance of environmental variables was found to be different, with habitat descriptors and urbanization factors that played a more relevant role according to the constrained procedure. The ecologically constrained procedure can be applied to any sensitivity analysis method for ANNs, but obviously it can also be applied to other types of empirical ecological models.
| S-EPMC6353184 | biostudies-literature
Project description:Chimera states, namely the coexistence of coherent and incoherent behavior, were previously analyzed in complex networks. However, they have not been extensively studied in modular networks. Here, we consider a neural network inspired by the connectome of the C. elegans soil worm, organized into six interconnected communities, where neurons obey chaotic bursting dynamics. Neurons are assumed to be connected with electrical synapses within their communities and with chemical synapses across them. As our numerical simulations reveal, the coaction of these two types of coupling can shape the dynamics in such a way that chimera-like states can happen. They consist of a fraction of synchronized neurons which belong to the larger communities, and a fraction of desynchronized neurons which are part of smaller communities. In addition to the Kuramoto order parameter ?, we also employ other measures of coherence, such as the chimera-like ? and metastability ? indices, which quantify the degree of synchronization among communities and along time, respectively. We perform the same analysis for networks that share common features with the C. elegans neural network. Similar results suggest that under certain assumptions, chimera-like states are prominent phenomena in modular networks, and might provide insight for the behavior of more complex modular networks.
| S-EPMC4726386 | biostudies-literature
Project description:BACKGROUND: Statistical approaches for protein design are relevant in the field of molecular evolutionary studies. In recent years, new, so-called structurally constrained (SC) models of protein-coding sequence evolution have been proposed, which use statistical potentials to assess sequence-structure compatibility. In a previous work, we defined a statistical framework for optimizing knowledge-based potentials especially suited to SC models. Our method used the maximum likelihood principle and provided what we call the joint potentials. However, the method required numerical estimations by the use of computationally heavy Markov Chain Monte Carlo sampling algorithms. RESULTS: Here, we develop an alternative optimization procedure, based on a leave-one-out argument coupled to fast gradient descent algorithms. We assess that the leave-one-out potential yields very similar results to the joint approach developed previously, both in terms of the resulting potential parameters, and by Bayes factor evaluation in a phylogenetic context. On the other hand, the leave-one-out approach results in a considerable computational benefit (up to a 1,000 fold decrease in computational time for the optimization procedure). CONCLUSION: Due to its computational speed, the optimization method we propose offers an attractive alternative for the design and empirical evaluation of alternative forms of potentials, using large data sets and high-dimensional parameterizations.
| S-EPMC2754480 | biostudies-literature
Project description:Coarse-grained modeling of conjugated polymers has become an increasingly popular route to investigate the physics of organic optoelectronic materials. While ultraviolet (UV)-vis spectroscopy remains one of the key experimental methods for the interrogation of these materials, a rigorous bridge between simulated coarse-grained structures and spectroscopy has not been established. Here, we address this challenge by developing a method that can predict spectra of conjugated polymers directly from coarse-grained representations while avoiding repetitive procedures such as ad hoc back-mapping from coarse-grained to atomistic representations followed by spectral computation using quantum chemistry. Our approach is based on a generative deep-learning model: the long-short-term memory recurrent neural network (LSTM-RNN). The latter is suggested by the apparent similarity between natural languages and the mathematical structure of perturbative expansions of, in our case, excited-state energies perturbed by conformational fluctuations. We also use this model to explore the level of sensitivity of spectra to the coarse-grained representation back-mapping protocol. Our approach presents a tool uniquely suited for improving postsimulation analysis protocols, as well as, potentially, for including spectral data as input in the refinement of coarse-grained potentials.
| S-EPMC7322026 | biostudies-literature
Project description:The calculation of temporally varying upstream process outcomes is a challenging task. Over the last years, several parametric, semi-parametric as well as non-parametric approaches were developed to provide reliable estimates for key process parameters. We present generic and product-specific recurrent neural network (RNN) models for the computation and study of growth and metabolite-related upstream process parameters as well as their temporal evolution. Our approach can be used for the control and study of single product-specific large-scale manufacturing runs as well as generic small-scale evaluations for combined processes and products at development stage. The computational results for the product titer as well as various major upstream outcomes in addition to relevant process parameters show a high degree of accuracy when compared to experimental data and, accordingly, a reasonable predictive capability of the RNN models. The calculated values for the root-mean squared errors of prediction are significantly smaller than the experimental standard deviation for the considered process run ensembles, which highlights the broad applicability of our approach. As a specific benefit for platform processes, the generic RNN model is also used to simulate process outcomes for different temperatures in good agreement with experimental results. The high level of accuracy and the straightforward usage of the approach without sophisticated parameterization and recalibration procedures highlight the benefits of the RNN models, which can be regarded as promising alternatives to existing parametric and semi-parametric methods.
| S-EPMC8193373 | biostudies-literature
Project description:In this paper we report the development of a cost-effective, modular, open source, and fully automated slide-scanning microscope, composed entirely of easily available off-the-shelf parts, and capable of bright field and fluorescence modes. The automated X-Y stage is composed of two low-cost micrometer stages coupled to stepper motors operated in open-loop mode. The microscope is composed of a low-cost CMOS sensor and low-cost board lenses placed in a 4f configuration. The system has approximately 1 micron resolution, limited by the f/# of available board lenses. The microscope is compact, measuring just 25×25×30 cm, and has an absolute positioning accuracy of ±1 ?m in the X and Y directions. A Z-stage enables autofocusing and imaging over large fields of view even on non-planar samples, and custom software enables automatic determination of sample boundaries and image mosaicking. We demonstrate the utility of our device through imaging of fluorescent- and transmission-dye stained blood and fecal smears containing human and animal parasites, as well as several prepared tissue samples. These results demonstrate image quality comparable to high-end commercial microscopes at a cost of less than US$400 for a bright-field system, with an extra US$100 needed for the fluorescence module.
| S-EPMC5854341 | biostudies-other
Project description:Quadratic programming (QP) is a common and important constrained optimization problem. Here, we derive a surprising duality between constrained optimization with inequality constraints, of which QP is a special case, and consumer resource models describing ecological dynamics. Combining this duality with a recent "cavity solution," we analyze high-dimensional, random QP where the optimization function and constraints are drawn randomly. Our theory shows remarkable agreement with numerics and points to a deep connection between optimization, dynamical systems, and ecology.
| S-EPMC6717431 | biostudies-literature
Project description:Impulsivity is a complex trait associated with a range of maladaptive behaviors, including many forms of psychopathology. Previous research has implicated multiple neural circuits and neurotransmitter systems in impulsive behavior, but the relationship between impulsivity and organization of whole-brain networks has not yet been explored. Using graph theory analyses, we characterized the relationship between impulsivity and the functional segregation ("modularity") of the whole-brain network architecture derived from resting-state functional magnetic resonance imaging (fMRI) data. These analyses revealed remarkable differences in network organization across the impulsivity spectrum. Specifically, in highly impulsive individuals, regulatory structures including medial and lateral regions of the prefrontal cortex were isolated from subcortical structures associated with appetitive drive, whereas these brain areas clustered together within the same module in less impulsive individuals. Further exploration of the modular organization of whole-brain networks revealed novel shifts in the functional connectivity between visual, sensorimotor, cortical, and subcortical structures across the impulsivity spectrum. The current findings highlight the utility of graph theory analyses of resting-state fMRI data in furthering our understanding of the neurobiological architecture of complex behaviors.
| S-EPMC3643719 | biostudies-other