Browse
Submit Data
Databases
API
Help

Dataset Information

28 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Nonequilibrium landscape theory of neural networks.

ABSTRACT: The brain map project aims to map out the neuron connections of the human brain. Even with all of the wirings mapped out, the global and physical understandings of the function and behavior are still challenging. Hopfield quantified the learning and memory process of symmetrically connected neural networks globally through equilibrium energy. The energy basins of attractions represent memories, and the memory retrieval dynamics is determined by the energy gradient. However, the realistic neural networks are asymmetrically connected, and oscillations cannot emerge from symmetric neural networks. Here, we developed a nonequilibrium landscape-flux theory for realistic asymmetrically connected neural networks. We uncovered the underlying potential landscape and the associated Lyapunov function for quantifying the global stability and function. We found the dynamics and oscillations in human brains responsible for cognitive processes and physiological rhythm regulations are determined not only by the landscape gradient but also by the flux. We found that the flux is closely related to the degrees of the asymmetric connections in neural networks and is the origin of the neural oscillations. The neural oscillation landscape shows a closed-ring attractor topology. The landscape gradient attracts the network down to the ring. The flux is responsible for coherent oscillations on the ring. We suggest the flux may provide the driving force for associations among memories. We applied our theory to rapid-eye movement sleep cycle. We identified the key regulation factors for function through global sensitivity analysis of landscape topography against wirings, which are in good agreements with experiments.

SUBMITTER: Yan H

PROVIDER: S-EPMC3831465 | biostudies-literature | 2013 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Potential landscape and flux framework of nonequilibrium networks: robustness, dissipation, and coherence of biochemical oscillations.

Project description:We established a theoretical framework for studying nonequilibrium networks with two distinct natures essential for characterizing the global probabilistic dynamics: the underlying potential landscape and the corresponding curl flux. We applied the idea to a biochemical oscillation network and found that the underlying potential landscape for the oscillation limit cycle has a distinct closed ring valley (Mexican hat-like) shape when the fluctuations are small. This global landscape structure leads to attractions of the system to the ring valley. On the ring, we found that the nonequilibrium flux is the driving force for oscillations. Therefore, both structured landscape and flux are needed to guarantee a robust oscillating network. The barrier height separating the oscillation ring and other areas derived from the landscape topography is shown to be correlated with the escaping time from the limit cycle attractor and provides a quantitative measure of the robustness for the network. The landscape becomes shallower and the closed ring valley shape structure becomes weaker (lower barrier height) with larger fluctuations. We observe that the period and the amplitude of the oscillations are more dispersed and oscillations become less coherent when the fluctuations increase. We also found that the entropy production of the whole network, characterizing the dissipation costs from the combined effects of both landscapes and fluxes, decreases when the fluctuations decrease. Therefore, less dissipation leads to more robust networks. Our approach is quite general and applicable to other networks, dynamical systems, and biological evolution. It can help in designing robust networks.

| S-EPMC2527901 | biostudies-literature

Nonequilibrium phase diagrams for actomyosin networks.

Project description:Living cells dynamically modulate the local morphologies of their actin networks to perform biological functions, including force transduction, intracellular transport, and cell division. A major challenge is to understand how diverse structures of the actin cytoskeleton are assembled from a limited set of molecular building blocks. Here we study the spontaneous self-assembly of a minimal model of cytoskeletal materials, consisting of semiflexible actin filaments, crosslinkers, and molecular motors. Using coarse-grained simulations, we demonstrate that by changing concentrations and kinetics of crosslinkers and motors, as well as filament lengths, we can generate three distinct structural phases of actomyosin assemblies: bundled, polarity-sorted, and contracted. We introduce new metrics to distinguish these structural phases and demonstrate their functional roles. We find that the binding kinetics of motors and crosslinkers can be tuned to optimize contractile force generation, motor transport, and mechanical response. By quantitatively characterizing the relationships between the modes of cytoskeletal self-assembly, the resulting structures, and their functional consequences, our work suggests new principles for the design of active materials.

| S-EPMC6192427 | biostudies-literature

Toward a theory of coactivation patterns in excitable neural networks.

Project description:The relationship between the structural connectivity (SC) and functional connectivity (FC) of neural systems is of central importance in brain network science. It is an open question, however, how the SC-FC relationship depends on specific topological features of brain networks or the models used for describing neural dynamics. Using a basic but general model of discrete excitable units that follow a susceptible-excited-refractory activity cycle (SER model), we here analyze how the network activity patterns underlying functional connectivity are shaped by the characteristic topological features of the network. We develop an analytical framework for describing the contribution of essential topological elements, such as common inputs and pacemakers, to the coactivation of nodes, and demonstrate the validity of the approach by comparison of the analytical predictions with numerical simulations of various exemplar networks. The present analytic framework may serve as an initial step for the mechanistic understanding of the contributions of brain network topology to brain dynamics.

| S-EPMC5908206 | biostudies-literature

A mathematical theory of semantic development in deep neural networks.

Project description:An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: What are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences? We address this question by mathematically analyzing the nonlinear dynamics of learning in deep linear networks. We find exact solutions to this learning dynamics that yield a conceptual explanation for the prevalence of many disparate phenomena in semantic cognition, including the hierarchical differentiation of concepts through rapid developmental transitions, the ubiquity of semantic illusions between such transitions, the emergence of item typicality and category coherence as factors controlling the speed of semantic processing, changing patterns of inductive projection over development, and the conservation of semantic similarity in neural representations across species. Thus, surprisingly, our simple neural model qualitatively recapitulates many diverse regularities underlying semantic development, while providing analytic insight into how the statistical structure of an environment can interact with nonlinear deep-learning dynamics to give rise to these regularities.

| S-EPMC6561300 | biostudies-literature

Density Functional Theory for Steady-State Nonequilibrium Molecular Junctions.

Project description:We present a density functional theory (DFT) for steady-state nonequilibrium quantum systems such as molecular junctions under a finite bias. Based on the steady-state nonequilibrium statistics that maps nonequilibrium to an effective equilibrium, we show that ground-state DFT (GS-DFT) is not applicable in this case and two densities, the total electron density and the density of current-carrying electrons, are needed to uniquely determine the properties of the corresponding nonequilibrium system. A self-consistent mean-field approach based on two densities is then derived. The theory is implemented into SIESTA computational package and applied to study nonequilibrium electronic/transport properties of a realistic carbon-nanotube (CNT)/Benzene junction. Results obtained from our steady-state DFT (SS-DFT) are compared with those of conventional GS-DFT based transport calculations. We show that SS-DFT yields energetically more stable nonequilibrium steady state, predicts significantly lower electric current, and is able to produce correct electronic structures in local equilibrium under a limiting case.

| S-EPMC4608000 | biostudies-literature

Quantifying the Landscape of Decision Making From Spiking Neural Networks.

Project description:The decision making function is governed by the complex coupled neural circuit in the brain. The underlying energy landscape provides a global picture for the dynamics of the neural decision making system and has been described extensively in the literature, but often as illustrations. In this work, we explicitly quantified the landscape for perceptual decision making based on biophysically-realistic cortical network with spiking neurons to mimic a two-alternative visual motion discrimination task. Under certain parameter regions, the underlying landscape displays bistable or tristable attractor states, which quantify the transition dynamics between different decision states. We identified two intermediate states: the spontaneous state which increases the plasticity and robustness of changes of minds and the "double-up" state which facilitates the state transitions. The irreversibility of the bistable and tristable switches due to the probabilistic curl flux demonstrates the inherent non-equilibrium characteristics of the neural decision system. The results of global stability of decision-making quantified by barrier height inferred from landscape topography and mean first passage time are in line with experimental observations. These results advance our understanding of the stochastic and dynamical transition mechanism of decision-making function, and the landscape and kinetic path approach can be applied to other cognitive function related problems (such as working memory) in brain networks.

| S-EPMC8581041 | biostudies-literature

A mean field view of the landscape of two-layer neural networks.

Project description:Multilayer neural networks are among the most powerful models in machine learning, yet the fundamental reasons for this success defy mathematical understanding. Learning a neural network requires optimizing a nonconvex high-dimensional objective (risk function), a problem that is usually attacked using stochastic gradient descent (SGD). Does SGD converge to a global optimum of the risk or only to a local optimum? In the former case, does this happen because local minima are absent or because SGD somehow avoids them? In the latter, why do local minima reached by SGD have good generalization properties? In this paper, we consider a simple case, namely two-layer neural networks, and prove that-in a suitable scaling limit-SGD dynamics is captured by a certain nonlinear partial differential equation (PDE) that we call distributional dynamics (DD). We then consider several specific examples and show how DD can be used to prove convergence of SGD to networks with nearly ideal generalization error. This description allows for "averaging out" some of the complexities of the landscape of neural networks and can be used to prove a general convergence result for noisy SGD.

| S-EPMC6099898 | biostudies-other

Mechanical theory of nonequilibrium coexistence and motility-induced phase separation.

Project description:Nonequilibrium phase transitions are routinely observed in both natural and synthetic systems. The ubiquity of these transitions highlights the conspicuous absence of a general theory of phase coexistence that is broadly applicable to both nonequilibrium and equilibrium systems. Here, we present a general mechanical theory for phase separation rooted in ideas explored nearly a half-century ago in the study of inhomogeneous fluids. The core idea is that the mechanical forces within the interface separating two coexisting phases uniquely determine coexistence criteria, regardless of whether a system is in equilibrium or not. We demonstrate the power and utility of this theory by applying it to active Brownian particles, predicting a quantitative phase diagram for motility-induced phase separation in both two and three dimensions. This formulation additionally allows for the prediction of novel interfacial phenomena, such as an increasing interface width while moving deeper into the two-phase region, a uniquely nonequilibrium effect confirmed by computer simulations. The self-consistent determination of bulk phase behavior and interfacial phenomena offered by this mechanical perspective provide a concrete path forward toward a general theory for nonequilibrium phase transitions.

| S-EPMC10160997 | biostudies-literature

Shaping the learning landscape in neural networks around wide flat minima.

Project description:Learning in deep neural networks takes place by minimizing a nonconvex high-dimensional loss function, typically by a stochastic gradient descent (SGD) strategy. The learning process is observed to be able to find good minimizers without getting stuck in local critical points and such minimizers are often satisfactory at avoiding overfitting. How these 2 features can be kept under control in nonlinear devices composed of millions of tunable connections is a profound and far-reaching open question. In this paper we study basic nonconvex 1- and 2-layer neural network models that learn random patterns and derive a number of basic geometrical and algorithmic features which suggest some answers. We first show that the error loss function presents few extremely wide flat minima (WFM) which coexist with narrower minima and critical points. We then show that the minimizers of the cross-entropy loss function overlap with the WFM of the error loss. We also show examples of learning devices for which WFM do not exist. From the algorithmic perspective we derive entropy-driven greedy and message-passing algorithms that focus their search on wide flat regions of minimizers. In the case of SGD and cross-entropy loss, we show that a slow reduction of the norm of the weights along the learning process also leads to WFM. We corroborate the results by a numerical study of the correlations between the volumes of the minimizers, their Hessian, and their generalization performance on real data.

| S-EPMC6955380 | biostudies-literature

Learning the local landscape of protein structures with convolutional neural networks.

Project description:One fundamental problem of protein biochemistry is to predict protein structure from amino acid sequence. The inverse problem, predicting either entire sequences or individual mutations that are consistent with a given protein structure, has received much less attention even though it has important applications in both protein engineering and evolutionary biology. Here, we ask whether 3D convolutional neural networks (3D CNNs) can learn the local fitness landscape of protein structure to reliably predict either the wild-type amino acid or the consensus in a multiple sequence alignment from the local structural context surrounding site of interest. We find that the network can predict wild type with good accuracy, and that network confidence is a reliable measure of whether a given prediction is likely going to be correct or not. Predictions of consensus are less accurate and are primarily driven by whether or not the consensus matches the wild type. Our work suggests that high-confidence mis-predictions of the wild type may identify sites that are primed for mutation and likely targets for protein engineering.

| S-EPMC8603988 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data