A maximum entropy framework for nonexponential distributions.
Ontology highlight
ABSTRACT:
SUBMITTER: Peterson J
PROVIDER: S-EPMC3870711 | biostudies-literature | 2013 Dec
REPOSITORIES: biostudies-literature
Publications
A maximum entropy framework for nonexponential distributions.
Peterson Jack J Dixit Purushottam D PD Dill Ken A KA
Proceedings of the National Academy of Sciences of the United States of America 20131202 51
Probability distributions having power-law tails are observed in a broad range of social, economic, and biological systems. We describe here a potentially useful common framework. We derive distribution functions for situations in which a "joiner particle" k pays some form of price to enter a community of size , where costs are subject to economies of scale. Maximizing the Boltzmann-Gibbs-Shannon entropy subject to this energy-like constraint predicts a distribution having a power-law tail; it r ...[more]
Similar Datasets
Project description:Quantiles are available in various problems for developing probability distributions. In some problems quantiles are elicited from experts and used for fitting parametric models, which induce non-elicited information. In some other problems comparisons are made with a quantile of an assumed model which is noncommittal to the quantile information. The maximum entropy (ME) principle provides models that avoid these issues. However, the information theory literature has been mainly concerned about models based on moment information. This paper explores the ME models that are the minimum elaborations of the uniform and moment-based ME models by quantiles. This property provides diagnostics for the utility of elaboration in terms of the information value of each type of information over the other. The ME model with quantiles and moments is represented as the mixture of truncated distributions on consecutive intervals whose shapes and existence are determined by the moments. Elaborations of several ME distributions by quantiles are presented. The ME model based only on quantiles elicited by the fixed interval method possesses a useful property for pooling information elicited from multiple experts. The elaboration of Laplace distribution is an extension of the information theory connection with minimum risk under symmetric loss functions to the asymmetric linear loss. This extension produces a new Asymmetric Laplace distribution. Application examples compare ME priors with a parametric model fitted to elicited quantiles, illustrate measuring uncertainty and disagreement of economic forecasters based on elicited probabilities, and adjust ME models for a fundamental quantile in an inventory management problem.
| S-EPMC7414396 | biostudies-literature
Project description:BackgroundAfrican trypanosomiasis is a tsetse-borne parasitic infection that affects humans, wildlife, and domesticated animals. Tsetse flies are endemic to much of Sub-Saharan Africa and a spatial and temporal understanding of tsetse habitat can aid surveillance and support disease risk management. Problematically, current fine spatial resolution remote sensing data are delivered with a temporal lag and are relatively coarse temporal resolution (e.g., 16 days), which results in disease control models often targeting incorrect places. The goal of this study was to devise a heuristic for identifying tsetse habitat (at a fine spatial resolution) into the future and in the temporal gaps where remote sensing and proximal data fail to supply information.MethodsThis paper introduces a generalizable and scalable open-access version of the tsetse ecological distribution (TED) model used to predict tsetse distributions across space and time, and contributes a geospatial Bayesian Maximum Entropy (BME) prediction model trained by TED output data to forecast where, herein the Morsitans group of tsetse, persist in Kenya, a method that mitigates the temporal lag problem. This model facilitates identification of tsetse habitat and provides critical information to control tsetse, mitigate the impact of trypanosomiasis on vulnerable human and animal populations, and guide disease minimization in places with ephemeral tsetse. Moreover, this BME analysis is one of the first to utilize cluster and parallel computing along with a Monte Carlo analysis to optimize BME computations. This allows for the analysis of an exceptionally large dataset (over 2 billion data points) at a finer resolution and larger spatiotemporal scale than what had previously been possible.ResultsUnder the most conservative assessment for Kenya, the BME kriging analysis showed an overall prediction accuracy of 74.8% (limited to the maximum suitability extent). In predicting tsetse distribution outcomes for the entire country the BME kriging analysis was 97% accurate in its forecasts.ConclusionsThis work offers a solution to the persistent temporal data gap in accurate and spatially precise rainfall predictions and the delayed processing of remotely sensed data collectively in the - 45 days past to + 180 days future temporal window. As is shown here, the BME model is a reliable alternative for forecasting future tsetse distributions to allow preplanning for tsetse control. Furthermore, this model provides guidance on disease control that would otherwise not be available. These 'big data' BME methods are particularly useful for large domain studies. Considering that past BME studies required reduction of the spatiotemporal grid to facilitate analysis. Both the GEE-TED and the BME libraries have been made open source to enable reproducibility and offer continual updates into the future as new remotely sensed data become available.
| S-EPMC10655428 | biostudies-literature
Project description:When constructing discrete (binned) distributions from samples of a data set, applications exist where it is desirable to assure that all bins of the sample distribution have nonzero probability. For example, if the sample distribution is part of a predictive model for which we require returning a response for the entire codomain, or if we use Kullback-Leibler divergence to measure the (dis-)agreement of the sample distribution and the original distribution of the variable, which, in the described case, is inconveniently infinite. Several sample-based distribution estimators exist which assure nonzero bin probability, such as adding one counter to each zero-probability bin of the sample histogram, adding a small probability to the sample pdf, smoothing methods such as Kernel-density smoothing, or Bayesian approaches based on the Dirichlet and Multinomial distribution. Here, we suggest and test an approach based on the Clopper-Pearson method, which makes use of the binominal distribution. Based on the sample distribution, confidence intervals for bin-occupation probability are calculated. The mean of each confidence interval is a strictly positive estimator of the true bin-occupation probability and is convergent with increasing sample size. For small samples, it converges towards a uniform distribution, i.e., the method effectively applies a maximum entropy approach. We apply this nonzero method and four alternative sample-based distribution estimators to a range of typical distributions (uniform, Dirac, normal, multimodal, and irregular) and measure the effect with Kullback-Leibler divergence. While the performance of each method strongly depends on the distribution type it is applied to, on average, and especially for small sample sizes, the nonzero, the simple "add one counter", and the Bayesian Dirichlet-multinomial model show very similar behavior and perform best. We conclude that, when estimating distributions without an a priori idea of their shape, applying one of these methods is favorable.
| S-EPMC7513126 | biostudies-literature
Project description:We present a maximum entropy framework to separate intrinsic and extrinsic contributions to noisy gene expression solely from the profile of expression. We express the experimentally accessible probability distribution of the copy number of the gene product (mRNA or protein) by accounting for possible variations in extrinsic factors. The distribution of extrinsic factors is estimated using the maximum entropy principle. Our results show that extrinsic factors qualitatively and quantitatively affect the probability distribution of the gene product. We work out, in detail, the transcription of mRNA from a constitutively expressed promoter in Escherichia coli. We suggest that the variation in extrinsic factors may account for the observed wider-than-Poisson distribution of mRNA copy numbers. We successfully test our framework on a numerical simulation of a simple gene expression scheme that accounts for the variation in extrinsic factors. We also make falsifiable predictions, some of which are tested on previous experiments in E. coli whereas others need verification. Application of the presented framework to more complex situations is also discussed.
| S-EPMC4098093 | biostudies-literature
Project description:Many chemical reaction networks in biological systems present complex oscillatory dynamics. In systems such as regulatory gene networks, cell cycle, and enzymatic processes, the number of molecules involved is often far from the thermodynamic limit. Although stochastic models based on the probabilistic approach of the Chemical Master Equation (CME) have been proposed, studies in the literature have been limited by the challenges of solving the CME and the lack of computational power to perform large-scale stochastic simulations. In this paper, we show that the infinite set of stationary moment equations describing the stochastic Brusselator and Schnakenberg oscillatory reactions networks can be truncated and solved using maximization of the entropy of the distributions. The results from our numerical experiments compare with the distributions obtained from well-established kinetic Monte Carlo methods and suggest that the accuracy of the prediction increases exponentially with the closure order chosen for the system. We conclude that maximum entropy models can be used as an efficient closure scheme alternative for moment equations to predict the non-equilibrium stationary distributions of stochastic chemical reactions with oscillatory dynamics. This prediction is accomplished without any prior knowledge of the system dynamics and without imposing any biased assumptions on the mathematical relations among species involved.
| S-EPMC6424531 | biostudies-literature
Project description:ObjectiveExisting neuroimaging studies of psychotic and mood disorders have reported regional brain activation differences (first-order properties) and alterations in functional connectivity based on pairwise correlations in activation (second-order properties). This study used a generalized Ising model, also called a pairwise maximum entropy model (MEM), to integrate first- and second-order properties to provide a comprehensive picture of BOLD patterns and a system-wide summary measure called energy. This study examines the usefulness of individual level MEMs, attempts to identify image-derived counterparts of the model, and explores potential applications to psychiatry.MethodMEMs are fit to resting state fMRI data of each individual of a sample of 132 participants consisting of schizophrenia/schizoaffective disorder, bipolar disorder, and major depression, and a demographically matched 132 participants without these diagnoses from the UK Biobank to examine the default mode network (DMN).ResultsThe model explained observed brain state occurrence probabilities well across all participants, and model parameters were highly correlated to image-derived parameters for all groups. Within clinical groups, schizophrenia/schizoaffective disorder and bipolar disorder patients showed significant differences in averaged energy distribution compared to controls for all sub-systems of the DMN except for depression, where differences in the energy distributions were only detected in the DMN of the regions from the right hemisphere.ConclusionsSubject-specific Ising modeling may offer an improved measure of biological functional correlates relative to traditional approaches. The observation of distinct patterns of energy distribution among the three clinical groups compared to controls suggests relative diagnostic specificity and potential for clinical translation.
| S-EPMC10849576 | biostudies-literature
Project description:Food webs are complex ecological networks whose structure is both ecologically and statistically constrained, with many network properties being correlated with each other. Despite the recognition of these invariable relationships in food webs, the use of the principle of maximum entropy (MaxEnt) in network ecology is still rare. This is surprising considering that MaxEnt is a statistical tool precisely designed for understanding and predicting many types of constrained systems. This principle asserts that the least-biased probability distribution of a system's property, constrained by prior knowledge about that system, is the one with maximum information entropy. MaxEnt has been proven useful in many ecological modeling problems, but its application in food webs and other ecological networks is limited. Here we show how MaxEnt can be used to derive many food-web properties both analytically and heuristically. First, we show how the joint degree distribution (the joint probability distribution of the numbers of prey and predators for each species in the network) can be derived analytically using the number of species and the number of interactions in food webs. Second, we present a heuristic and flexible approach of finding a network's adjacency matrix (the network's representation in matrix format) based on simulated annealing and SVD entropy. We built two heuristic models using the connectance and the joint degree sequence as statistical constraints, respectively. We compared both models' predictions against corresponding null and neutral models commonly used in network ecology using open access data of terrestrial and aquatic food webs sampled globally (N = 257). We found that the heuristic model constrained by the joint degree sequence was a good predictor of many measures of food-web structure, especially the nestedness and motifs distribution. Specifically, our results suggest that the structure of terrestrial and aquatic food webs is mainly driven by their joint degree distribution.
| S-EPMC10503755 | biostudies-literature
Project description:Predictive models of signaling networks are essential for understanding cell population heterogeneity and designing rational interventions in disease. However, using computational models to predict heterogeneity of signaling dynamics is often challenging because of the extensive variability of biochemical parameters across cell populations. Here, we describe a maximum entropy-based framework for inference of heterogeneity in dynamics of signaling networks (MERIDIAN). MERIDIAN estimates the joint probability distribution over signaling network parameters that is consistent with experimentally measured cell-to-cell variability of biochemical species. We apply the developed approach to investigate the response heterogeneity in the EGFR/Akt signaling network. Our analysis demonstrates that a significant fraction of cells exhibits high phosphorylated Akt (pAkt) levels hours after EGF stimulation. Our findings also suggest that cells with high EGFR levels predominantly contribute to the subpopulation of cells with high pAkt activity. We also discuss how MERIDIAN can be extended to accommodate various experimental measurements.
| S-EPMC7047530 | biostudies-literature
Project description:Modelling the spatial distributions of human parasite species is crucial to understanding the environmental determinants of infection as well as for guiding the planning of control programmes. Here, we use ecological niche modelling to map the current potential distribution of the macroparasitic disease, lymphatic filariasis (LF), in Africa, and to estimate how future changes in climate and population could affect its spread and burden across the continent. We used 508 community-specific infection presence data collated from the published literature in conjunction with five predictive environmental/climatic and demographic variables, and a maximum entropy niche modelling method to construct the first ecological niche maps describing potential distribution and burden of LF in Africa. We also ran the best-fit model against climate projections made by the HADCM3 and CCCMA models for 2050 under A2a and B2a scenarios to simulate the likely distribution of LF under future climate and population changes. We predict a broad geographic distribution of LF in Africa extending from the west to the east across the middle region of the continent, with high probabilities of occurrence in the Western Africa compared to large areas of medium probability interspersed with smaller areas of high probability in Central and Eastern Africa and in Madagascar. We uncovered complex relationships between predictor ecological niche variables and the probability of LF occurrence. We show for the first time that predicted climate change and population growth will expand both the range and risk of LF infection (and ultimately disease) in an endemic region. We estimate that populations at risk to LF may range from 543 and 804 million currently, and that this could rise to between 1.65 to 1.86 billion in the future depending on the climate scenario used and thresholds applied to signify infection presence.
| S-EPMC3281123 | biostudies-literature
Project description:Maximum entropy (MaxEnt) spectral reconstruction methods provide a powerful framework for spectral estimation of nonuniformly sampled datasets. Many methods exist within this framework, usually defined based on the magnitude of a Lagrange multiplier in the MaxEnt objective function. An algorithm is presented here that utilizes accelerated first-order convex optimization techniques to rapidly and reliably reconstruct nonuniformly sampled NMR datasets using the principle of maximum entropy. This algorithm - called CAMERA for Convex Accelerated Maximum Entropy Reconstruction Algorithm - is a new approach to spectral reconstruction that exhibits fast, tunable convergence in both constant-aim and constant-lambda modes. A high-performance, open source NMR data processing tool is described that implements CAMERA, and brief comparisons to existing reconstruction methods are made on several example spectra.
| S-EPMC4818692 | biostudies-literature