Optimal prediction with resource constraints using the information bottleneck.
Ontology highlight
ABSTRACT:
SUBMITTER: Sachdeva V
PROVIDER: S-EPMC7971903 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:A large literature considers the optimal size and growth rate of the human population, trading off the utility value of additional people with the costs of a larger population. In this literature, an important parameter is the social weight placed on population size; a standard result is that a planner with a larger weight on population chooses larger population levels and growth rates. We demonstrate that this result is conditionally overturned when an exhaustible resource constraint is introduced: if the discount rate is small enough, the optimal population today decreases with the welfare weight on population size. That is, a more total-utilitarian social planner could prefer a smaller population today than a more average-utilitarian social planner. We also present a numerical illustration applied to the case of climate change, where we show that under plausible real-world parameter values, our result matters for the direction and magnitude of optimal population policy.
| S-EPMC5764548 | biostudies-literature
Project description:A common practice in predictive medicine is to use current study data to construct a stratification procedure, which groups subjects according to baseline information and forms stratum-specific prevention or intervention strategies. A desirable stratification scheme would not only have small intra-stratum variation but also have a clinically meaningful discriminatory capability. We show how to obtain optimal stratification rules with such desirable properties from fitting a set of regression models relating the outcome to baseline covariates and creating scoring systems for predicting potential outcomes. We propose that all available optimal stratifications be evaluated with an independent dataset to select a final stratification. Lastly, we obtain inferential results for this selected stratification scheme with a holdout dataset. When only one study of moderate size is available, we combine the first two steps via crossvalidation. Extensive simulation studies are used to compare the proposed stratification strategy with alternatives. We illustrate the new proposal using an AIDS clinical trial for binary outcomes and a cardiovascular clinical study for censored event time outcomes.
| S-EPMC5793688 | biostudies-literature
Project description:MotivationWe present a multi-sequence generalization of Variational Information Bottleneck and call the resulting model Attentive Variational Information Bottleneck (AVIB). Our AVIB model leverages multi-head self-attention to implicitly approximate a posterior distribution over latent encodings conditioned on multiple input sequences. We apply AVIB to a fundamental immuno-oncology problem: predicting the interactions between T-cell receptors (TCRs) and peptides.ResultsExperimental results on various datasets show that AVIB significantly outperforms state-of-the-art methods for TCR-peptide interaction prediction. Additionally, we show that the latent posterior distribution learned by AVIB is particularly effective for the unsupervised detection of out-of-distribution amino acid sequences.Availability and implementationThe code and the data used for this study are publicly available at: https://github.com/nec-research/vibtcr.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC9825246 | biostudies-literature
Project description:Information bottleneck (IB) is a technique for extracting information in one random variable X that is relevant for predicting another random variable Y. IB works by encoding X in a compressed “bottleneck” random variable M from which Y can be accurately decoded. However, finding the optimal bottleneck variable involves a difficult optimization problem, which until recently has been considered for only two limited cases: discrete X and Y with small state spaces, and continuous X and Y with a Gaussian joint distribution (in which case optimal encoding and decoding maps are linear). We propose a method for performing IB on arbitrarily-distributed discrete and/or continuous X and Y, while allowing for nonlinear encoding and decoding maps. Our approach relies on a novel non-parametric upper bound for mutual information. We describe how to implement our method using neural networks. We then show that it achieves better performance than the recently-proposed “variational IB” method on several real-world datasets.
| S-EPMC7514526 | biostudies-literature
Project description:We present a two-step approach to modeling the transmembrane spanning helical bundles of integral membrane proteins using only sparse distance constraints, such as those derived from chemical cross-linking, dipolar EPR and FRET experiments. In Step 1, using an algorithm, we developed, the conformational space of membrane protein folds matching a set of distance constraints is explored to provide initial structures for local conformational searches. In Step 2, these structures refined against a custom penalty function that incorporates both measures derived from statistical analysis of solved membrane protein structures and distance constraints obtained from experiments. We begin by describing the statistical analysis of the solved membrane protein structures from which the theoretical portion of the penalty function was derived. We then describe the penalty function, and, using a set of six test cases, demonstrate that it is capable of distinguishing helical bundles that are close to the native bundle from those that are far from the native bundle. Finally, using a set of only 27 distance constraints extracted from the literature, we show that our method successfully recovers the structure of dark-adapted rhodopsin to within 3.2 A of the crystal structure.
| S-EPMC2286557 | biostudies-literature
Project description:In magnetic resonance (MR) imaging, a lack of standardization in acquisition often causes pulse sequence-based contrast variations in MR images from site to site, which impedes consistent measurements in automatic analyses. In this paper, we propose an unsupervised MR image harmonization approach, CALAMITI (Contrast Anatomy Learning and Analysis for MR Intensity Translation and Integration), which aims to alleviate contrast variations in multi-site MR imaging. Designed using information bottleneck theory, CALAMITI learns a globally disentangled latent space containing both anatomical and contrast information, which permits harmonization. In contrast to supervised harmonization methods, our approach does not need a sample population to be imaged across sites. Unlike traditional unsupervised harmonization approaches which often suffer from geometry shifts, CALAMITI better preserves anatomy by design. The proposed method is also able to adapt to a new testing site with a straightforward fine-tuning process. Experiments on MR images acquired from ten sites show that CALAMITI achieves superior performance compared with other harmonization approaches.
| S-EPMC10473284 | biostudies-literature
Project description:Diauxie, or the sequential consumption of carbohydrates in bacteria such as Escherichia coli, has been hypothesized to be an evolutionary strategy which allows the organism to maximize its instantaneous specific growth-giving the bacterium a competitive advantage. Currently, the computational techniques used in industrial biotechnology fall short of explaining the intracellular dynamics underlying diauxic behavior. In particular, the understanding of the proteome dynamics in diauxie can be improved. We developed a robust iterative dynamic method based on expression- and thermodynamically enabled flux models to simulate the kinetic evolution of carbohydrate consumption and cellular growth. With minimal modeling assumptions, we couple kinetic uptakes, gene expression, and metabolic networks, at the genome scale, to produce dynamic simulations of cell cultures. The method successfully predicts the preferential uptake of glucose over lactose in E. coli cultures grown on a mixture of carbohydrates, a manifestation of diauxie. The simulated cellular states also show the reprogramming in the content of the proteome in response to fluctuations in the availability of carbon sources, and it captures the associated time lag during the diauxie phenotype. Our models suggest that the diauxic behavior of cells is the result of the evolutionary objective of maximization of the specific growth of the cell. We propose that genetic regulatory networks, such as the lac operon in E. coli, are the biological implementation of a robust control system to ensure optimal growth.
| S-EPMC7923608 | biostudies-literature
Project description:In order to target threatening pathogens, the adaptive immune system performs a continuous reorganization of its lymphocyte repertoire. Following an immune challenge, the B cell repertoire can evolve cells of increased specificity for the encountered strain. This process of affinity maturation generates a memory pool whose diversity and size remain difficult to predict. We assume that the immune system follows a strategy that maximizes the long-term immune coverage and minimizes the short-term metabolic costs associated with affinity maturation. This strategy is defined as an optimal decision process on a finite dimensional phenotypic space, where a preexisting population of cells is sequentially challenged with a neutrally evolving strain. We show that the low specificity and high diversity of memory B cells-a key experimental result-can be explained as a strategy to protect against pathogens that evolve fast enough to escape highly potent but narrow memory. This plasticity of the repertoire drives the emergence of distinct regimes for the size and diversity of the memory pool, depending on the density of de novo responding cells and on the mutation rate of the strain. The model predicts power-law distributions of clonotype sizes observed in data and rationalizes antigenic imprinting as a strategy to minimize metabolic costs while keeping good immune protection against future strains.
| S-EPMC8872716 | biostudies-literature
Project description:Extracting relevant information from data is crucial for all forms of learning. The information bottleneck (IB) method formalizes this, offering a mathematically precise and conceptually appealing framework for understanding learning phenomena. However the nonlinearity of the IB problem makes it computationally expensive and analytically intractable in general. Here we derive a perturbation theory for the IB method and report the first complete characterization of the learning onset-the limit of maximum relevant information per bit extracted from data. We test our results on synthetic probability distributions, finding good agreement with the exact numerical solution near the onset of learning. We explore the difference and subtleties in our derivation and previous attempts at deriving a perturbation theory for the learning onset and attribute the discrepancy to a flawed assumption. Our work also provides a fresh perspective on the intimate relationship between the IB method and the strong data processing inequality.
| S-EPMC9806839 | biostudies-literature
Project description:Difference constraints have been used for termination analysis in the literature, where they denote relational inequalities of the form x'?y+c , and describe that the value of x in the current state is at most the value of y in the previous state plus some constant c?Z . We believe that difference constraints are also a good choice for complexity and resource bound analysis because the complexity of imperative programs typically arises from counter increments and resets, which can be modeled naturally by difference constraints. In this article we propose a bound analysis based on difference constraints. We make the following contributions: (1) our analysis handles bound analysis problems of high practical relevance which current approaches cannot handle: we extend the range of bound analysis to a class of challenging but natural loop iteration patterns which typically appear in parsing and string-matching routines. (2) We advocate the idea of using bound analysis to infer invariants: our soundness proven algorithm obtains invariants through bound analysis, the inferred invariants are in turn used for obtaining bounds. Our bound analysis therefore does not rely on external techniques for invariant generation. (3) We demonstrate that difference constraints are a suitable abstract program model for automatic complexity and resource bound analysis: we provide efficient abstraction techniques for obtaining difference constraint programs from imperative code. (4) We report on a thorough experimental comparison of state-of-the-art bound analysis tools: we set up a tool comparison on (a) a large benchmark of real-world C code, (b) a benchmark built of examples taken from the bound analysis literature and (c) a benchmark of challenging iteration patterns which we found in real source code. (5) Our analysis is more scalable than existing approaches: we discuss how we achieve scalability.
| S-EPMC6044401 | biostudies-literature