Dataset Information

Statistical Uncertainty Analysis for Small-Sample, High Log-Variance Data: Cautions for Bootstrapping and Bayesian Bootstrapping.

ABSTRACT: Recent advances in molecular simulations allow the evaluation of previously unattainable observables, such as rate constants for protein folding. However, these calculations are usually computationally expensive, and even significant computing resources may result in a small number of independent estimates spread over many orders of magnitude. Such small-sample, high "log-variance" data are not readily amenable to analysis using the standard uncertainty (i.e., "standard error of the mean") because unphysical negative limits of confidence intervals result. Bootstrapping, a natural alternative guaranteed to yield a confidence interval within the minimum and maximum values, also exhibits a striking systematic bias of the lower confidence limit in log space. As we show, bootstrapping artifactually assigns high probability to improbably low mean values. A second alternative, the Bayesian bootstrap strategy, does not suffer from the same deficit and is more logically consistent with the type of confidence interval desired. The Bayesian bootstrap provides uncertainty intervals that are more reliable than those from the standard bootstrap method but must be used with caution nevertheless. Neither standard nor Bayesian bootstrapping can overcome the intrinsic challenge of underestimating the mean from small-size, high log-variance samples. Our conclusions are based on extensive analysis of model distributions and reanalysis of multiple independent atomistic simulations. Although we only analyze rate constants, similar considerations will apply to related calculations, potentially including highly nonlinear averages like the Jarzynski relation.

SUBMITTER: Mostofian B

PROVIDER: S-EPMC6754704 | biostudies-literature | 2019 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Statistical Uncertainty Analysis for Small-Sample, High Log-Variance Data: Cautions for Bootstrapping and Bayesian Bootstrapping.

Mostofian Barmak B Zuckerman Daniel M DM

Journal of chemical theory and computation 20190507 6

Recent advances in molecular simulations allow the evaluation of previously unattainable observables, such as rate constants for protein folding. However, these calculations are usually computationally expensive, and even significant computing resources may result in a small number of independent estimates spread over many orders of magnitude. Such small-sample, high "log-variance" data are not readily amenable to analysis using the standard uncertainty (i.e., "standard error of the mean") becau ...[more]

PMID: 31002504

Similar Datasets

Project description:Uncertainty analysis is the process of identifying limitations in scientific knowledge and evaluating their implications for scientific conclusions. In the context of microbial risk assessment, the uncertainty in the predicted microbial behavior can be an important component of the overall uncertainty. Conventional deterministic modeling approaches which provide point estimates of the pathogen's levels cannot quantify the uncertainty around the predictions. The objective of this study was to use Bayesian statistical modeling for describing uncertainty in predicted microbial thermal inactivation of Salmonella enterica Typhimurium DT104. A set of thermal inactivation data in broth with water activity adjusted to 0.75 at 9 different temperature conditions obtained from the ComBase database (www.combase.cc) was used. A log-linear microbial inactivation was used as a primary model while for secondary modeling, a linear relation between the logarithm of inactivation rate and temperature was assumed. For comparison, data were fitted with a two-step and a global Bayesian regression. Posterior distributions of model's parameters were used to predict Salmonella thermal inactivation. The combination of the joint posterior distributions of model's parameters allowed the prediction of cell density over time, total reduction time and inactivation rate as probability distributions at different time and temperature conditions. For example, for the time required to eliminate a Salmonella population of about 107 CFU/ml at 65°C, the model predicted a time distribution with a median of 0.40 min and 5th and 95th percentiles of 0.24 and 0.60 min, respectively. The validation of the model showed that it can describe successfully uncertainty in predicted thermal inactivation with most observed data being within the 95% prediction intervals of the model. The global regression approach resulted in less uncertain predictions compared to the two-step regression. The developed model could be used to quantify uncertainty in thermal inactivation in risk-based processing design as well as in risk assessment studies.

Project description:DNA microarrays open up a new horizon for studying the genetic determinants of disease. The high throughput nature of these arrays creates an enormous wealth of information, but also poses a challenge to data analysis. Inferential problems become even more pronounced as experimental designs used to collect data become more complex. An important example is multigroup data collected over different experimental groups, such as data collected from distinct stages of a disease process. We have developed a method specifically addressing these issues termed Bayesian ANOVA for microarrays (BAM). The BAM approach uses a special inferential regularization known as spike-and-slab shrinkage that provides an optimal balance between total false detections and total false non-detections. This translates into more reproducible differential calls. Spike and slab shrinkage is a form of regularization achieved by using information across all genes and groups simultaneously.BAMarray is a graphically oriented Java-based software package that implements the BAM method for detecting differentially expressing genes in multigroup microarray experiments (up to 256 experimental groups can be analyzed). Drop-down menus allow the user to easily select between different models and to choose various run options. BAMarraycan also be operated in a fully automated mode with preselected run options. Tuning parameters have been preset at theoretically optimal values freeing the user from such specifications. BAMarray provides estimates for gene differential effects and automatically estimates data adaptive, optimal cutoff values for classifying genes into biological patterns of differential activity across experimental groups. A graphical suite is a core feature of the product and includes diagnostic plots for assessing model assumptions and interactive plots that enable tracking of prespecified gene lists to study such things as biological pathway perturbations. The user can zoom in and lasso genes of interest that can then be saved for downstream analyses.BAMarray is user friendly platform independent software that effectively and efficiently implements the BAM methodology. Classifying patterns of differential activity is greatly facilitated by a data adaptive cutoff rule and a graphical suite. BAMarray is licensed software freely available to academic institutions. More information can be found at http://www.bamarray.com.

Dataset Information

Statistical Uncertainty Analysis for Small-Sample, High Log-Variance Data: Cautions for Bootstrapping and Bayesian Bootstrapping.

Publications

Statistical Uncertainty Analysis for Small-Sample, High Log-Variance Data: Cautions for Bootstrapping and Bayesian Bootstrapping.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure