So many correlated tests, so little time! Rapid adjustment of P values for multiple correlated tests.
Ontology highlight
ABSTRACT:
SUBMITTER: Conneely KN
PROVIDER: S-EPMC2276357 | biostudies-other | 2007 Dec
REPOSITORIES: biostudies-other
Similar Datasets
Project description:In high dimensional data analysis (such as gene expression, spatial epidemiology, or brain imaging studies), we often test thousands or more hypotheses simultaneously. As the number of tests increases, the chance of observing some statistically significant tests is very high even when all null hypotheses are true. Consequently, we could reach incorrect conclusions regarding the hypotheses. Researchers frequently use multiplicity adjustment methods to control type I error rates-primarily the family-wise error rate (FWER) or the false discovery rate (FDR)-while still desiring high statistical power. In practice, such studies may have dependent test statistics (or p-values) as tests can be dependent on each other. However, some commonly-used multiplicity adjustment methods assume independent tests. We perform a simulation study comparing several of the most common adjustment methods involved in multiple hypothesis testing, under varying degrees of block-correlation positive dependence among tests.
| S-EPMC5409054 | biostudies-literature
Project description:The relative contribution of advantageous and neutral mutations to the evolutionary process is a central problem in evolutionary biology. Current estimates suggest that whereas Drosophila, mice, and bacteria have undergone extensive adaptive evolution, hominids show little or no evidence of adaptive evolution in protein-coding sequences. This may be a consequence of differences in effective population size. To study the matter further, we have investigated whether plants show evidence of adaptive evolution using an extension of the McDonald-Kreitman test that explicitly models slightly deleterious mutations by estimating the distribution of fitness effects of new mutations. We apply this method to data from nine pairs of species. Altogether more than 2,400 loci with an average length of approximately 280 nucleotides were analyzed. We observe very similar results in all species; we find little evidence of adaptive amino acid substitution in any comparison except sunflowers. This may be because many plant species have modest effective population sizes.
| S-EPMC2915642 | biostudies-literature
Project description:In plants, the carotenoid biosynthesis pathway (CBP) is essential for the production of photosynthetic and protective pigments, plant hormones, and visual/olfactory attractants for animal pollinators and seed dispersers. The regulation of carotenoid biosynthesis at the transcriptional level is vitally important for all of these functions and has been the subject of intensive research. Many putative transcriptional regulators, both direct and indirect, have been identified through conventional mutant analysis, transcriptome profiling, yeast one-hybrid screening, and candidate gene approaches. Despite this progress, our understanding of the transcriptional regulation of carotenoid biosynthesis remains fragmented and incomplete. Frequently, a stimulus or regulator is known, but the mechanism by which it affects transcription has not been elucidated. In other cases, mechanisms have been proposed (such as direct binding of a CBP gene promoter by a transcription factor), but function was tested only in vitro or in heterologous systems, making it unclear whether these proteins actually play a role in carotenoid regulation in their endogenous environments. Even in cases where the mechanism is relatively well understood, regulators are often studied in isolation, either in a single plant species or outside the context of other known regulators. This presents a conundrum: why so many candidate regulators but so little consensus? Here we summarize current knowledge on transcriptional regulation of the CBP, lay out the challenges contributing to this conundrum, identify remaining knowledge gaps, and suggest future research directions to address these challenges and knowledge gaps.
| S-EPMC6695471 | biostudies-literature
Project description:With the development of high-throughput sequencing and genotyping technologies, the number of markers collected in genetic association studies is growing rapidly, increasing the importance of methods for correcting for multiple hypothesis testing. The permutation test is widely considered the gold standard for accurate multiple testing correction, but it is often computationally impractical for these large datasets. Recently, several studies proposed efficient alternative approaches to the permutation test based on the multivariate normal distribution (MVN). However, they cannot accurately correct for multiple testing in genome-wide association studies for two reasons. First, these methods require partitioning of the genome into many disjoint blocks and ignore all correlations between markers from different blocks. Second, the true null distribution of the test statistic often fails to follow the asymptotic distribution at the tails of the distribution. We propose an accurate and efficient method for multiple testing correction in genome-wide association studies--SLIDE. Our method accounts for all correlation within a sliding window and corrects for the departure of the true null distribution of the statistic from the asymptotic distribution. In simulations using the Wellcome Trust Case Control Consortium data, the error rate of SLIDE's corrected p-values is more than 20 times smaller than the error rate of the previous MVN-based methods' corrected p-values, while SLIDE is orders of magnitude faster than the permutation test and other competing methods. We also extend the MVN framework to the problem of estimating the statistical power of an association study with correlated markers and propose an efficient and accurate power estimation method SLIP. SLIP and SLIDE are available at http://slide.cs.ucla.edu.
| S-EPMC2663787 | biostudies-literature
Project description:BackgroundMachine learning (ML) can be an effective tool to extract information from attribute-rich molecular datasets for the generation of molecular diagnostic tests. However, the way in which the resulting scores or classifications are produced from the input data may not be transparent. Algorithmic explainability or interpretability has become a focus of ML research. Shapley values, first introduced in game theory, can provide explanations of the result generated from a specific set of input data by a complex ML algorithm.MethodsFor a multivariate molecular diagnostic test in clinical use (the VeriStrat® test), we calculate and discuss the interpretation of exact Shapley values. We also employ some standard approximation techniques for Shapley value computation (local interpretable model-agnostic explanation (LIME) and Shapley Additive Explanations (SHAP) based methods) and compare the results with exact Shapley values.ResultsExact Shapley values calculated for data collected from a cohort of 256 patients showed that the relative importance of attributes for test classification varied by sample. While all eight features used in the VeriStrat® test contributed equally to classification for some samples, other samples showed more complex patterns of attribute importance for classification generation. Exact Shapley values and Shapley-based interaction metrics were able to provide interpretable classification explanations at the sample or patient level, while patient subgroups could be defined by comparing Shapley value profiles between patients. LIME and SHAP approximation approaches, even those seeking to include correlations between attributes, produced results that were quantitatively and, in some cases qualitatively, different from the exact Shapley values.ConclusionsShapley values can be used to determine the relative importance of input attributes to the result generated by a multivariate molecular diagnostic test for an individual sample or patient. Patient subgroups defined by Shapley value profiles may motivate translational research. However, correlations inherent in molecular data and the typically small ML training sets available for molecular diagnostic test development may cause some approximation methods to produce approximate Shapley values that differ both qualitatively and quantitatively from exact Shapley values. Hence, caution is advised when using approximate methods to evaluate Shapley explanations of the results of molecular diagnostic tests.
| S-EPMC8265031 | biostudies-literature
Project description:We consider large-scale studies in which thousands of significance tests are performed simultaneously. In some of these studies, the multiple testing procedure can be severely biased by latent confounding factors such as batch effects and unmeasured covariates that correlate with both primary variable(s) of interest (e.g., treatment variable, phenotype) and the outcome. Over the past decade, many statistical methods have been proposed to adjust for the confounders in hypothesis testing. We unify these methods in the same framework, generalize them to include multiple primary variables and multiple nuisance variables, and analyze their statistical properties. In particular, we provide theoretical guarantees for RUV-4 [Gagnon-Bartsch, Jacob and Speed (2013)] and LEAPP [Ann. Appl. Stat. 6 (2012) 1664-1688], which correspond to two different identification conditions in the framework: the first requires a set of "negative controls" that are known a priori to follow the null distribution; the second requires the true nonnulls to be sparse. Two different estimators which are based on RUV-4 and LEAPP are then applied to these two scenarios. We show that if the confounding factors are strong, the resulting estimators can be asymptotically as powerful as the oracle estimator which observes the latent confounding factors. For hypothesis testing, we show the asymptotic z-tests based on the estimators can control the type I error. Numerical experiments show that the false discovery rate is also controlled by the Benjamini-Hochberg procedure when the sample size is reasonably large.
| S-EPMC6706069 | biostudies-literature
Project description:In this paper, we present and benchmark FilterNet, a flexible deep learning architecture for time series classification tasks, such as activity recognition via multichannel sensor data. It adapts popular convolutional neural network (CNN) and long short-term memory (LSTM) motifs which have excelled in activity recognition benchmarks, implementing them in a many-to-many architecture to markedly improve frame-by-frame accuracy, event segmentation accuracy, model size, and computational efficiency. We propose several model variants, evaluate them alongside other published models using the Opportunity benchmark dataset, demonstrate the effect of model ensembling and of altering key parameters, and quantify the quality of the models' segmentation of discrete events. We also offer recommendations for use and suggest potential model extensions. FilterNet advances the state of the art in all measured accuracy and speed metrics when applied to the benchmarked dataset, and it can be extensively customized for other applications.
| S-EPMC7249062 | biostudies-literature
Project description:In the EMO (evolutionary multi-objective, EMO) algorithm, MaOPs (many objective optimization problems, MaOPs) are sometimes difficult to keep the balance of convergence and diversity. The decomposition based EMO developed for MaOPs has been proved to be effective, and BBO/Complex (the biogeography based optimization for complex system, BBO/Complex) algorithm is a low complexity algorithm. In this paper, a decomposition and adaptive weight adjustment based BBO/Complex algorithm (DAWA-BBO/Complex) for MaOPs is proposed. First, a new method based on crowding distance is designed to generate a set of weight vectors with good uniformly. Second, an adaptive weight adjustment method is used to solve MaOPs with complex Pareto optimal front. Subsystem space obtains a non-dominated solution by a new selection strategy. The experimental results show that the algorithm is superior to other new algorithms in terms of convergence and diversity in DTLZ benchmark problems. Finally, the algorithm is used to solve the problem of NC (numerical control machine, NC) cutting parameters, and the final optimization result is obtained by AHP (Analytic Hierarchy Process, AHP) method. The results show that the cutting speed is 10.8m/min, back cutting depth is 0.13mm, the cutting time is 504s and the cutting cost is 22.15yuan. The proposed algorithm can effectively solve the practical optimization problem.
| S-EPMC7546505 | biostudies-literature
Project description:With recent developments of data technology in biomedicine, factor data such as diagnosis codes and genomic features, which can have tens to hundreds of discrete and unorderable categorical values, have emerged. While considered as a fundamental problem in statistical analyses, the estimation of probability distribution for such factor variables has not studied much because the previous studies have mainly focused on continuous variables and discrete factor variables with a few categories such as sex and race. In this work, we propose a nonparametric Bayesian procedure to estimate the probability distribution of factors with many categories. The proposed method was demonstrated through simulation studies under various conditions and showed significant improvements on the estimation errors from the previous conventional methods. In addition, the method was applied to the analysis of diagnosis data of intensive care unit patients, and generated interesting medical hypotheses. The overall results indicate that the proposed method will be useful in the analysis of biomedical factor data.
| S-EPMC6108477 | biostudies-literature
Project description:PurposeTo describe and demonstrate appropriate statistical approaches for estimating sensitivity, specificity, predictive values and their 95% confidence intervals (95% CI) for correlated eye data.MethodsWe described generalized estimating equations (GEE) and cluster bootstrap to account for inter-eye correlation and applied them for analyzing the data from a clinical study of telemedicine for the detection of retinopathy of prematurity (ROP).ResultsAmong 100 infants (200 eyes) selected for analysis, 20 infants had referral-warranted ROP (RW-ROP) in both eyes and 9 infants with RW-ROP only in one eye based on clinical eye examination. In the per-eye analysis that included both eyes of an infant, the image evaluation for RW-ROP had sensitivity of 83.7% and specificity of 86.8%. The 95% CI's from the naïve approach that ignored the inter-eye correlation were narrower than those of the GEE approach and cluster bootstrap for both sensitivity (width of 95% CI: 22.4% vs. 23.2% vs. 23.9%) and specificity (11.4% vs. 12.5% vs. 11.6%). The 95% CIs for sensitivity and specificity calculated from left eyes and right eyes separately were wider (35.2% and 30.8% respectively for sensitivity, 25.4% and 17.3% respectively for specificity).ConclusionsWhen an ocular test is performed in both eyes of some or all of the study subjects, the statistical analyses are best performed at the eye-level and account for the inter-eye correlation by using either the GEE or cluster bootstrap. Ignoring the inter-eye correlation results in 95% CIs that are inappropriately narrow and analyzing data from two eyes separately are not efficient.
| S-EPMC7500131 | biostudies-literature