A Bayesian Null Interval Hypothesis Test Controls False Discovery Rates and Improves Sensitivity in Label-Free Quantitative Proteomics.
Ontology highlight
ABSTRACT:
SUBMITTER: Millikin RJ
PROVIDER: S-EPMC7294927 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:We compared 2 statistical hypothesis-test approaches (no-observed-effect concentration [NOEC] and test of significant toxicity [TST]) to determine the influence of laboratory test performance on the false-positive error rate using the US Environmental Protection Agency's Ceriodaphnia dubia reproduction whole-effluent toxicity (WET) test endpoint. Simulation and power calculations were used to determine error rates based on observed control coefficients of variation (CV) for 8 laboratories over a range of effect levels. Average C. dubia control reproduction among laboratories was 20 to 40 offspring per female, and the 75th percentile CV was 0.10 to 0.31, reflecting a range in laboratory performance. The 2 approaches behave similarly for CVs of 0.2 to 0.3. At effects <10%, as CV decreases, TST is less likely to declare toxicity and NOEC is more likely to do so. Laboratory performance affects whether a sample is declared toxic and influences the probability of false-positive (and -negative) error rates using either approach. At the 75th percentile control CV observed for each laboratory, 4 laboratories would achieve approximately a 5% false-positive rate using 13 or fewer replicates for this test method. For the remaining 4 laboratories, more replicates would be needed to achieve a 5% false-positive rate. The present analyses demonstrate how false-positive rates are influenced by laboratory performance and WET test design. Environ Toxicol Chem 2019;38:511-523. Published 2019 Wiley Periodicals Inc. on behalf of SETAC. This article is a US government work and, as such, is in the public domain in the United States of America.
| S-EPMC6869336 | biostudies-literature
Project description:Null hypothesis significance testing (NHST) has been the subject of debate for decades and alternative approaches to data analysis have been proposed. This article addresses this debate from the perspective of scientific inquiry and inference. Inference is an inverse problem and application of statistical methods cannot reveal whether effects exist or whether they are empirically meaningful. Hence, raising conclusions from the outcomes of statistical analyses is subject to limitations. NHST has been criticized for its misuse and the misconstruction of its outcomes, also stressing its inability to meet expectations that it was never designed to fulfil. Ironically, alternatives to NHST are identical in these respects, something that has been overlooked in their presentation. Three of those alternatives are discussed here (estimation via confidence intervals and effect sizes, quantification of evidence via Bayes factors, and mere reporting of descriptive statistics). None of them offers a solution to the problems that NHST is purported to have, all of them are susceptible to misuse and misinterpretation, and some bring around their own problems (e.g., Bayes factors have a one-to-one correspondence with p values, but they are entirely deprived of an inferential framework). Those alternatives also fail to cover a broad area of inference not involving distributional parameters, where NHST procedures remain the only (and suitable) option. Like knives or axes, NHST is not inherently evil; only misuse and misinterpretation of its outcomes needs to be eradicated.
| S-EPMC5991793 | biostudies-literature
Project description:Global hypothesis tests are a useful tool in the context of clinical trials, genetic studies, or meta-analyses, when researchers are not interested in testing individual hypotheses, but in testing whether none of the hypotheses is false. There are several possibilities how to test the global null hypothesis when the individual null hypotheses are independent. If it is assumed that many of the individual null hypotheses are false, combination tests have been recommended to maximize power. If, however, it is assumed that only one or a few null hypotheses are false, global tests based on individual test statistics are more powerful (e.g. Bonferroni or Simes test). However, usually there is no a priori knowledge on the number of false individual null hypotheses. We therefore propose an omnibus test based on cumulative sums of the transformed p-values. We show that this test yields an impressive overall performance. The proposed method is implemented in an R-package called omnibus.
| S-EPMC6676337 | biostudies-literature
Project description:BackgroundMany studies have provided algorithms or methods to assess a statistical significance in quantitative proteomics when multiple replicates for a protein sample and a LC/MS analysis are available. But, confidence is still lacking in using datasets for a biological interpretation without protein sample replicates. Although a fold-change is a conventional threshold that can be used when there are no sample replicates, it does not provide an assessment of statistical significance such as a false discovery rate (FDR) which is an important indicator of the reliability to identify differentially expressed proteins. In this work, we investigate whether differentially expressed proteins can be detected with a statistical significance from a pair of unlabeled protein samples without replicates and with only duplicate LC/MS injections per sample. A FDR is used to gauge the statistical significance of the differentially expressed proteins.ResultsWe have experimented to operate on several parameters to control a FDR, including a fold-change, a statistical test, and a minimum number of permuted significant pairings. Although none of these parameters alone gives a satisfactory control of a FDR, we find that a combination of these parameters provides a very effective means to control a FDR without compromising the sensitivity. The results suggest that it is possible to perform a significance analysis without protein sample replicates. Only duplicate LC/MS injections per sample are needed. We illustrate that differentially expressed proteins can be detected with a FDR between 0 and 15% at a positive rate of 4-16%. The method is evaluated for its sensitivity and specificity by a ROC analysis, and is further validated with a [15N]-labeled internal-standard protein sample and additional unlabeled protein sample replicates.ConclusionWe demonstrate that a statistical significance can be inferred without protein sample replicates in label-free quantitative proteomics. The approach described in this study would be useful in many exploratory experiments where a sample amount or instrument time is limited. Naturally, this method is also suitable for proteomics experiments where multiple sample replicates are available. It is simple, and is complementary to other more sophisticated algorithms that are not designed for dealing with a small number of sample replicates.
| S-EPMC2645366 | biostudies-literature
Project description:Planktic foraminifer Globigerinoides ruber (G. ruber), due to its abundance and ubiquity in the tropical/subtropical mixed layer, has been the workhorse of paleoceanographic studies investigating past sea-surface conditions on a range of timescales. Recent geochemical work on the two principal white G. ruber (W) morphotypes, sensu stricto (ss) and sensu lato (sl), has hypothesized differences in seasonal preferences or calcification depths, implying that reconstructions using a non-selective mixture of morphotypes could potentially be biased. Here, we test these hypotheses by performing stable isotope and abundance measurements on the two morphotypes in sediment trap, core-top, and downcore samples from the northern Gulf of Mexico. As a test of null hypothesis, we perform the same analyses on couplets of G. ruber (W) specimens with attributes intermediate to the holotypic ss and sl morphologies. We find no systematic or significant offsets in coeval ss-sl ?(18)O, and ?(13)C. These offsets are no larger than those in the intermediate pairs. Coupling our results with foraminiferal statistical model INFAUNAL, we find that contrary to previous work elsewhere, there is no evidence for discrepancies in ss-sl calcifying depth habitat or seasonality in the Gulf of Mexico.
| S-EPMC5381429 | biostudies-literature
Project description:BACKGROUND:Although null hypothesis significance testing (NHST) is the agreed gold standard in medical decision making and the most widespread inferential framework used in medical research, it has several drawbacks. Bayesian methods can complement or even replace frequentist NHST, but these methods have been underutilised mainly due to a lack of easy-to-use software. JASP is an open-source software for common operating systems, which has recently been developed to make Bayesian inference more accessible to researchers, including the most common tests, an intuitive graphical user interface and publication-ready output plots. This article provides a non-technical introduction to Bayesian hypothesis testing in JASP by comparing traditional tests and statistical methods with their Bayesian counterparts. RESULTS:The comparison shows the strengths and limitations of JASP for frequentist NHST and Bayesian inference. Specifically, Bayesian hypothesis testing via Bayes factors can complement and even replace NHST in most situations in JASP. While p-values can only reject the null hypothesis, the Bayes factor can state evidence for both the null and the alternative hypothesis, making confirmation of hypotheses possible. Also, effect sizes can be precisely estimated in the Bayesian paradigm via JASP. CONCLUSIONS:Bayesian inference has not been widely used by now due to the dearth of accessible software. Medical decision making can be complemented by Bayesian hypothesis testing in JASP, providing richer information than single p-values and thus strengthening the credibility of an analysis. Through an easy point-and-click interface researchers used to other graphical statistical packages like SPSS can seemlessly transition to JASP and benefit from the listed advantages with only few limitations.
| S-EPMC7275319 | biostudies-literature
Project description:Parkinson's disease (PD) is characterized by loss of dopaminergic neurons in the substantia nigra and formation of intracytoplasmic Lewy bodies (LBs). Loss-of-function mutations in parkin which encodes an E3 ubiquitin protein ligase contribute to a predominant cause of a familial form of PD termed autosomal recessive juvenile Parkinsonism (AR-JP). Drosophila parkin null mutants display muscle degeneration and mitochondrial dysfunction, providing an animal model to study Parkin-associated molecular pathways in PD. To define protein alterations involved in Parkin pathogenesis, we performed quantitative proteomic analyses of Drosophila parkin null mutants and age-matched controls utilizing both global internal standard technology (GIST) and extracted ion chromatogram peak area (XICPA) label-free approaches. A total of 375 proteins were quantified with a minimum of two peptide identifications from the combination of the XICPA and GIST measurements applied to two independent biological replicates. Sixteen proteins exhibited significant alteration. Seven of the dysregulated proteins are involved in energy metabolism, of which six were down-regulated. All five proteins involved in transporter activity exhibited higher levels, of which larval serum protein 1alpha, larval serum protein 1beta, larval serum protein 1gamma, and fat body protein 1 showed >10-fold up-regulation and substantially higher level of fat body protein 1 was confirmed by Western blot analysis. These findings suggest that abnormalities in energy metabolism and protein transporter activity pathways may be associated with the pathogenesis of Parkin-associated AR-JP.
| S-EPMC2766925 | biostudies-literature
Project description:Classical null hypothesis significance testing is limited to the rejection of the point-null hypothesis; it does not allow the interpretation of non-significant results. This leads to a bias against the null hypothesis. Herein, we discuss statistical approaches to 'null effect' assessment focusing on the Bayesian parameter inference (BPI). Although Bayesian methods have been theoretically elaborated and implemented in common neuroimaging software packages, they are not widely used for 'null effect' assessment. BPI considers the posterior probability of finding the effect within or outside the region of practical equivalence to the null value. It can be used to find both 'activated/deactivated' and 'not activated' voxels or to indicate that the obtained data are not sufficient using a single decision rule. It also allows to evaluate the data as the sample size increases and decide to stop the experiment if the obtained data are sufficient to make a confident inference. To demonstrate the advantages of using BPI for fMRI data group analysis, we compare it with classical null hypothesis significance testing on empirical data. We also use simulated data to show how BPI performs under different effect sizes, noise levels, noise distributions and sample sizes. Finally, we consider the problem of defining the region of practical equivalence for BPI and discuss possible applications of BPI in fMRI studies. To facilitate 'null effect' assessment for fMRI practitioners, we provide Statistical Parametric Mapping 12 based toolbox for Bayesian inference.
| S-EPMC8674455 | biostudies-literature
Project description:MOTIVATION: Finding genes that are preferentially expressed in a particular tissue or condition is a problem that cannot be solved by standard statistical testing procedures. A relatively unknown procedure that can be used is the intersection-union test (IUT). However, two disadvantages of the IUT are that it is conservative and it conveys only the information of the least differing target tissue-other tissue pair. RESULTS: We propose a Bayesian procedure that quantifies how much evidence there is in the overall expression profile for selective over-expression. In a small simulation study, it is shown that the proposed method outperforms the IUT when it comes to finding selectively expressed genes. An application to publicly available data consisting of 22 tissues shows that the Bayesian method indeed selects genes with functions that reflect the specific tissue functions. The proposed method can also be used to find genes that are underexpressed in a particular tissue. AVAILABILITY: Both MATLAB and R code that implement the IUT and the Bayesian procedure in an efficient way, can be downloaded at http://ppw.kuleuven.be/okp/software/BayesianIUT/.
| S-EPMC2752611 | biostudies-literature
Project description:BackgroundProteomic protein identification results need to be compared across laboratories and platforms, and thus a reliable method is needed to estimate false discovery rates. The target-decoy strategy is a platform-independent and thus a prime candidate for standardized reporting of data. In its current usage based on global population parameters, the method does not utilize individual peptide scores optimally.ResultsHere we show that proteomic analyses largely benefit from using separate treatment of peptides matching to proteins alone or in groups based on locally estimated false discovery rates. Our implementation reduces the number of false positives and simultaneously increases the number of proteins identified. Importantly, single peptide identifications achieve defined confidence and the sequence coverage of proteins is optimized. As a result, we improve the number of proteins identified in a human serum analysis by 58% without compromising identification confidence.ConclusionWe show that proteins can reliably be identified with a single peptide and the sequence coverage for multi-peptide proteins can be increased when using an improved estimation of false discovery rates.
| S-EPMC2709624 | biostudies-literature