Controlling the FDR in imperfect matches to an incomplete database.
Ontology highlight
ABSTRACT:
SUBMITTER: Keich U
PROVIDER: S-EPMC6287756 | biostudies-literature | 2018
REPOSITORIES: biostudies-literature
Publications
Controlling the FDR in imperfect matches to an incomplete database.
Keich Uri U Noble William Stafford WS
Journal of the American Statistical Association 20180628 523
We consider the problem of controlling the FDR among discoveries from searching an incomplete database. This problem differs from the classical multiple testing setting because there are two different types of false discoveries: those arising from objects that have no match in the database and those that are incorrectly matched. We show that commonly used FDR controlling procedures are inadequate for this setup, a special case of which is tandem mass spectrum identification. We then derive a nov ...[more]
Similar Datasets
Project description:Database search algorithms are the primary workhorses for the identification of tandem mass spectra. However, these methods are limited to the identification of spectra for which peptides are present in the database, preventing the identification of peptides from mutated or alternatively spliced sequences. A variety of methods has been developed to search a spectrum against a sequence allowing for variations. Some tools determine the sequence of the homologous protein in the related species but do not report the peptide in the target organism. Other tools consider variations, including modifications and mutations, in reconstructing the target sequence. However, these tools will not work if the template (homologous peptide) is missing in the database, and they do not attempt to reconstruct the entire protein target sequence. De novo identification of peptide sequences is another possibility, because it does not require a protein database. However, the lack of database reduces the accuracy. We present a novel proteogenomic approach, GenoMS, that draws on the strengths of database and de novo peptide identification methods. Protein sequence templates (i.e. proteins or genomic sequences that are similar to the target protein) are identified using the database search tool InsPecT. The templates are then used to recruit, align, and de novo sequence regions of the target protein that have diverged from the database or are missing. We used GenoMS to reconstruct the full sequence of an antibody by using spectra acquired from multiple digests using different proteases. Antibodies are a prime example of proteins that confound standard database identification techniques. The mature antibody genes result from large-scale genome rearrangements with flexible fusion boundaries and somatic hypermutation. Using GenoMS we automatically reconstruct the complete sequences of two immunoglobulin chains with accuracy greater than 98% using a diverged protein database. Using the genome as the template, we achieve accuracy exceeding 97%.
| S-EPMC2877985 | biostudies-literature
Project description:The prevailing method of analyzing GWAS data is still to test each marker individually, although from a statistical point of view it is quite obvious that in case of complex traits such single marker tests are not ideal. Recently several model selection approaches for GWAS have been suggested, most of them based on LASSO-type procedures. Here we will discuss an alternative model selection approach which is based on a modification of the Bayesian Information Criterion (mBIC2) which was previously shown to have certain asymptotic optimality properties in terms of minimizing the misclassification error. Heuristic search strategies are introduced which attempt to find the model which minimizes mBIC2, and which are efficient enough to allow the analysis of GWAS data. Our approach is implemented in a software package called MOSGWA. Its performance in case control GWAS is compared with the two algorithms HLASSO and d-GWASelect, as well as with single marker tests, where we performed a simulation study based on real SNP data from the POPRES sample. Our results show that MOSGWA performs slightly better than HLASSO, where specifically for more complex models MOSGWA is more powerful with only a slight increase in Type I error. On the other hand according to our simulations GWASelect does not at all control the type I error when used to automatically determine the number of important SNPs. We also reanalyze the GWAS data from the Wellcome Trust Case-Control Consortium and compare the findings of the different procedures, where MOSGWA detects for complex diseases a number of interesting SNPs which are not found by other methods.
| S-EPMC4111553 | biostudies-literature
Project description:Acute heart failure is a common condition associated with considerable morbidity, mortality, and cost. However, evidence-based data on treating heart failure in the acute setting are limited, and current individual treatment options have variable efficacy. The healthcare team must often individualize patient care in ways that may extend beyond available clinical guidelines. In this review, we address the question, "How do you do the best you can clinically with incomplete evidence and imperfect drugs?" Expert opinion is provided to supplement guideline-based recommendations and help address the typical challenges that are involved in the management of patients with acute heart failure. Specifically, we discuss 4 key areas that are important in the continuum of patient care: differential diagnosis and risk stratification; choice and implementation of initial therapy; assessment of the adequacy of therapy during hospitalization or observation; and considerations for discharge/transition of care. A case study is presented to highlight the decision-making process throughout each of these areas. Evidence is accumulating that should help guide patients and healthcare providers on a path to better quality of care.
| S-EPMC4342318 | biostudies-other
Project description:BACKGROUND: In high-dimensional data analysis such as differential gene expression analysis, people often use filtering methods like fold-change or variance filters in an attempt to reduce the multiple testing penalty and improve power. However, filtering may introduce a bias on the multiple testing correction. The precise amount of bias depends on many quantities, such as fraction of probes filtered out, filter statistic and test statistic used. RESULTS: We show that a biased multiple testing correction results if non-differentially expressed probes are not filtered out with equal probability from the entire range of p-values. We illustrate our results using both a simulation study and an experimental dataset, where the FDR is shown to be biased mostly by filters that are associated with the hypothesis being tested, such as the fold change. Filters that induce little bias on the FDR yield less additional power of detecting differentially expressed genes. Finally, we propose a statistical test that can be used in practice to determine whether any chosen filter introduces bias on the FDR estimate used, given a general experimental setup. CONCLUSIONS: Filtering out of probes must be used with care as it may bias the multiple testing correction. Researchers can use our test for FDR bias to guide their choice of filter and amount of filtering in practice.
| S-EPMC2949886 | biostudies-other
Project description:Emerging evidence suggests that contact tracing has had limited success in the UK in reducing the R number across the COVID-19 pandemic. We investigate potential pitfalls and areas for improvement by extending an existing branching process contact tracing model, adding diagnostic testing and refining parameter estimates. Our results demonstrate that reporting and adherence are the most important predictors of programme impact but tracing coverage and speed plus diagnostic sensitivity also play an important role. We conclude that well-implemented contact tracing could bring small but potentially important benefits to controlling and preventing outbreaks, providing up to a 15% reduction in R. We reaffirm that contact tracing is not currently appropriate as the sole control measure.
| S-EPMC8438018 | biostudies-literature
Project description:MotivationWhile classical approaches for controlling the false discovery rate (FDR) of RNA sequencing (RNAseq) experiments have been well described, modern research workflows and growing databases enable a new paradigm of controlling the FDR globally across RNAseq experiments in the past, present and future. The simplest analysis strategy that analyses each RNAseq experiment separately and applies an FDR correction method can lead to inflation of the overall FDR. We propose applying recently developed methodology for online multiple hypothesis testing to control the global FDR in a principled way across multiple RNAseq experiments.ResultsWe show that repeated application of classical repeated offline approaches has variable control of global FDR of RNAseq experiments over time. We demonstrate that the online FDR algorithms are a principled way to control FDR. Furthermore, in certain simulation scenarios, we observe empirically that online approaches have comparable power to repeated offline approaches.Availability and implementationThe onlineFDR package is freely available at http://www.bioconductor.org/packages/onlineFDR. Additional code used for the simulation studies can be found at https://github.com/latlio/onlinefdr_rnaseq_simulation.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC9805573 | biostudies-literature
Project description:Large databases (>10(6) sequences) used in metaproteomic and proteogenomic studies present challenges in matching peptide sequences to MS/MS data using database-search programs. Most notably, strict filtering to avoid false-positive matches leads to more false negatives, thus constraining the number of peptide matches. To address this challenge, we developed a two-step method wherein matches derived from a primary search against a large database were used to create a smaller subset database. The second search was performed against a target-decoy version of this subset database merged with a host database. High confidence peptide sequence matches were then used to infer protein identities. Applying our two-step method for both metaproteomic and proteogenomic analysis resulted in twice the number of high confidence peptide sequence matches in each case, as compared to the conventional one-step method. The two-step method captured almost all of the same peptides matched by the one-step method, with a majority of the additional matches being false negatives from the one-step method. Furthermore, the two-step method improved results regardless of the database search program used. Our results show that our two-step method maximizes the peptide matching sensitivity for applications requiring large databases, especially valuable for proteogenomics and metaproteomics studies.
| S-EPMC3633484 | biostudies-literature
Project description:Rapidly improving methods for glycoproteomics have enabled increasingly large-scale analyses of complex glycopeptide samples, but annotating the resulting mass spectrometry data with high confidence remains a major bottleneck. We recently introduced a fast and sensitive glycoproteomics search method in our MSFragger search engine, which reports glycopeptides as a combination of a peptide sequence and the mass of the attached glycan. In samples with complex glycosylation patterns, converting this mass to a specific glycan composition is not straightforward; however, as many glycans have similar or identical masses. Here, we have developed a new method for determining the glycan composition of N-linked glycopeptides fragmented by collisional or hybrid activation that uses multiple sources of information from the spectrum, including observed glycan B-type (oxonium) and Y-type ions and mass and precursor monoisotopic selection errors to discriminate between possible glycan candidates. Combined with false discovery rate estimation for the glycan assignment, we show that this method is capable of specifically and sensitively identifying glycans in complex glycopeptide analyses and effectively controls the rate of false glycan assignments. The new method has been incorporated into the PTM-Shepherd modification analysis tool to work directly with the MSFragger glyco search in the FragPipe graphical user interface, providing a complete computational pipeline for annotation of N-glycopeptide spectra with false discovery rate control of both peptide and glycan components that is both sensitive and robust against false identifications.
| S-EPMC8933705 | biostudies-literature
Project description:Motivation:The biological pathways linking exposures and disease risk are often poorly understood. To gain insight into these pathways, studies may try to identify biomarkers that mediate the exposure/disease relationship. Such studies often simultaneously test hundreds or thousands of biomarkers. Results:We consider a set of m biomarkers and a corresponding set of null hypotheses, where the jth null hypothesis states that biomarker j does not mediate the exposure/disease relationship. We propose a Multiple Comparison Procedure (MCP) that rejects a set of null hypotheses or, equivalently, identifies a set of mediators, while asymptotically controlling the Family-Wise Error Rate (FWER) or False Discovery Rate (FDR). We use simulations to show that, compared to currently available methods, our proposed method has higher statistical power to detect true mediators. We then apply our method to a breast cancer study and identify nine metabolites that may mediate the known relationship between an increased BMI and an increased risk of breast cancer. Availability and implementation:R package MultiMed on https://github.com/SiminaB/MultiMed. Supplementary information:Supplementary data are available at Bioinformatics online.
| S-EPMC6355102 | biostudies-literature
Project description:Gene set enrichment analysis for analyzing large profiling and screening experiments can reveal unifying biological schemes based on previously accumulated knowledge represented as "gene sets". Most of the existing implementations use a fixed fold-change or P value cutoff to generate regulated gene lists. However, the threshold selection in most cases is arbitrary, and has a significant effect on the test outcome and interpretation of the experiment. We developed a new gene set enrichment analysis method, ie, FDR-FET, which dynamically optimizes the threshold choice and improves the sensitivity and selectivity of gene set enrichment analysis. The procedure translates experimental results into a series of regulated gene lists at multiple false discovery rate (FDR) cutoffs, and computes the P value of the overrepresentation of a gene set using a Fisher's exact test (FET) in each of these gene lists. The lowest P value is retained to represent the significance of the gene set. We also implemented improved methods to define a more relevant global reference set for the FET. We demonstrate the validity of the method using a published microarray study of three protease inhibitors of the human immunodeficiency virus and compare the results with those from other popular gene set enrichment analysis algorithms. Our results show that combining FDR with multiple cutoffs allows us to control the error while retaining genes that increase information content. We conclude that FDR-FET can selectively identify significant affected biological processes. Our method can be used for any user-generated gene list in the area of transcriptome, proteome, and other biological and scientific applications.
| S-EPMC3169954 | biostudies-literature