Dataset Information

QVALITY: non-parametric estimation of q-values and posterior error probabilities.

ABSTRACT: Qvality is a C++ program for estimating two types of standard statistical confidence measures: the q-value, which is an analog of the p-value that incorporates multiple testing correction, and the posterior error probability (PEP, also known as the local false discovery rate), which corresponds to the probability that a given observation is drawn from the null distribution. In computing q-values, qvality employs a standard bootstrap procedure to estimate the prior probability of a score being from the null distribution; for PEP estimation, qvality relies upon non-parametric logistic regression. Relative to other tools for estimating statistical confidence measures, qvality is unique in its ability to estimate both types of scores directly from a null distribution, without requiring the user to calculate p-values.A web server, C++ source code and binaries are available under MIT license at http://noble.gs.washington.edu/proj/qvality.Supplementary data are available at Bioinformatics online.

SUBMITTER: Kall L

PROVIDER: S-EPMC2660870 | biostudies-other | 2009 Apr

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

QVALITY: non-parametric estimation of q-values and posterior error probabilities.

Käll Lukas L Storey John D JD Noble William Stafford WS

Bioinformatics (Oxford, England) 20090204 7

<h4>Unlabelled</h4>Qvality is a C++ program for estimating two types of standard statistical confidence measures: the q-value, which is an analog of the p-value that incorporates multiple testing correction, and the posterior error probability (PEP, also known as the local false discovery rate), which corresponds to the probability that a given observation is drawn from the null distribution. In computing q-values, qvality employs a standard bootstrap procedure to estimate the prior probability ...[more]

PMID: 19193729

Similar Datasets

Project description:BACKGROUND:In shotgun proteomics, database searching of tandem mass spectra results in a great number of peptide-spectrum matches (PSMs), many of which are false positives. Quality control of PSMs is a multiple hypothesis testing problem, and the false discovery rate (FDR) or the posterior error probability (PEP) is the commonly used statistical confidence measure. PEP, also called local FDR, can evaluate the confidence of individual PSMs and thus is more desirable than FDR, which evaluates the global confidence of a collection of PSMs. Estimation of PEP can be achieved by decomposing the null and alternative distributions of PSM scores as long as the given data is sufficient. However, in many proteomic studies, only a group (subset) of PSMs, e.g. those with specific post-translational modifications, are of interest. The group can be very small, making the direct PEP estimation by the group data inaccurate, especially for the high-score area where the score threshold is taken. Using the whole set of PSMs to estimate the group PEP is inappropriate either, because the null and/or alternative distributions of the group can be very different from those of combined scores. RESULTS:The transfer PEP algorithm is proposed to more accurately estimate the PEPs of peptide identifications in small groups. Transfer PEP derives the group null distribution through its empirical relationship with the combined null distribution, and estimates the group alternative distribution, as well as the null proportion, using an iterative semi-parametric method. Validated on both simulated data and real proteomic data, transfer PEP showed remarkably higher accuracy than the direct combined and separate PEP estimation methods. CONCLUSIONS:We presented a novel approach to group PEP estimation for small groups and implemented it for the peptide identification problem in proteomics. The methodology of the approach is in principle applicable to the small-group PEP estimation problems in other fields.

Project description:BackgroundEnsuring fair comparisons of cancer survival statistics across population groups requires careful consideration of differential competing mortality due to other causes, and adjusting for imbalances over groups in other prognostic covariates (e.g. age). This has typically been achieved using comparisons of age-standardised net survival, with age standardisation addressing covariate imbalance, and the net estimates removing differences in competing mortality from other causes. However, these estimates lack ease of interpretability. In this paper, we motivate an alternative non-parametric approach that uses a common rate of other cause mortality across groups to give reference-adjusted estimates of the all-cause and cause-specific crude probability of death in contrast to solely reporting net survival estimates.MethodsWe develop the methodology for a non-parametric equivalent of standardised and reference adjusted crude probabilities of death, building on the estimation of non-parametric crude probabilities of death. We illustrate the approach using regional comparisons of survival following a diagnosis of rectal cancer for men in England. We standardise to the covariate distribution and other cause mortality of England as a whole to offer comparability, but with close approximation to the observed all-cause region-specific mortality.ResultsThe approach gives comparable estimates to observed crude probabilities of death, but allows direct comparison across population groups with different covariate profiles and competing mortality patterns. In our illustrative example, we show that regional variations in survival following a diagnosis of rectal cancer persist even after accounting for the variation in deprivation, age at diagnosis and other cause mortality.ConclusionsThe methodological approach of using standardised and reference adjusted metrics offers an appealing approach for future cancer survival comparison studies and routinely published cancer statistics. Our non-parametric estimation approach through the use of weighting offers the ability to estimate comparable survival estimates without the need for statistical modelling.

Project description:BackgroundIn investigating differentially expressed genes or other selected features, researchers conduct hypothesis tests to determine which biological categories, such as those of the Gene Ontology (GO), are enriched for the selected features. Multiple comparison procedures (MCPs) are commonly used to prevent excessive false positive rates. Traditional MCPs, e.g., the Bonferroni method, go to the opposite extreme: strictly controlling a family-wise error rate, resulting in excessive false negative rates. Researchers generally prefer the more balanced approach of instead controlling the false discovery rate (FDR). However, the q-values that methods of FDR control assign to biological categories tend to be too low to reliably estimate the probability that a biological category is not enriched for the preselected features. Thus, we study an application of the other estimators of that probability, which is called the local FDR (LFDR).ResultsWe considered five LFDR estimators for detecting enriched GO terms: a binomial-based estimator (BBE), a maximum likelihood estimator (MLE), a normalized MLE (NMLE), a histogram-based estimator assuming a theoretical null hypothesis (HBE), and a histogram-based estimator assuming an empirical null hypothesis (HBE-EN). Since NMLE depends not only on the data but also on the specified value of π0, the proportion of non-enriched GO terms, it is only advantageous when either π0 is already known with sufficient accuracy or there are data for only 1 GO term. By contrast, the other estimators work without specifying π0 but require data for at least 2 GO terms. Our simulation studies yielded the following summaries of the relative performance of each of those four estimators. HBE and HBE-EN produced larger biases for 2, 4, 8, 32, and 100 GO terms than BBE and MLE. BBE has the lowest bias if π0 is 1 and if the number of GO terms is between 2 and 32. The bias of MLE is no worse than that of BBE for 100 GO terms even when the ideal number of components in its underlying mixture model is unknown, but has high bias when the number of GO terms is small compared to the number of estimated parameters. For unknown values of π0, BBE has the lowest bias for a small number of GO terms (2-32 GO terms), and MLE has the lowest bias for a medium number of GO terms (100 GO terms).ConclusionsFor enrichment detection, we recommend estimating the LFDR by MLE given at least a medium number of GO terms, by BBE given a small number of GO terms, and by NMLE given either only 1 GO term or precise knowledge of π0.

Dataset Information

QVALITY: non-parametric estimation of q-values and posterior error probabilities.

Publications

QVALITY: non-parametric estimation of q-values and posterior error probabilities.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets