Dataset Information

Valection: design optimization for validation and verification studies.

ABSTRACT:

Background

Platform-specific error profiles necessitate confirmatory studies where predictions made on data generated using one technology are additionally verified by processing the same samples on an orthogonal technology. However, verifying all predictions can be costly and redundant, and testing a subset of findings is often used to estimate the true error profile.

Results

To determine how to create subsets of predictions for validation that maximize accuracy of global error profile inference, we developed Valection, a software program that implements multiple strategies for the selection of verification candidates. We evaluated these selection strategies on one simulated and two experimental datasets.

Conclusions

Valection is implemented in multiple programming languages, available at: http://labs.oicr.on.ca/boutros-lab/software/valection.

SUBMITTER: Cooper CI

PROVIDER: S-EPMC6157051 | biostudies-literature | 2018 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Valection: design optimization for validation and verification studies.

Cooper Christopher I CI Yao Delia D Sendorek Dorota H DH Yamaguchi Takafumi N TN P'ng Christine C Houlahan Kathleen E KE Caloian Cristian C Fraser Michael M Ellrott Kyle K Margolin Adam A AA Bristow Robert G RG Stuart Joshua M JM Boutros Paul C PC

BMC bioinformatics 20180925 1

<h4>Background</h4>Platform-specific error profiles necessitate confirmatory studies where predictions made on data generated using one technology are additionally verified by processing the same samples on an orthogonal technology. However, verifying all predictions can be costly and redundant, and testing a subset of findings is often used to estimate the true error profile.<h4>Results</h4>To determine how to create subsets of predictions for validation that maximize accuracy of global error p ...[more]

PMID: 30253747

Similar Datasets

Project description:BackgroundPopulation-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a "gold standard" measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper.MethodsThe general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions.ResultsIndicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are "don't know" or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported.ConclusionsThese methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.

Dataset Information

Valection: design optimization for validation and verification studies.

Background

Results

Conclusions

Publications

Valection: design optimization for validation and verification studies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets