Dataset Information

Temporal bias in case-control design: preventing reliable predictions of the future.

ABSTRACT: One of the primary tools that researchers use to predict risk is the case-control study. We identify a flaw, temporal bias, that is specific to and uniquely associated with these studies that occurs when the study period is not representative of the data that clinicians have during the diagnostic process. Temporal bias acts to undermine the validity of predictions by over-emphasizing features close to the outcome of interest. We examine the impact of temporal bias across the medical literature, and highlight examples of exaggerated effect sizes, false-negative predictions, and replication failure. Given the ubiquity and practical advantages of case-control studies, we discuss strategies for estimating the influence of and preventing temporal bias where it exists.

SUBMITTER: Yuan W

PROVIDER: S-EPMC7889612 | biostudies-literature | 2021 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Temporal bias in case-control design: preventing reliable predictions of the future.

Yuan William W Beaulieu-Jones Brett K BK Yu Kun-Hsing KH Lipnick Scott L SL Palmer Nathan N Loscalzo Joseph J Cai Tianxi T Kohane Isaac S IS

Nature communications 20210217 1

One of the primary tools that researchers use to predict risk is the case-control study. We identify a flaw, temporal bias, that is specific to and uniquely associated with these studies that occurs when the study period is not representative of the data that clinicians have during the diagnostic process. Temporal bias acts to undermine the validity of predictions by over-emphasizing features close to the outcome of interest. We examine the impact of temporal bias across the medical literature, ...[more]

PMID: 33597541

Similar Datasets

Project description:BackgroundCross-validation tools are used increasingly to validate and compare genetic evaluation methods but analytical properties of cross-validation methods are rarely described. There is also a lack of cross-validation tools for complex problems such as prediction of indirect effects (e.g. maternal effects) or for breeding schemes with small progeny group sizes.ResultsWe derive the expected value of several quadratic forms by comparing genetic evaluations including "partial" and "whole" data. We propose statistics that compare genetic evaluations including "partial" and "whole" data based on differences in means, covariance, and correlation, and term the use of these statistics "method LR" (from linear regression). Contrary to common belief, the regression of true on estimated breeding values is (on expectation) lower than 1 for small or related validation sets, due to family structures. For validation sets that are sufficiently large, we show that these statistics yield estimators of bias, slope or dispersion, and population accuracy for estimated breeding values. Similar results hold for prediction of future phenotypes although we show that estimates of bias, slope or dispersion using prediction of future phenotypes are sensitive to incorrect heritabilities or precorrection for fixed effects. We present an example for a set of 2111 Brahman beef cattle for which, in repeated partitioning of the data into training and validation sets, there is very good agreement of statistics of method LR with prediction of future phenotypes.ConclusionsAnalytical properties of cross-validation measures are presented. We present a new method named LR for cross-validation that is automatic, easy to use, and which yields the quantities of interest. The method compares predictions based on partial and whole data, which results in estimates of accuracy and biases. Prediction of observed records may yield biased results due to precorrection or use of incorrect heritabilities.

Dataset Information

Temporal bias in case-control design: preventing reliable predictions of the future.

Publications

Temporal bias in case-control design: preventing reliable predictions of the future.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure