Evaluating the predictiveness of a continuous marker.
Ontology highlight
ABSTRACT:
SUBMITTER: Huang Y
PROVIDER: S-EPMC3059154 | biostudies-literature | 2007 Dec
REPOSITORIES: biostudies-literature
Publications
Evaluating the predictiveness of a continuous marker.
Huang Ying Y Sullivan Pepe Margaret M Feng Ziding Z
Biometrics 20070508 4
Consider a continuous marker for predicting a binary outcome. For example, the serum concentration of prostate specific antigen may be used to calculate the risk of finding prostate cancer in a biopsy. In this article, we argue that the predictive capacity of a marker has to do with the population distribution of risk given the marker and suggest a graphical tool, the predictiveness curve, that displays this distribution. The display provides a common meaningful scale for comparing markers that ...[more]
Similar Datasets
Project description:A potential venue to improve healthcare efficiency is to effectively tailor individualized treatment strategies by incorporating patient level predictor information such as environmental exposure, biological, and genetic marker measurements. Many useful statistical methods for deriving individualized treatment rules (ITR) have become available in recent years. Prior to adopting any ITR in clinical practice, it is crucial to evaluate its value in improving patient outcomes. Existing methods for quantifying such values mainly consider either a single marker or semi-parametric methods that are subject to bias under model misspecification. In this article, we consider a general setting with multiple markers and propose a two-step robust method to derive ITRs and evaluate their values. We also propose procedures for comparing different ITRs, which can be used to quantify the incremental value of new markers in improving treatment selection. While working models are used in step I to approximate optimal ITRs, we add a layer of calibration to guard against model misspecification and further assess the value of the ITR non-parametrically, which ensures the validity of the inference. To account for the sampling variability of the estimated rules and their corresponding values, we propose a resampling procedure to provide valid confidence intervals for the value functions as well as for the incremental value of new markers for treatment selection. Our proposals are examined through extensive simulation studies and illustrated with the data from a clinical trial that studies the effects of two drug combinations on HIV-1 infected patients.
| S-EPMC4213325 | biostudies-literature
Project description:In the evaluation of a biomarker for risk prediction, one can assess the performance of the biomarker in the population of interest by displaying the predictiveness curve. In conjunction with an assessment of the classification accuracy of a biomarker, the predictiveness curve is an important tool for assessing the usefulness of a risk prediction model. Inference for a single biomarker or for multiple biomarkers can be performed using summary measures of the predictiveness curve. We propose two partial summary measures, the partial total gain and the partial proportion of explained variation, that summarize the predictiveness curve over a restricted range of risk. The methods we describe can be used to compare two biomarkers when there are existing thresholds for risk stratification. We describe inferential tools for one and two samples that are shown to have adequate power in a simulation study. The methods are illustrated by assessing the accuracy of a risk score for predicting the onset of Alzheimer's disease.
| S-EPMC3806283 | biostudies-literature
Project description:Given the long follow-up periods that are often required for treatment or intervention studies, the potential to use surrogate markers to decrease the required follow-up time is a very attractive goal. However, previous studies have shown that using inadequate markers or making inappropriate assumptions about the relationship between the primary outcome and surrogate marker can lead to inaccurate conclusions regarding the treatment effect. Currently available methods for identifying and validating surrogate markers tend to rely on restrictive model assumptions and/or focus on uncensored outcomes. The ability to use such methods in practice when the primary outcome of interest is a time-to-event outcome is difficult because of censoring and missing surrogate information among those who experience the primary outcome before surrogate marker measurement. In this paper, we propose a novel definition of the proportion of treatment effect explained by surrogate information collected up to a specified time in the setting of a time-to-event primary outcome. Our proposed approach accommodates a setting where individuals may experience the primary outcome before the surrogate marker is measured. We propose a robust non-parametric procedure to estimate the defined quantity using censored data and use a perturbation-resampling procedure for variance estimation. Simulation studies demonstrate that the proposed procedures perform well in finite samples. We illustrate the proposed procedures by investigating two potential surrogate markers for diabetes using data from the Diabetes Prevention Program. Copyright © 2017 John Wiley & Sons, Ltd.
| S-EPMC5413393 | biostudies-literature
Project description:[Correction Notice: An Erratum for this article was reported online in Journal of Experimental Psychology: General on Jan 14 2021 (see record 2021-07705-001). In the article, formatting for UK Research Councils funding was omitted. The author note and copyright line now reflect the standard acknowledgment of and formatting for the funding received for this article. All versions of this article have been corrected.] Attention determines which cues receive processing and are learned about. Learning, however, leads to attentional biases. In the study of animal learning, in some circumstances, cues that have been previously predictive of their consequences are subsequently learned about more than are nonpredictive cues, suggesting that they receive more attention. In other circumstances, cues that have previously led to uncertain consequences are learned about more than are predictive cues. In human learning, there is a clear role for predictiveness, but a role for uncertainty has been less clear. Here, in a human learning task, we show that cues that led to uncertain outcomes were subsequently learned about more than were cues that were previously predictive of their outcomes. This effect occurred when there were few uncertain cues. When the number of uncertain cues was increased, attention switched to predictive cues. This pattern of results was found for cues (1) that were uncertain because they led to 2 different outcomes equally often in a nonpredictable manner and (2) that were used in a nonlinear discrimination and were not predictive individually but were predictive in combination with other cues. This suggests that both the opposing predictiveness and uncertainty effects were determined by the relationship between individual cues and outcomes rather than the predictive strength of combined cues. These results demonstrate that learning affects attention; however, the precise nature of the effect on attention depends on the level of task complexity, which reflects a potential switch between exploration and exploitation of cues. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
| S-EPMC8515774 | biostudies-literature
Project description:Correct specification of the exposure model is essential for unbiased estimation in marginal structural models with inverse-probability-of-treatment weights. However, although flexible modeling is commonplace when estimating effects of continuous covariates in outcome models, its use is less frequent in estimation of inverse probability weights. Using simulations, we assess the accuracy of the treatment effect estimates and covariate balance obtained with different exposure model specifications when the true relationship between a continuous, possibly time-varying covariate Lt and the logit of the probability of exposure is nonlinear. Specifically, we compare 4 approaches to modeling the effect of Lt when estimating inverse probability weights: a linear function, the covariate-balancing propensity score, and 2 easy-to-implement flexible methods that relax the assumption of linearity: cubic regression splines and fractional polynomials. Using data from 2 empirical studies, we compare linear exposure models with flexible exposure models to estimate the effect of sustained virological response to hepatitis C virus treatment on the progression of liver fibrosis. Our simulation results demonstrate that ignoring important nonlinear relationships when fitting the exposure model may provide poorer covariate balance and induce substantial bias in the estimated exposure-outcome associations. Analysts should routinely consider flexible modeling of continuous covariates when estimating inverse-probability-of-treatment weights.
| S-EPMC6545287 | biostudies-literature
Project description:In higher plants, the plane of cell division is faithfully predicted by the preprophase band (PPB). The PPB, a cortical ring of microtubules and F-actin, disassembles upon nuclear-envelope breakdown. During cytokinesis, the expanding cell plate fuses with the plasma membrane at the cortical division site, the site of the former PPB. The nature of the "molecular memory" that is left behind by the PPB and is proposed to guide the cell plate to the cortical division site is unknown. RanGAP is the GTPase activating protein of the small GTPase Ran, which provides spatial information for nucleocytoplasmic transport and various mitotic processes in animals. Here, we show that, in dividing root cells, Arabidopsis RanGAP1 concentrates at the PPB and remains associated with the cortical division site during mitosis and cytokinesis, requiring its N-terminal targeting domain. In a fass/ton2 mutant, which affects PPB formation, RanGAP1 recruitment to the PPB site is lost, while its PPB retention is microtubule-independent. RanGAP1 persistence at the cortical division site, but not its initial accumulation at the PPB requires the 2 cytokinesis-regulating kinesins POK1 and POK2. Depletion of RanGAP by inducible RNAi leads to oblique cell walls and cell-wall stubs in root cell files, consistent with cytokinesis defects. We propose that Arabidopsis RanGAP, a continuous positive protein marker of the plant division plane, has a role in spatial signaling during plant cell division.
| S-EPMC2587610 | biostudies-literature
Project description:The Lorenz curve is a graphical tool that is used widely in econometrics. It represents the spread of a probability distribution, and its traditional use has been to characterize population distributions of wealth or income, or more specifically, inequalities in wealth or income. However, its utility in public health research has not been broadly established. The purpose of this article is to explain its special usefulness for characterizing the population distribution of disease risks, and in particular for identifying the precise disease burden that can be predicted to occur in segments of the population that are known to have especially high (or low) risks, a feature that is important for evaluating the yield of screening or other disease prevention initiatives. We demonstrate that, although the Lorenz curve represents the distribution of predicted risks in a population at risk for the disease, in fact it can be estimated from a case-control study conducted in the population without the need for information on absolute risks. We explore two different estimation strategies and compare their statistical properties using simulations. The Lorenz curve is a statistical tool that deserves wider use in public health research.
| S-EPMC5495014 | biostudies-literature
Project description:This study investigated how humans process probabilistic-associated information when encountering varying levels of uncertainty during implicit visual statistical learning. A novel probabilistic cueing validation paradigm was developed to probe the representation of cues with high (75% probability), medium (50%), low (25%), or zero levels of predictiveness in response to preceding targets that appeared with high (75%), medium (50%), or low (25%) transitional probabilities (TPs). Experiments 1 and 2 demonstrated a significant negative association between cue probe identification accuracy and cue predictiveness when these cues appeared after high-TP but not medium-TP or low-TP targets, establishing exploration-like cue processing triggered by lower-uncertainty rather than high-uncertainty inputs. Experiment 3 ruled out the confounding factor of probe repetition and extended this finding by demonstrating (1) enhanced representation of low-predictive and zero-predictive but not high-predictive cues across blocks after high-TP targets and (2) enhanced representation of high-predictive but not low-predictive and zero-predictive cues across blocks after low-TP targets for learners who exhibited above-chance awareness of cue-target transition. These results suggest that during implicit statistical learning, input characteristics alter cue-processing mechanisms, such that exploration-like and exploitation-like mechanisms are triggered by lower-uncertainty and higher-uncertainty cue-target sequences, respectively.
| S-EPMC10631146 | biostudies-literature
Project description:ObjectiveThis study evaluated the accuracy and large inaccuracy of the Freestyle Navigator (FSN) (Abbott Diabetes Care, Alameda, CA) and Dexcom SEVEN PLUS (DSP) (Dexcom, Inc., San Diego, CA) continuous glucose monitoring (CGM) systems during closed-loop studies.Research design and methodsPaired CGM and plasma glucose values (7,182 data pairs) were collected, every 15-60?min, from 32 adults (36.2±9.3 years) and 20 adolescents (15.3±1.5 years) with type 1 diabetes who participated in closed-loop studies. Levels 1, 2, and 3 of large sensor error with increasing severity were defined according to absolute relative deviation greater than or equal to ±40%, ±50%, and ±60% at a reference glucose level of ?6 mmol/L or absolute deviation greater than or equal to ±2.4 mmol/L,±3.0 mmol/L, and ±3.6 mmol/L at a reference glucose level of <6 mmol/L.ResultsMedian absolute relative deviation was 9.9% for FSN and 12.6% for DSP. Proportions of data points in Zones A and B of Clarke error grid analysis were similar (96.4% for FSN vs. 97.8% for DSP). Large sensor over-reading, which increases risk of insulin over-delivery and hypoglycemia, occurred two- to threefold more frequently with DSP than FSN (once every 2.5, 4.6, and 10.7 days of FSN use vs. 1.2, 2.0, and 3.7 days of DSP use for Level 1-3 errors, respectively). At levels 2 and 3, large sensor errors lasting 1?h or longer were absent with FSN but persisted with DSP.ConclusionsFSN and DSP differ substantially in the frequency and duration of large inaccuracy despite only modest differences in conventional measures of numerical and clinical accuracy. Further evaluations are required to confirm that FSN is more suitable for integration into closed-loop delivery systems.
| S-EPMC3558677 | biostudies-literature
Project description:Genetic association studies using high-throughput genotyping and sequencing technologies have identified a large number of genetic variants associated with complex human diseases. These findings have provided an unprecedented opportunity to identify individuals in the population at high risk for disease who carry causal genetic mutations and hold great promise for early intervention and individualized medicine. While interest is high in building risk prediction models based on recent genetic findings, it is crucial to have appropriate statistical measurements to assess the performance of a genetic risk prediction model. Predictiveness curves were recently proposed as a graphic tool for evaluating a risk prediction model on the basis of a single continuous biomarker. The curve evaluates a risk prediction model for classification performance as well as its usefulness when applied to a population. In this article, we extend the predictiveness curve to measure the collective contribution of multiple genetic variants. We further propose a nonparametric, U-statistics-based measurement, referred to as the U-Index, to quantify the performance of a multi-locus predictiveness curve. In particular, a global U-Index and a partial U-Index can be used in the general population and a subpopulation of particular clinical interest, respectively. Through simulation studies, we demonstrate that the proposed U-Index has advantages over several existing summary statistics under various disease models. We also show that the partial U-Index can have its own uniqueness when rare variants have a substantial contribution to disease risk. Finally, we use the proposed predictiveness curve and its corresponding U-Index to evaluate the performance of a genetic risk prediction model for nicotine dependence.
| S-EPMC6612460 | biostudies-literature