Receiver Operating Characteristic Analysis and Clinical Trial Simulation to Inform Dose Titration Decisions.
Ontology highlight
ABSTRACT:
SUBMITTER: Clements JD
PROVIDER: S-EPMC6263661 | biostudies-literature | 2018 Nov
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Unlabelled: Lu and Elston have recently proposed a procedure for developing optimal receiver operating characteristic curves that maximize the area under a receiver operating characteristic curve in the setting of a predictive genetic test. The method requires only summary data, not individual level genetic data. In an era of increased data sharing, we investigate the performance of this algorithm when individual level genetic data are available and compare this approach to more standard receiver operating characteristic curve-building methods.ConclusionThough the Lu-Elston method can produce an optimal area under the curve under some assumptions, the method typically has little advantage over standard multivariable logistic methods when data are available. Also, the standard approach easily allows comparison of nested models via likelihood ratio tests and incorporation of covariates - the Lu-Elston approach is shown to have some difficulties with such analyses. These conclusions are based on evaluations using the Genetic Analysis Workshop 16 rheumatoid arthritis data set.
| S-EPMC2795956 | biostudies-literature
Project description:For censored survival outcomes, it can be of great interest to evaluate the predictive power of individual markers or their functions. Compared with alternative evaluation approaches, approaches based on the time-dependent receiver operating characteristics (ROC) rely on much weaker assumptions, can be more robust, and hence are preferred. In this article, we examine evaluation of markers' predictive power using the time-dependent ROC curve and a concordance measure that can be viewed as a weighted area under the time-dependent area under the ROC curve profile. This study significantly advances from existing time-dependent ROC studies by developing nonparametric estimators of the summary indexes and, more importantly, rigorously establishing their asymptotic properties. It reinforces the statistical foundation of the time-dependent ROC-based evaluation approaches for censored survival outcomes. Numerical studies, including simulations and application to an HIV clinical trial, demonstrate the satisfactory finite-sample performance of the proposed approaches.
| S-EPMC3743052 | biostudies-literature
Project description:Receiver operating characteristic analysis is widely used for evaluating diagnostic systems. Recent studies have shown that estimating an area under receiver operating characteristic curve with standard cross-validation methods suffers from a large bias. The leave-pair-out cross-validation has been shown to correct this bias. However, while leave-pair-out produces an almost unbiased estimate of area under receiver operating characteristic curve, it does not provide a ranking of the data needed for plotting and analyzing the receiver operating characteristic curve. In this study, we propose a new method called tournament leave-pair-out cross-validation. This method extends leave-pair-out by creating a tournament from pair comparisons to produce a ranking for the data. Tournament leave-pair-out preserves the advantage of leave-pair-out for estimating area under receiver operating characteristic curve, while it also allows performing receiver operating characteristic analyses. We have shown using both synthetic and real-world data that tournament leave-pair-out is as reliable as leave-pair-out for area under receiver operating characteristic curve estimation and confirmed the bias in leave-one-out cross-validation on low-dimensional data. As a case study on receiver operating characteristic analysis, we also evaluate how reliably sensitivity and specificity can be estimated from tournament leave-pair-out receiver operating characteristic curves.
| S-EPMC6745617 | biostudies-literature
Project description:The matched case-control design is frequently used in the study of complex disorders and can result in significant gains in efficiency, especially in the context of measuring biomarkers; however, risk prediction in this setting is not straightforward. We propose an inverse-probability weighting approach to estimate the predictive ability associated with a set of covariates. In particular, we propose an algorithm for estimating the summary index, area under the curve corresponding to the Receiver Operating Characteristic curve associated with a set of pre-defined covariates for predicting a binary outcome. By combining data from the parent cohort with that generated in a matched case control study, we describe methods for estimation of the population parameters of interest and the corresponding area under the curve. We evaluate the bias associated with the proposed methods in simulations by considering a range of parameter settings. We illustrate the methods in two data applications: (1) a prospective cohort study of cardiovascular disease in women, the Women's Health Study, and (2) a matched case-control study nested within the Nurses' Health Study aimed at risk prediction of invasive breast cancer.
| S-EPMC6768691 | biostudies-literature
Project description:Convergence insufficiency (CI) is a dysfunction of binocular vision that is associated with various signs and symptoms in near work. However, CI screening is performed less frequently in adults than in children. We aimed to evaluate the ability of screening tests to discriminate CI from other binocular vision anomalies and normal binocular vision in young adults. One hundred eighty-four university students (age, 18-28 years) who underwent an eye examination due to ocular discomfort were included. Near point of convergence (NPC), phoria, accommodative amplitude, fusional vergence, the ratio of accommodative convergence to accommodation, relative accommodation, binocular accommodative facility, vergence facility, and the values corresponding to Sheard's and Percival's criteria were evaluated. Receiver operating characteristic (ROC) curve analysis for each test was also performed. The prevalence of CI ranged from 10.3% to 21.2%, depending on the signs and the presence of CI associated with accommodative disorders. Assessments based on NPC, Sheard's criterion, and Percival's criterion showed high discriminative ability, with the ability being higher between the CI and normal binocular vision groups than between the CI and non-CI groups. Sheard's criterion showed the highest diagnostic performance in discriminating CI with three signs from the non-CI group. The cut-off values were 7.2 cm for NPC, -0.23 to 1.00 for Sheard's criterion, and -4.00 to -2.33 for Percival's criterion. Our results suggest that the use of Sheard's criterion with NPC shows high performance for screening of CI.
| S-EPMC6980576 | biostudies-literature
Project description:Current ongoing genome-wide association (GWA) studies represent a powerful approach to uncover common unknown genetic variants causing common complex diseases. The discovery of these genetic variants offers an important opportunity for early disease prediction, prevention, and individualized treatment. We describe here a method of combining multiple genetic variants for early disease prediction, based on the optimality theory of the likelihood ratio (LR). Such theory simply shows that the receiver operating characteristic (ROC) curve based on the LR has maximum performance at each cutoff point and that the area under the ROC curve so obtained is highest among that of all approaches. Through simulations and a real data application, we compared it with the commonly used logistic regression and classification tree approaches. The three approaches show similar performance if we know the underlying disease model. However, for most common diseases we have little prior knowledge of the disease model and in this situation the new method has an advantage over logistic regression and classification tree approaches. We applied the new method to the type 1 diabetes GWA data from the Wellcome Trust Case Control Consortium. Based on five single nucleotide polymorphisms, the test reaches medium level classification accuracy. With more genetic findings to be discovered in the future, we believe a predictive genetic test for type 1 diabetes can be successfully constructed and eventually implemented for clinical use.
| S-EPMC3039874 | biostudies-literature
Project description:Receiver operating characteristic analysis provides an important methodology for assessing traditional (e.g., imaging technologies and clinical practices) and new (e.g., genomic studies, biomarker development) diagnostic problems. The area under the clinically/practically relevant part of the receiver operating characteristic curve (partial area or partial area under the receiver operating characteristic curve) is an important performance index summarizing diagnostic accuracy at multiple operating points (decision thresholds) that are relevant to actual clinical practice. A robust estimate of the partial area under the receiver operating characteristic curve is provided by the area under the corresponding part of the empirical receiver operating characteristic curve. We derive a closed-form expression for the jackknife variance of the partial area under the empirical receiver operating characteristic curve. Using the derived analytical expression, we investigate the differences between the jackknife variance and a conventional variance estimator. The relative properties in finite samples are demonstrated in a simulation study. The developed formula enables an easy way to estimate the variance of the empirical partial area under the receiver operating characteristic curve, thereby substantially reducing the computation burden, and provides important insight into the structure of the variability. We demonstrate that when compared with the conventional approach, the jackknife variance has substantially smaller bias, and leads to a more appropriate type I error rate of the Wald-type test. The use of the jackknife variance is illustrated in the analysis of a data set from a diagnostic imaging study.
| S-EPMC5133195 | biostudies-literature
Project description:BackgroundDisease prevalence is rarely explicitly considered in the early stages of the development of novel prognostic tests. Rather, researchers use the area under the receiver operating characteristic (AUROC) as the key metric to gauge and report predictive performance ability. Because this statistic does not account for disease prevalence, proposed tests may not appropriately address clinical requirements. This ultimately impedes the translation of prognostic tests into clinical practice.MethodsA method to express positive- and/or negative predictive value criteria (PPV, NPV) within the ROC space is presented. Equations are derived for so-called equi-PPV (and equi-NPV) lines. Herewith it is possible, for any given prevalence, to plot a series of sensitivity-specificity pairs which meet a specified PPV (or NPV) criterion onto the ROC space.This concept is introduced by firstly reviewing the well-established "mechanics", strengths and limitations of the ROC analysis in the context of developing prognostic models. Then, the use of PPV (and/or) NPV criteria to augment the ROC analysis is elaborated.Additionally, an interactive web tool was also created to enable people to explore the dynamics of lines of equi-predictive value in function of prevalence. The web tool also allows to gauge what ROC curve shapes best meet specific positive and/or negative predictive value criteria (http://d4ta.link/ppvnpv/).ResultsTo illustrate the merits and implications of this concept, an example on the prediction of pre-eclampsia risk in low-risk nulliparous pregnancies is elaborated.ConclusionsIn risk stratification, the clinical usefulness of a prognostic test can be expressed in positive- and negative predictive value criteria; the development of novel prognostic tests will be facilitated by the possibility to co-visualise such criteria together with ROC curves. To achieve clinically meaningful risk stratification, the development of separate tests to meet either a pre-specified positive value (rule-in) or a negative predictive value (rule-out) criteria should be considered: the characteristics of successful rule-in and rule-out tests may markedly differ.
| S-EPMC6460848 | biostudies-literature
Project description:This study assessed the ability to distinguish between type 1 diabetes-affected individuals and their unaffected relatives using HLA and single nucleotide polymorphism (SNP) genotypes.Eight models, ranging from only the high-risk DR3/DR4 genotype to all significantly associated HLA genotypes and two SNPs mapping to the cytotoxic T-cell-associated antigen-4 gene (CTLA4) and insulin (INS) genes, were fitted to high-resolution class I and class II HLA genotyping data for patients from the Type 1 Diabetes Genetics Consortium collection. Pairs of affected individuals and their unaffected siblings were divided into a "discovery" (n = 1,015 pairs) and a "validation" set (n = 318 pairs). The discriminating performance of various combinations of genetic information was estimated using receiver operating characteristic (ROC) curve analysis.The use of only the presence or absence of the high-risk DR3/DR4 genotype achieved very modest discriminating ability, yielding an area under the curve (AUC) of 0.62 in the discovery set and 0.59 in the validation set. The full model-which included HLA information from the class II loci DPB1, DRB1, and DQB1; selected alleles from HLA class I loci A and B; and SNPs from the CTLA4 and INS genes-increased the AUC to 0.74 in the discovery set and to 0.71 in the validation set. A cost-effective alternative is proposed, using genotype information equivalent to typing four SNPs (DR3, DR4-DQB1*03:02, CTLA-4, and INS), which achieved an AUC of 0.72 in the discovery set and 0.69 in the validation set.Genotyping data sufficient to tag DR3, DR4-DQB1*03:02, CTLA4, and INS were shown to distinguish between subjects with type 1 diabetes and their unaffected siblings adequately to achieve clinically utility to identify children in multiplex families to be considered for early intervention.
| S-EPMC3747897 | biostudies-literature
Project description:Assess patient-level utility of suggested pretreatment biomarkers of sunitinib in advanced renal cell carcinoma.Kaplan-Meier analysis of data from a randomized, Phase II study (n = 292) suggested baseline predictive value for circulating soluble Ang-2 and MMP-2 and HIF-1? percentage of tumor expression. Using this dataset, the sensitivity, specificity and area under the curve (AUC) were calculated, using receiver operating characteristic (ROC) curves.Based on a ROC (sensitivity vs 1 - specificity) threshold AUC value of >0.8, neither Ang-2 (0.67) nor MMP-2 (0.65), nor HIF-1? percentage of tumor expression (0.65), performed appropriately from a patient-selection standpoint.To properly assess potential biomarkers, sensitivity and specificity characteristics should be obtained by ROC analysis.
| S-EPMC5549778 | biostudies-literature