A serial risk score approach to disease classification that accounts for accuracy and cost.
Ontology highlight
ABSTRACT:
SUBMITTER: Huynh D
PROVIDER: S-EPMC4790436 | biostudies-literature | 2014 Dec
REPOSITORIES: biostudies-literature
Publications
A serial risk score approach to disease classification that accounts for accuracy and cost.
Huynh Dat D Laeyendecker Oliver O Brookmeyer Ron R
Biometrics 20140825 4
The performance of diagnostic tests for disease classification is often measured by accuracy (e.g., sensitivity or specificity); however, costs of the diagnostic test are a concern as well. Combinations of multiple diagnostic tests may improve accuracy, but incur additional costs. Here, we consider serial testing approaches that maintain accuracy while controlling costs of the diagnostic tests. We present a serial risk score classification approach. The basic idea is to sequentially test with ad ...[more]
Similar Datasets
Project description:BackgroundThe value of polygenic risk scores (PRSs) towards improving guideline-recommended clinical risk models for coronary artery disease (CAD) prediction is controversial. Here we examine whether an integrated polygenic risk score improves the prediction of CAD beyond pooled cohort equations. METHODS: An observation study of 291,305 unrelated White British UK Biobank participants enrolled from 2006 to 2010 was conducted. A case-control sample of 9499 prevalent CAD cases and an equal number of randomly selected controls was used for tuning and integrating of the polygenic risk scores. A separate cohort of 272,307 individuals (with follow-up to 2020) was used to examine the risk prediction performance of pooled cohort equations, integrated polygenic risk score, and PRS-enhanced pooled cohort equation for incident CAD cases. The performance of each model was analyzed by discrimination and risk reclassification using a 7.5% threshold.ResultsIn the cohort of 272,307 individuals (mean age, 56.7 years) used to analyze predictive accuracy, there were 7036 incident CAD cases over a 12-year follow-up period. Model discrimination was tested for integrated polygenic risk score, pooled cohort equation, and PRS-enhanced pooled cohort equation with reported C-statistics of 0.640 (95% CI, 0.634-0.646), 0.718 (95% CI, 0.713-0.723), and 0.753 (95% CI, 0.748-0.758), respectively. Risk reclassification for the addition of the integrated polygenic risk score to the pooled cohort equation at a 7.5% risk threshold resulted in a net reclassification improvement of 0.117 (95% CI, 0.102 to 0.129) for cases and - 0.023 (95% CI, - 0.025 to - 0.022) for noncases [overall: 0.093 (95% CI, 0.08 to 0.104)]. For incident CAD cases, this represented 14.2% correctly reclassified to the higher-risk category and 2.6% incorrectly reclassified to the lower-risk category.ConclusionsAddition of the integrated polygenic risk score for CAD to the pooled cohort questions improves the predictive accuracy for incident CAD and clinical risk classification in the White British from the UK Biobank. These findings suggest that an integrated polygenic risk score may enhance CAD risk prediction and screening in the White British population.
| S-EPMC9639312 | biostudies-literature
Project description:ImportancePolygenic risk scores comprising millions of single-nucleotide polymorphisms (SNPs) could be useful for population-wide coronary heart disease (CHD) screening.ObjectiveTo determine whether a polygenic risk score improves prediction of CHD compared with a guideline-recommended clinical risk equation.Design, setting, and participantsA retrospective cohort study of the predictive accuracy of a previously validated polygenic risk score was assessed among 4847 adults of white European ancestry, aged 45 through 79 years, participating in the Atherosclerosis Risk in Communities (ARIC) study and 2390 participating in the Multi-Ethnic Study of Atherosclerosis (MESA) from 1996 through December 31, 2015, the final day of follow-up. The performance of the polygenic risk score was compared with that of the 2013 American College of Cardiology and American Heart Association pooled cohort equations.ExposuresGenetic risk was computed for each participant by summing the product of the weights and allele dosage across 6 630 149 SNPs. Weights were based on an international genome-wide association study.Main outcomes and measuresPrediction of 10-year first CHD events (including myocardial infarctions, fatal coronary events, silent infarctions, revascularization procedures, or resuscitated cardiac arrest) assessed using measures of model discrimination, calibration, and net reclassification improvement (NRI).ResultsThe study population included 4847 adults from the ARIC study (mean [SD] age, 62.9 [5.6] years; 56.4% women) and 2390 adults from the MESA cohort (mean [SD] age, 61.8 [9.6] years; 52.2% women). Incident CHD events occurred in 696 participants (14.4%) and 227 participants (9.5%), respectively, over median follow-up of 15.5 years (interquartile range [IQR], 6.3 years) and 14.2 (IQR, 2.5 years) years. The polygenic risk score was significantly associated with 10-year CHD incidence in ARIC with hazard ratios per SD increment of 1.24 (95% CI, 1.15 to 1.34) and in MESA, 1.38 (95% CI, 1.21 to 1.58). Addition of the polygenic risk score to the pooled cohort equations did not significantly increase the C statistic in either cohort (ARIC, change in C statistic, -0.001; 95% CI, -0.009 to 0.006; MESA, 0.021; 95% CI, -0.0004 to 0.043). At the 10-year risk threshold of 7.5%, the addition of the polygenic risk score to the pooled cohort equations did not provide significant improvement in reclassification in either ARIC (NRI, 0.018, 95% CI, -0.012 to 0.036) or MESA (NRI, 0.001, 95% CI, -0.038 to 0.076). The polygenic risk score did not significantly improve calibration in either cohort.Conclusions and relevanceIn this analysis of 2 cohorts of US adults, the polygenic risk score was associated with incident coronary heart disease events but did not significantly improve discrimination, calibration, or risk reclassification compared with conventional predictors. These findings suggest that a polygenic risk score may not enhance risk prediction in a general, white middle-aged population.
| S-EPMC7042849 | biostudies-literature
Project description:ImportanceThe incremental value of polygenic risk scores in addition to well-established risk prediction models for coronary artery disease (CAD) is uncertain.ObjectiveTo examine whether a polygenic risk score for CAD improves risk prediction beyond pooled cohort equations.Design, setting, and participantsObservational study of UK Biobank participants enrolled from 2006 to 2010. A case-control sample of 15 947 prevalent CAD cases and equal number of age and sex frequency-matched controls was used to optimize the predictive performance of a polygenic risk score for CAD based on summary statistics from published genome-wide association studies. A separate cohort of 352 660 individuals (with follow-up to 2017) was used to evaluate the predictive accuracy of the polygenic risk score, pooled cohort equations, and both combined for incident CAD.ExposuresPolygenic risk score for CAD, pooled cohort equations, and both combined.Main outcomes and measuresCAD (myocardial infarction and its related sequelae). Discrimination, calibration, and reclassification using a risk threshold of 7.5% were assessed.ResultsIn the cohort of 352 660 participants (mean age, 55.9 years; 205 297 women [58.2%]) used to evaluate the predictive accuracy of the examined models, there were 6272 incident CAD events over a median of 8 years of follow-up. CAD discrimination for polygenic risk score, pooled cohort equations, and both combined resulted in C statistics of 0.61 (95% CI, 0.60 to 0.62), 0.76 (95% CI, 0.75 to 0.77), and 0.78 (95% CI, 0.77 to 0.79), respectively. The change in C statistic between the latter 2 models was 0.02 (95% CI, 0.01 to 0.03). Calibration of the models showed overestimation of risk by pooled cohort equations, which was corrected after recalibration. Using a risk threshold of 7.5%, addition of the polygenic risk score to pooled cohort equations resulted in a net reclassification improvement of 4.4% (95% CI, 3.5% to 5.3%) for cases and -0.4% (95% CI, -0.5% to -0.4%) for noncases (overall net reclassification improvement, 4.0% [95% CI, 3.1% to 4.9%]).Conclusions and relevanceThe addition of a polygenic risk score for CAD to pooled cohort equations was associated with a statistically significant, yet modest, improvement in the predictive accuracy for incident CAD and improved risk stratification for only a small proportion of individuals. The use of genetic information over the pooled cohort equations model warrants further investigation before clinical implementation.
| S-EPMC7042853 | biostudies-literature
Project description:Many community-based translations of evidence-based interventions are designed as one-arm studies due to ethical and other considerations. Evaluating the impacts of such programs is challenging. Here, we examine the effectiveness of the lifestyle intervention implemented by the Special Diabetes Program for Indians Diabetes Prevention (SDPI-DP) demonstration project, a translational lifestyle intervention among American Indian and Alaska Native communities. Data from the landmark Diabetes Prevention Program placebo group was used as a historical control. We compared the use of propensity score (PS) and disease risk score (DRS) matching to adjust for potential confounder imbalance between groups. The unadjusted hazard ratio (HR) for diabetes risk was 0.35 for SDPI-DP lifestyle intervention vs. control. However, when relevant diabetes risk factors were considered, the adjusted HR estimates were attenuated toward 1, ranging from 0.56 (95% CI 0.44-0.71) to 0.69 (95% CI 0.56-0.96). The differences in estimated HRs using the PS and DRS approaches were relatively small but DRS matching resulted in more participants being matched and smaller standard errors of effect estimates. Carefully employed, publicly available randomized clinical trial data can be used as a historical control to evaluate the intervention effectiveness of one-arm community translational initiatives. It is critical to use a proper statistical method to balance the distributions of potential confounders between comparison groups in this kind of evaluations.
| S-EPMC6520136 | biostudies-literature
Project description:Electroencephalography (EEG) datasets are often small and high dimensional, owing to cumbersome recording processes. In these conditions, powerful machine learning techniques are essential to deal with the large amount of information and overcome the curse of dimensionality. Artificial Neural Networks (ANNs) have achieved promising performance in EEG-based Brain-Computer Interface (BCI) applications, but they involve computationally intensive training algorithms and hyperparameter optimization methods. Thus, an awareness of the quality-cost trade-off, although usually overlooked, is highly beneficial. In this paper, we apply a hyperparameter optimization procedure based on Genetic Algorithms to Convolutional Neural Networks (CNNs), Feed-Forward Neural Networks (FFNNs), and Recurrent Neural Networks (RNNs), all of them purposely shallow. We compare their relative quality and energy-time cost, but we also analyze the variability in the structural complexity of networks of the same type with similar accuracies. The experimental results show that the optimization procedure improves accuracy in all models, and that CNN models with only one hidden convolutional layer can equal or slightly outperform a 6-layer Deep Belief Network. FFNN and RNN were not able to reach the same quality, although the cost was significantly lower. The results also highlight the fact that size within the same type of network is not necessarily correlated with accuracy, as smaller models can and do match, or even surpass, bigger ones in performance. In this regard, overfitting is likely a contributing factor since deep learning approaches struggle with limited training examples.
| S-EPMC7289369 | biostudies-literature
Project description:OBJECTIVE We studied the utility of the Diabetes Prevention Trial-Type 1 Risk Score (DPTRS) for improving the accuracy of type 1 diabetes (T1D) risk classification in TrialNet Natural History Study (TNNHS) participants. RESEARCH DESIGN AND METHODS The cumulative incidence of T1D was compared between normoglycemic individuals with DPTRS values >7.00 and dysglycemic individuals in the TNNHS (n = 991). It was also compared between individuals with DPTRS values <7.00 or >7.00 among those with dysglycemia and those with multiple autoantibodies in the TNNHS. DPTRS values >7.00 were compared with dysglycemia for characterizing risk in Diabetes Prevention Trial-Type 1 (DPT-1) (n = 670) and TNNHS participants. The reliability of DPTRS values >7.00 was compared with dysglycemia in the TNNHS. RESULTS The cumulative incidence of T1D for normoglycemic TNNHS participants with DPTRS values >7.00 was comparable to those with dysglycemia. Among those with dysglycemia, the cumulative incidence was much higher (P < 0.001) for those with DPTRS values >7.00 than for those with values <7.00 (3-year risks: 0.16 for <7.00 and 0.46 for >7.00). Dysglycemic individuals in DPT-1 were at much higher risk for T1D than those with dysglycemia in the TNNHS (P < 0.001); there was no significant difference in risk between the studies among those with DPTRS values >7.00. The proportion in the TNNHS reverting from dysglycemia to normoglycemia at the next visit was higher than the proportion reverting from DPTRS values >7.00 to values <7.00 (36 vs. 23%). CONCLUSIONS DPTRS thresholds can improve T1D risk classification accuracy by identifying high-risk normoglycemic and low-risk dysglycemic individuals. The 7.00 DPTRS threshold characterizes risk more consistently between populations and has greater reliability than dysglycemia.
| S-EPMC3964487 | biostudies-literature
Project description:Progressive supranuclear palsy (PSP), multiple system atrophy (MSA) and idiopathic Parkinson's disease (IPD) can be clinically indistinguishable, especially in the early stages, despite distinct patterns of molecular pathology. Structural neuroimaging holds promise for providing objective biomarkers for discriminating these diseases at the single subject level but all studies to date have reported incomplete separation of disease groups. In this study, we employed multi-class pattern recognition to assess the value of anatomical patterns derived from a widely available structural neuroimaging sequence for automated classification of these disorders. To achieve this, 17 patients with PSP, 14 with IPD and 19 with MSA were scanned using structural MRI along with 19 healthy controls (HCs). An advanced probabilistic pattern recognition approach was employed to evaluate the diagnostic value of several pre-defined anatomical patterns for discriminating the disorders, including: (i) a subcortical motor network; (ii) each of its component regions and (iii) the whole brain. All disease groups could be discriminated simultaneously with high accuracy using the subcortical motor network. The region providing the most accurate predictions overall was the midbrain/brainstem, which discriminated all disease groups from one another and from HCs. The subcortical network also produced more accurate predictions than the whole brain and all of its constituent regions. PSP was accurately predicted from the midbrain/brainstem, cerebellum and all basal ganglia compartments; MSA from the midbrain/brainstem and cerebellum and IPD from the midbrain/brainstem only. This study demonstrates that automated analysis of structural MRI can accurately predict diagnosis in individual patients with Parkinsonian disorders, and identifies distinct patterns of regional atrophy particularly useful for this process.
| S-EPMC3711905 | biostudies-literature
Project description:BackgroundThe dataset from genes used to predict hepatitis C virus outcome was evaluated in a previous study using a conventional statistical methodology.ObjectiveThe aim of this study was to reanalyze this same dataset using the data mining approach in order to find models that improve the classification accuracy of the genes studied.MethodsWe built predictive models using different subsets of factors, selected according to their importance in predicting patient classification. We then evaluated each independent model and also a combination of them, leading to a better predictive model.ResultsOur data mining approach identified genetic patterns that escaped detection using conventional statistics. More specifically, the partial decision trees and ensemble models increased the classification accuracy of hepatitis C virus outcome compared with conventional methods.ConclusionsData mining can be used more extensively in biomedicine, facilitating knowledge building and management of human diseases.
| S-EPMC7946589 | biostudies-literature
Project description:BackgroundThe prediction of the genetic disease risk of an individual is a powerful public health tool. While predicting risk has been successful in diseases which follow simple Mendelian inheritance, it has proven challenging in complex diseases for which a large number of loci contribute to the genetic variance. The large numbers of single nucleotide polymorphisms now available provide new opportunities for predicting genetic risk of complex diseases with high accuracy.Methodology/principal findingsWe have derived simple deterministic formulae to predict the accuracy of predicted genetic risk from population or case control studies using a genome-wide approach and assuming a dichotomous disease phenotype with an underlying continuous liability. We show that the prediction equations are special cases of the more general problem of predicting the accuracy of estimates of genetic values of a continuous phenotype. Our predictive equations are responsive to all parameters that affect accuracy and they are independent of allele frequency and effect distributions. Deterministic prediction errors when tested by simulation were generally small. The common link among the expressions for accuracy is that they are best summarized as the product of the ratio of number of phenotypic records per number of risk loci and the observed heritability.Conclusions/significanceThis study advances the understanding of the relative power of case control and population studies of disease. The predictions represent an upper bound of accuracy which may be achievable with improved effect estimation methods. The formulae derived will help researchers determine an appropriate sample size to attain a certain accuracy when predicting genetic risk.
| S-EPMC2561058 | biostudies-literature
Project description:Polygenic risk scores (PRS) hold promise for the early identification of those at risk from neurodegenerative disorders such as Alzheimer's Disease (AD), allowing for intervention to occur prior to neuronal damage. The current selection of informative single nucleotide polymorphisms (SNPs) to generate the risk scores is based on the modelling of large genome-wide association data using significance thresholds. However, the biological relevance of these SNPs is largely unknown. This study, in contrast, aims to identify SNPs with biological relevance to AD and then assess them for their ability to accurately classify cases and controls. Samples selected from the Brains for Dementia Research (BDR) were used to produce gene expression data to identify potential expression quantitative trait loci (eQTLs) relevant to AD. These SNPs were then incorporated into a PRS model to classify AD and controls in the full BDR cohort. Models derived from these eQTLs demonstrate modest classification potential with an accuracy between 61% and 67%. Although the model accuracy is not as high as some values in the literature based on significance thresholds from genome-wide association studies, these models may reflect a more biologically relevant model, which may provide novel targets for therapeutic intervention.
| S-EPMC10454324 | biostudies-literature