Dataset Information

Doubly Optimized Calibrated Support Vector Machine (DOC-SVM): an algorithm for joint optimization of discrimination and calibration.

ABSTRACT: Historically, probabilistic models for decision support have focused on discrimination, e.g., minimizing the ranking error of predicted outcomes. Unfortunately, these models ignore another important aspect, calibration, which indicates the magnitude of correctness of model predictions. Using discrimination and calibration simultaneously can be helpful for many clinical decisions. We investigated tradeoffs between these goals, and developed a unified maximum-margin method to handle them jointly. Our approach called, Doubly Optimized Calibrated Support Vector Machine (DOC-SVM), concurrently optimizes two loss functions: the ridge regression loss and the hinge loss. Experiments using three breast cancer gene-expression datasets (i.e., GSE2034, GSE2990, and Chanrion's datasets) showed that our model generated more calibrated outputs when compared to other state-of-the-art models like Support Vector Machine (p=0.03, p=0.13, and p<0.001) and Logistic Regression (p=0.006, p=0.008, and p<0.001). DOC-SVM also demonstrated better discrimination (i.e., higher AUCs) when compared to Support Vector Machine (p=0.38, p=0.29, and p=0.047) and Logistic Regression (p=0.38, p=0.04, and p<0.0001). DOC-SVM produced a model that was better calibrated without sacrificing discrimination, and hence may be helpful in clinical decision making.

SUBMITTER: Jiang X

PROVIDER: S-EPMC3490990 | biostudies-literature | 2012

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Doubly Optimized Calibrated Support Vector Machine (DOC-SVM): an algorithm for joint optimization of discrimination and calibration.

Jiang Xiaoqian X Menon Aditya A Wang Shuang S Kim Jihoon J Ohno-Machado Lucila L

PloS one 20121106 11

Historically, probabilistic models for decision support have focused on discrimination, e.g., minimizing the ranking error of predicted outcomes. Unfortunately, these models ignore another important aspect, calibration, which indicates the magnitude of correctness of model predictions. Using discrimination and calibration simultaneously can be helpful for many clinical decisions. We investigated tradeoffs between these goals, and developed a unified maximum-margin method to handle them jointly. ...[more]

PMID: 23139819

Similar Datasets

Project description:BackgroundThe body mass index (BMI) is closely related to mortality risk, and energy intake (EI) is essential for maintaining energy balance in weight control. However, self-reported EI has been shown to lead to a systematic underestimation. Total energy expenditure measured using the doubly labelled water (DLW) method is considered an objective biomarker of EI and the gold standard for its estimation in individuals with stable body weight. We aimed to examine the association between DLW-calibrated EI and BMI on overall mortality risk in older adults.MethodsA prospective cohort study was performed using data of 8051 (4267 women and 3784 men) Japanese older adults from the Kyoto-Kameoka Study in Japan. Calibrated EI was calculated from the estimated EI using a food frequency questionnaire and equation developed based on DLW. Participants were classified by quartiles based on their EI stratified by sex. BMI was calculated using self-reported height and body weight. Mortality data were collected between 30 July 2011 and 30 November 2016. Statistical analysis was performed using the multivariable-adjusted Cox proportional hazard model with a restricted cubic spline.ResultsThe 8051 participants' mean (standard deviation) age and BMI were 73.5 (6.1) years and 22.6 (3.0) kg/m2 , respectively. The mean (standard deviation) EI with and without calibration was 1909 (145) kcal/day and 1569 (358) kcal/day in women and 2383 (160) kcal/day and 1980 (515) kcal/day in men, respectively. During the median 4.75 years of follow-up (36 552 person-years), 661 deaths were recorded. In both women (hazard ratio [HR], 0.63; 95% confidence interval [CI] [0.41, 0.98]) and men (HR, 0.62; 95% CI [0.44, 0.87]), after adjusting for confounders, the top quartile as compared with the bottom calibrated EI quartile showed a negative association with risk of all-cause mortality. The lowest HR for all-cause mortality was 1900-2000 kcal/day in women and 2400-2600 kcal/day in men. However, after adjusting for BMI, no significant association was observed between the calibrated EI and the risk of death. These associations could not be confirmed in the uncalibrated EI. The HR for mortality was minimal at a BMI of 23 kg/m2 in both men and women, with or without adjustment for the calibrated EI.ConclusionsCalibrated EI was negatively associated with mortality risk but not uncalibrated EI. This may be mediated by an increase in body weight over time. Caution is required when interpreting the association between EI and mortality risk without adjusting for self-reported measurement errors and outcomes.

Project description:Protein S-sulfenylation, which results from oxidation of free thiols on cysteine residues, has recently emerged as an important post-translational modification that regulates the structure and function of proteins involved in a variety of physiological and pathological processes. By altering the size and physiochemical properties of modified cysteine residues, sulfenylation can impact the cellular function of proteins in several different ways. Thus, the ability to rapidly and accurately identify putative sulfenylation sites in proteins will provide important insights into redox-dependent regulation of protein function in a variety of cellular contexts. Though bottom-up proteomic approaches, such as tandem mass spectrometry (MS/MS), provide a wealth of information about global changes in the sulfenylation state of proteins, MS/MS-based experiments are often labor-intensive, costly and technically challenging. Therefore, to complement existing proteomic approaches, researchers have developed a series of computational tools to identify putative sulfenylation sites on proteins. However, existing methods often suffer from low accuracy, specificity, and/or sensitivity. In this study, we developed SVM-SulfoSite, a novel sulfenylation prediction tool that uses support vector machines (SVM) to identify key determinants of sulfenylation among five feature classes: binary code, physiochemical properties, k-space amino acid pairs, amino acid composition and high-quality physiochemical indices. Using 10-fold cross-validation, SVM-SulfoSite achieved 95% sensitivity and 83% specificity, with an overall accuracy of 89% and Matthew's correlation coefficient (MCC) of 0.79. Likewise, using an independent test set of experimentally identified sulfenylation sites, our method achieved scores of 74%, 62%, 80% and 0.42 for accuracy, sensitivity, specificity and MCC, with an area under the receiver operator characteristic (ROC) curve of 0.81. Moreover, in side-by-side comparisons, SVM-SulfoSite performed as well as or better than existing sulfenylation prediction tools. Together, these results suggest that our method represents a robust and complementary technique for advanced exploration of protein S-sulfenylation.

Dataset Information

Doubly Optimized Calibrated Support Vector Machine (DOC-SVM): an algorithm for joint optimization of discrimination and calibration.

Publications

Doubly Optimized Calibrated Support Vector Machine (DOC-SVM): an algorithm for joint optimization of discrimination and calibration.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets