Dataset Information

Complex Machine-Learning Algorithms and Multivariable Logistic Regression on Par in the Prediction of Insufficient Clinical Response to Methotrexate in Rheumatoid Arthritis.

ABSTRACT: The goals of this study were to examine whether machine-learning algorithms outperform multivariable logistic regression in the prediction of insufficient response to methotrexate (MTX); secondly, to examine which features are essential for correct prediction; and finally, to investigate whether the best performing model specifically identifies insufficient responders to MTX (combination) therapy. The prediction of insufficient response (3-month Disease Activity Score 28-Erythrocyte-sedimentation rate (DAS28-ESR) > 3.2) was assessed using logistic regression, least absolute shrinkage and selection operator (LASSO), random forest, and extreme gradient boosting (XGBoost). The baseline features of 355 rheumatoid arthritis (RA) patients from the "treatment in the Rotterdam Early Arthritis CoHort" (tREACH) and the U-Act-Early trial were combined for analyses. The model performances were compared using area under the curve (AUC) of receiver operating characteristic (ROC) curves, 95% confidence intervals (95% CI), and sensitivity and specificity. Finally, the best performing model following feature selection was tested on 101 RA patients starting tocilizumab (TCZ)-monotherapy. Logistic regression (AUC = 0.77 95% CI: 0.68-0.86) performed as well as LASSO (AUC = 0.76, 95% CI: 0.67-0.85), random forest (AUC = 0.71, 95% CI: 0.61 = 0.81), and XGBoost (AUC = 0.70, 95% CI: 0.61-0.81), yet logistic regression reached the highest sensitivity (81%). The most important features were baseline DAS28 (components). For all algorithms, models with six features performed similarly to those with 16. When applied to the TCZ-monotherapy group, logistic regression's sensitivity significantly dropped from 83% to 69% (p = 0.03). In the current dataset, logistic regression performed equally well compared to machine-learning algorithms in the prediction of insufficient response to MTX. Models could be reduced to six features, which are more conducive for clinical implementation. Interestingly, the prediction model was specific to MTX (combination) therapy response.

SUBMITTER: Gosselt HR

PROVIDER: S-EPMC7828730 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Complex Machine-Learning Algorithms and Multivariable Logistic Regression on Par in the Prediction of Insufficient Clinical Response to Methotrexate in Rheumatoid Arthritis.

Gosselt Helen R HR Verhoeven Maxime M A MMA Bulatović-Ćalasan Maja M Welsing Paco M PM de Rotte Maurits C F J MCFJ Hazes Johanna M W JMW Lafeber Floris P J G FPJG Hoogendoorn Mark M de Jonge Robert R

Journal of personalized medicine 20210114 1

The goals of this study were to examine whether machine-learning algorithms outperform multivariable logistic regression in the prediction of insufficient response to methotrexate (MTX); secondly, to examine which features are essential for correct prediction; and finally, to investigate whether the best performing model specifically identifies insufficient responders to MTX (combination) therapy. The prediction of insufficient response (3-month Disease Activity Score 28-Erythrocyte-sedimentatio ...[more]

PMID: 33466633

Similar Datasets

Project description:BackgroundPredictions in pregnancy care are complex because of interactions among multiple factors. Hence, pregnancy outcomes are not easily predicted by a single predictor using only one algorithm or modeling method.ObjectiveThis study aims to review and compare the predictive performances between logistic regression (LR) and other machine learning algorithms for developing or validating a multivariable prognostic prediction model for pregnancy care to inform clinicians' decision making.MethodsResearch articles from MEDLINE, Scopus, Web of Science, and Google Scholar were reviewed following several guidelines for a prognostic prediction study, including a risk of bias (ROB) assessment. We report the results based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Studies were primarily framed as PICOTS (population, index, comparator, outcomes, timing, and setting): Population: men or women in procreative management, pregnant women, and fetuses or newborns; Index: multivariable prognostic prediction models using non-LR algorithms for risk classification to inform clinicians' decision making; Comparator: the models applying an LR; Outcomes: pregnancy-related outcomes of procreation or pregnancy outcomes for pregnant women and fetuses or newborns; Timing: pre-, inter-, and peripregnancy periods (predictors), at the pregnancy, delivery, and either puerperal or neonatal period (outcome), and either short- or long-term prognoses (time interval); and Setting: primary care or hospital. The results were synthesized by reporting study characteristics and ROBs and by random effects modeling of the difference of the logit area under the receiver operating characteristic curve of each non-LR model compared with the LR model for the same pregnancy outcomes. We also reported between-study heterogeneity by using τ2 and I2.ResultsOf the 2093 records, we included 142 studies for the systematic review and 62 studies for a meta-analysis. Most prediction models used LR (92/142, 64.8%) and artificial neural networks (20/142, 14.1%) among non-LR algorithms. Only 16.9% (24/142) of studies had a low ROB. A total of 2 non-LR algorithms from low ROB studies significantly outperformed LR. The first algorithm was a random forest for preterm delivery (logit AUROC 2.51, 95% CI 1.49-3.53; I2=86%; τ2=0.77) and pre-eclampsia (logit AUROC 1.2, 95% CI 0.72-1.67; I2=75%; τ2=0.09). The second algorithm was gradient boosting for cesarean section (logit AUROC 2.26, 95% CI 1.39-3.13; I2=75%; τ2=0.43) and gestational diabetes (logit AUROC 1.03, 95% CI 0.69-1.37; I2=83%; τ2=0.07).ConclusionsPrediction models with the best performances across studies were not necessarily those that used LR but also used random forest and gradient boosting that also performed well. We recommend a reanalysis of existing LR models for several pregnancy outcomes by comparing them with those algorithms that apply standard guidelines.Trial registrationPROSPERO (International Prospective Register of Systematic Reviews) CRD42019136106; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=136106.

Project description:IntroductionMethotrexate (MTX) constitutes the first-line therapy in rheumatoid arthritis (RA), yet approximately 30% of the patients do not benefit from MTX. Recently, we reported a prognostic multivariable prediction model for insufficient clinical response to MTX at 3 months of treatment in the treatment in the Rotterdam Early Arthritis Cohort (tREACH), including baseline predictors: Disease activity score 28 (DAS28), Health Assessment Questionnaire (HAQ), erythrocyte folate, single-nucleotide polymorphisms (SNPs; ABCB1, ABCC3), smoking, and BMI. The purpose of the current study was (1) to externally validate the model and (2) to enhance the model's clinical applicability.MethodsErythrocyte folate and SNPs were assessed in 91 early disease-modifying antirheumatic drug (DMARD)-naïve RA patients starting MTX in the external validation cohort (U-Act-Early). Insufficient response (DAS28 > 3.2) was determined after 3 months and non-response after 6 months of therapy. The previously developed prediction model was considered successfully validated in the U-Act-Early (validation cohort) if the area under the curve (AUC) of the receiver operating characteristic (ROC) was not significantly lower than in the tREACH (derivation cohort).ResultsThe AUCs in U-Act-Early at three and 6 months were 0.75 (95% CI 0.64-0.85) and 0.71 (95% CI 0.60-0.82) respectively, similar to the tREACH. Baseline DAS28 > 5.1 and HAQ > 0.6 were the strongest predictors. The model was simplified by excluding the SNPs, while still classifying 73% correctly. Furthermore, interaction terms between BMI and HAQ and BMI and erythrocyte folate significantly improved the model increasing correct classification to 75%. Results were successfully implemented in Evidencio online platform assisting clinicians in shared decision-making to intensify treatment when appropriate.ConclusionsWe successfully externally validated our recently reported prediction model for MTX non-response and enhanced its clinical application thus enabling its evaluation in a clinical trial.Trial registrationThe U-Act-Early is registered at ClinicalTrials.gov. number: NCT01034137. tREACH is registered retrospectively at ISRCTN registry, number: ISRCTN26791028 at 23 August 2007.

Dataset Information

Complex Machine-Learning Algorithms and Multivariable Logistic Regression on Par in the Prediction of Insufficient Clinical Response to Methotrexate in Rheumatoid Arthritis.

Publications

Complex Machine-Learning Algorithms and Multivariable Logistic Regression on Par in the Prediction of Insufficient Clinical Response to Methotrexate in Rheumatoid Arthritis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets