Dataset Information

Development and Validation of Machine Learning Models in Prediction of Remission in Patients With Moderate to Severe Crohn Disease.

ABSTRACT:

Importance

Biological therapies have revolutionized inflammatory bowel disease management, but many patients do not respond to biological monotherapy. Identification of likely responders could reduce costs and delays in remission.

Objective

To identify patients with Crohn disease likely to be durable responders to ustekinumab before committing to long-term treatment.

Design, setting, and participants

This cohort study analyzed data from 3 phase 3 randomized clinical trials (UNITI-1, UNITI-2, and IM-UNITI) conducted from 2011 to 2015. Participants (n = 401) were individuals with active (C-reactive protein [CRP] measurement of ≥5 mg/L at enrollment) Crohn disease who received ustekinumab therapy. Data analysis was performed from November 1, 2017, to June 1, 2018.

Exposures

All included patients were exposed to 1 or more dose of ustekinumab for 8 weeks or more.

Main outcomes and measures

Random forest methods were used in building 2 models for predicting Crohn disease remission, with a CRP level lower than 5 mg/dL as a proxy for biological remission, beyond week 42 of ustekinumab treatment. The first model used only baseline data, and the second used data through week 8.

Results

In total, 401 participants, with a mean (SD) age of 36.3 (12.6) years and 170 male (42.4%), were included. The week-8 model had a mean area under the receiver operating characteristic curve (AUROC) of 0.78 (95% CI, 0.69-0.87). In the testing data set, 27 of 55 participants (49.1%) classified as likely to have treatment success achieved success with a CRP level lower than 5 mg/L after week 42, and 7 of 65 participants (10.8%) classified as likely to have treatment failure achieved this outcome. In the full cohort, 87 patients (21.7%) attained remission after week 42. A prediction model using the week-6 albumin to CRP ratio had an AUROC of 0.76 (95% CI, 0.71-0.82). Baseline ustekinumab serum levels did not improve the model's prediction performance.

Conclusions and relevance

In patients with active Crohn disease, demographic and laboratory data before week 8 of treatment appeared to allow the prompt identification of likely nonresponders to ustekinumab without the need for costly drug-level monitoring.

SUBMITTER: Waljee AK

PROVIDER: S-EPMC6512283 | biostudies-literature | 2019 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and Validation of Machine Learning Models in Prediction of Remission in Patients With Moderate to Severe Crohn Disease.

Waljee Akbar K AK Wallace Beth I BI Cohen-Mekelburg Shirley S Liu Yumu Y Liu Boang B Sauder Kay K Stidham Ryan W RW Zhu Ji J Higgins Peter D R PDR

JAMA network open 20190503 5

<h4>Importance</h4>Biological therapies have revolutionized inflammatory bowel disease management, but many patients do not respond to biological monotherapy. Identification of likely responders could reduce costs and delays in remission.<h4>Objective</h4>To identify patients with Crohn disease likely to be durable responders to ustekinumab before committing to long-term treatment.<h4>Design, setting, and participants</h4>This cohort study analyzed data from 3 phase 3 randomized clinical trials ...[more]

PMID: 31074823

Similar Datasets

Project description:BackgroundAlthough papillary thyroid cancer (PTC) patients are known to have an excellent prognosis, up to 30% of patients experience disease recurrence after initial treatment. Accurately predicting disease prognosis remains a challenge given that the predictive value of several predictors remains controversial. Thus, we investigated whether machine learning (ML) approaches based on comprehensive predictors can predict the risk of structural recurrence for PTC patients.MethodsA total of 2244 patients treated with thyroid surgery and radioiodine were included. Twenty-nine perioperative variables consisting of four dimensions (demographic characteristics and comorbidities, tumor-related variables, lymph node (LN)-related variables, and metabolic and inflammatory markers) were analyzed. We applied five ML algorithms-logistic regression (LR), support vector machine (SVM), extreme gradient boosting (XGBoost), random forest (RF), and neural network (NN)-to develop the models. The area under the receiver operating characteristic (AUC-ROC) curve, calibration curve, and variable importance were used to evaluate the models' performance.ResultsDuring a median follow-up of 45.5 months, 179 patients (8.0%) experienced structural recurrence. The non-stimulated thyroglobulin, LN dissection, number of LNs dissected, lymph node metastasis ratio, N stage, comorbidity of hypertension, comorbidity of diabetes, body mass index, and low-density lipoprotein were used to develop the models. All models showed a greater AUC (AUC = 0.738 to 0.767) than did the ATA risk stratification (AUC = 0.620, DeLong test: P < 0.01). The SVM, XGBoost, and RF model showed greater sensitivity (0.568, 0.595, 0.676), specificity (0.903, 0.857, 0.784), accuracy (0.875, 0.835, 0.775), positive predictive value (PPV) (0.344, 0.272, 0.219), negative predictive value (NPV) (0.959, 0.959, 0.964), and F1 score (0.429, 0.373, 0.331) than did the ATA risk stratification (sensitivity = 0.432, specificity = 0.770, accuracy = 0.742, PPV = 0.144, NPV = 0.938, F1 score = 0.216). The RF model had generally consistent calibration compared with the other models. The Tg and the LNR were the top 2 important variables in all the models, the N stage was the top 5 important variables in all the models.ConclusionsThe RF model achieved the expected prediction performance with generally good discrimination, calibration and interpretability in this study. This study sheds light on the potential of ML approaches for improving the accuracy of risk stratification for PTC patients.Trial registrationRetrospectively registered at www.chictr.org.cn (trial registration number: ChiCTR2300075574, date of registration: 2023-09-08).

Project description:BackgroundPatient safety in the intensive care unit (ICU) is one of the most critical issues, and unplanned extubation (UE) is considered the most adverse event for patient safety. Prevention and early detection of such an event is an essential but difficult component of quality care.ObjectiveThis study aimed to develop and validate prediction models for UE in ICU patients using machine learning.MethodsThis study was conducted in an academic tertiary hospital in Seoul, Republic of Korea. The hospital had approximately 2000 inpatient beds and 120 ICU beds. As of January 2019, the hospital had approximately 9000 outpatients on a daily basis. The number of annual ICU admissions was approximately 10,000. We conducted a retrospective study between January 1, 2010, and December 31, 2018. A total of 6914 extubation cases were included. We developed a UE prediction model using machine learning algorithms, which included random forest (RF), logistic regression (LR), artificial neural network (ANN), and support vector machine (SVM). For evaluating the model's performance, we used the area under the receiver operating characteristic curve (AUROC). The sensitivity, specificity, positive predictive value, negative predictive value, and F1 score were also determined for each model. For performance evaluation, we also used a calibration curve, the Brier score, and the integrated calibration index (ICI) to compare different models. The potential clinical usefulness of the best model at the best threshold was assessed through a net benefit approach using a decision curve.ResultsAmong the 6914 extubation cases, 248 underwent UE. In the UE group, there were more males than females, higher use of physical restraints, and fewer surgeries. The incidence of UE was higher during the night shift as compared to the planned extubation group. The rate of reintubation within 24 hours and hospital mortality were higher in the UE group. The UE prediction algorithm was developed, and the AUROC for RF was 0.787, for LR was 0.762, for ANN was 0.763, and for SVM was 0.740.ConclusionsWe successfully developed and validated machine learning-based prediction models to predict UE in ICU patients using electronic health record data. The best AUROC was 0.787 and the sensitivity was 0.949, which was obtained using the RF algorithm. The RF model was well-calibrated, and the Brier score and ICI were 0.129 and 0.048, respectively. The proposed prediction model uses widely available variables to limit the additional workload on the clinician. Further, this evaluation suggests that the model holds potential for clinical usefulness.

Project description:BackgroundIn health research, several chronic diseases are susceptible to competing risks (CRs). Initially, statistical models (SM) were developed to estimate the cumulative incidence of an event in the presence of CRs. As recently there is a growing interest in applying machine learning (ML) for clinical prediction, these techniques have also been extended to model CRs but literature is limited. Here, our aim is to investigate the potential role of ML versus SM for CRs within non-complex data (small/medium sample size, low dimensional setting).MethodsA dataset with 3826 retrospectively collected patients with extremity soft-tissue sarcoma (eSTS) and nine predictors is used to evaluate model-predictive performance in terms of discrimination and calibration. Two SM (cause-specific Cox, Fine-Gray) and three ML techniques are compared for CRs in a simple clinical setting. ML models include an original partial logistic artificial neural network for CRs (PLANNCR original), a PLANNCR with novel specifications in terms of architecture (PLANNCR extended), and a random survival forest for CRs (RSFCR). The clinical endpoint is the time in years between surgery and disease progression (event of interest) or death (competing event). Time points of interest are 2, 5, and 10 years.ResultsBased on the original eSTS data, 100 bootstrapped training datasets are drawn. Performance of the final models is assessed on validation data (left out samples) by employing as measures the Brier score and the Area Under the Curve (AUC) with CRs. Miscalibration (absolute accuracy error) is also estimated. Results show that the ML models are able to reach a comparable performance versus the SM at 2, 5, and 10 years regarding both Brier score and AUC (95% confidence intervals overlapped). However, the SM are frequently better calibrated.ConclusionsOverall, ML techniques are less practical as they require substantial implementation time (data preprocessing, hyperparameter tuning, computational intensity), whereas regression methods can perform well without the additional workload of model training. As such, for non-complex real life survival data, these techniques should only be applied complementary to SM as exploratory tools of model's performance. More attention to model calibration is urgently needed.

Dataset Information

Development and Validation of Machine Learning Models in Prediction of Remission in Patients With Moderate to Severe Crohn Disease.

Importance

Objective

Design, setting, and participants

Exposures

Main outcomes and measures

Results

Conclusions and relevance

Publications

Development and Validation of Machine Learning Models in Prediction of Remission in Patients With Moderate to Severe Crohn Disease.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets