Dataset Information

Comparing machine and deep learning-based algorithms for prediction of clinical improvement in psychosis with functional magnetic resonance imaging.

ABSTRACT: Previous work using logistic regression suggests that cognitive control-related frontoparietal activation in early psychosis can predict symptomatic improvement after 1?year of coordinated specialty care with 66% accuracy. Here, we evaluated the ability of six machine learning (ML) algorithms and deep learning (DL) to predict "Improver" status (>20% improvement on Brief Psychiatric Rating Scale [BPRS] total score at 1-year follow-up vs. baseline) and continuous change in BPRS score using the same functional magnetic resonance imaging-based features (frontoparietal activations during the AX-continuous performance task) in the same sample (individuals with either schizophrenia (n?= 65, 49M/16F, mean age 20.8?years) or Type I bipolar disorder (n =?17, 9M/8F, mean age 21.6?years)). 138 healthy controls were included as a reference group. "Shallow" ML methods included Naive Bayes, support vector machine, K Star, AdaBoost, J48 decision tree, and random forest. DL included an explainable artificial intelligence (XAI) procedure for understanding results. The best overall performances (70% accuracy for the binary outcome and root mean square error = 9.47 for the continuous outcome) were achieved using DL. XAI revealed left DLPFC activation was the strongest feature used to make binary classification decisions, with a classification activation threshold (adjusted beta = .017) intermediate to the healthy control mean (adjusted beta = .15, 95% CI = -0.02 to 0.31) and patient mean (adjusted beta = -.13, 95% CI = -0.37 to 0.11). Our results suggest DL is more powerful than shallow ML methods for predicting symptomatic improvement. The left DLPFC may be a functional target for future biomarker development as its activation was particularly important for predicting improvement.

SUBMITTER: Smucny J

PROVIDER: S-EPMC7856652 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparing machine and deep learning-based algorithms for prediction of clinical improvement in psychosis with functional magnetic resonance imaging.

Smucny Jason J Davidson Ian I Carter Cameron S CS

Human brain mapping 20201113 4

Previous work using logistic regression suggests that cognitive control-related frontoparietal activation in early psychosis can predict symptomatic improvement after 1 year of coordinated specialty care with 66% accuracy. Here, we evaluated the ability of six machine learning (ML) algorithms and deep learning (DL) to predict "Improver" status (>20% improvement on Brief Psychiatric Rating Scale [BPRS] total score at 1-year follow-up vs. baseline) and continuous change in BPRS score using the sam ...[more]

PMID: 33185307

Similar Datasets

Project description:Many chemicals that disrupt endocrine function have been linked to a variety of adverse biological outcomes. However, screening for endocrine disruption using in vitro or in vivo approaches is costly and time-consuming. Computational methods, e.g., quantitative structure-activity relationship models, have become more reliable due to bigger training sets, increased computing power, and advanced machine learning algorithms, such as multilayered artificial neural networks. Machine learning models can be used to predict compounds for endocrine disrupting capabilities, such as binding to the estrogen receptor (ER), and allow for prioritization and further testing. In this work, an exhaustive comparison of multiple machine learning algorithms, chemical spaces, and evaluation metrics for ER binding was performed on public data sets curated using in-house cheminformatics software (Assay Central). Chemical features utilized in modeling consisted of binary fingerprints (ECFP6, FCFP6, ToxPrint, or MACCS keys) and continuous molecular descriptors from RDKit. Each feature set was subjected to classic machine learning algorithms (Bernoulli Naive Bayes, AdaBoost Decision Tree, Random Forest, Support Vector Machine) and Deep Neural Networks (DNN). Models were evaluated using a variety of metrics: recall, precision, F1-score, accuracy, area under the receiver operating characteristic curve, Cohen's Kappa, and Matthews correlation coefficient. For predicting compounds within the training set, DNN has an accuracy higher than that of other methods; however, in 5-fold cross validation and external test set predictions, DNN and most classic machine learning models perform similarly regardless of the data set or molecular descriptors used. We have also used the rank normalized scores as a performance-criteria for each machine learning method, and Random Forest performed best on the validation set when ranked by metric or by data sets. These results suggest classic machine learning algorithms may be sufficient to develop high quality predictive models of ER activity.

Project description:BackgroundTrauma is one of the most critical public health issues worldwide, leading to death and disability and influencing all age groups. Therefore, there is great interest in models for predicting mortality in trauma patients admitted to the ICU. The main objective of the present study is to develop and evaluate SMOTE-based machine-learning tools for predicting hospital mortality in trauma patients with imbalanced data.MethodsThis retrospective cohort study was conducted on 126 trauma patients admitted to an intensive care unit at Besat hospital in Hamadan Province, western Iran, from March 2020 to March 2021. Data were extracted from the medical information records of patients. According to the imbalanced property of the data, SMOTE techniques, namely SMOTE, Borderline-SMOTE1, Borderline-SMOTE2, SMOTE-NC, and SVM-SMOTE, were used for primary preprocessing. Then, the Decision Tree (DT), Random Forest (RF), Naive Bayes (NB), Artificial Neural Network (ANN), Support Vector Machine (SVM), and Extreme Gradient Boosting (XGBoost) methods were used to predict patients' hospital mortality with traumatic injuries. The performance of the methods used was evaluated by sensitivity, specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), accuracy, Area Under the Curve (AUC), Geometric Mean (G-means), F1 score, and P-value of McNemar's test.ResultsOf the 126 patients admitted to an ICU, 117 (92.9%) survived and 9 (7.1%) died. The mean follow-up time from the date of trauma to the date of outcome was 3.98 ± 4.65 days. The performance of ML algorithms is not good with imbalanced data, whereas the performance of SMOTE-based ML algorithms is significantly improved. The mean area under the ROC curve (AUC) of all SMOTE-based models was more than 91%. F1-score and G-means before balancing the dataset were below 70% for all ML models except ANN. In contrast, F1-score and G-means for the balanced datasets reached more than 90% for all SMOTE-based models. Among all SMOTE-based ML methods, RF and ANN based on SMOTE and XGBoost based on SMOTE-NC achieved the highest value for all evaluation criteria.ConclusionsThis study has shown that SMOTE-based ML algorithms better predict outcomes in traumatic injuries than ML algorithms. They have the potential to assist ICU physicians in making clinical decisions.

Project description:A relatively large number of studies have investigated the power of structural magnetic resonance imaging (sMRI) data to discriminate patients with schizophrenia from healthy controls. However, very few of them have also included patients with bipolar disorder, allowing the clinically relevant discrimination between both psychotic diagnostics. To assess the efficacy of sMRI data for diagnostic prediction in psychosis we objectively evaluated the discriminative power of a wide range of commonly used machine learning algorithms (ridge, lasso, elastic net and L0 norm regularized logistic regressions, a support vector classifier, regularized discriminant analysis, random forests and a Gaussian process classifier) on main sMRI features including grey and white matter voxel-based morphometry (VBM), vertex-based cortical thickness and volume, region of interest volumetric measures and wavelet-based morphometry (WBM) maps. All possible combinations of algorithms and data features were considered in pairwise classifications of matched samples of healthy controls (N = 127), patients with schizophrenia (N = 128) and patients with bipolar disorder (N = 128). Results show that the selection of feature type is important, with grey matter VBM (without data reduction) delivering the best diagnostic prediction rates (averaging over classifiers: schizophrenia vs. healthy 75%, bipolar disorder vs. healthy 63% and schizophrenia vs. bipolar disorder 62%) whereas algorithms usually yielded very similar results. Indeed, those grey matter VBM accuracy rates were not even improved by combining all feature types in a single prediction model. Further multi-class classifications considering the three groups simultaneously made evident a lack of predictive power for the bipolar group, probably due to its intermediate anatomical features, located between those observed in healthy controls and those found in patients with schizophrenia. Finally, we provide MRIPredict (https://www.nitrc.org/projects/mripredict/), a free tool for SPM, FSL and R, to easily carry out voxelwise predictions based on VBM images.

Project description:BackgroundAlthough there are many studies on the prognostic factors of left ventricular myocardial noncompaction (LVNC), the determinants are varied and not entirely consistent. This study aimed to build predictive models using radiomics features and machine learning to predict major adverse cardiovascular events (MACEs) in patients with LVNC.MethodsIn total, 96 patients with LVNC were included and randomly divided into training and test cohorts. A total of 105 cine cardiac magnetic resonance (CMR)-derived radiomics features and 35 clinical characteristics were extracted. Five different oversampling algorithms were compared for selection of the optimal imbalanced processing. Feature importance was assessed with extreme gradient boosting (XGBoost). We compared the performance of 5 machine learning classification methods with different sample:feature ratios to determine the optimal hybrid classification strategy. Subsequently, radiomics, clinical, and combined radiomics-clinical models were developed and compared.ResultsThe machine learning pipeline included an adaptive synthetic (ADASYN) algorithm for imbalanced processing, XGBoost feature selection with a sample:feature ratio of 10, and support vector machine (SVM) modeling. The areas under the receiver operating characteristic curves (AUCs) of the radiomics model, clinical model, and combined model in the validation cohort were 0.87 (sensitivity 83.33%, specificity 64.29%), 0.65 (sensitivity 16.67%, specificity 78.57%), and 0.92 (specificity 33.33%, sensitivity 100.00%), respectively. The radiomics model performed similarly to the clinical and combined models (P=0.124 and P=0.621, respectively). The performance of the combined model was significantly better than that of the clinical model (P=0.003).ConclusionsThe machine learning-based cine CMR radiomics model performed well at predicting MACEs in patients with LVNC. Adding radiomics features offered incremental prognostic value over clinical factors alone.

Dataset Information

Comparing machine and deep learning-based algorithms for prediction of clinical improvement in psychosis with functional magnetic resonance imaging.

Publications

Comparing machine and deep learning-based algorithms for prediction of clinical improvement in psychosis with functional magnetic resonance imaging.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets