Dataset Information

Validation of Random Forest Machine Learning Models to Predict Dementia-Related Neuropsychiatric Symptoms in Real-World Data.

ABSTRACT:

Background

Neuropsychiatric symptoms (NPS) are the leading cause of the social burden of dementia but their role is underestimated.

Objective

The objective of the study was to validate predictive models to separately identify psychotic and depressive symptoms in patients diagnosed with dementia using clinical databases representing the whole population to inform decision-makers.

Methods

First, we searched the electronic health records of 4,003 patients with dementia to identify NPS. Second, machine learning (random forest) algorithms were applied to build separate predictive models for psychotic and depressive symptom clusters in the training set (N = 3,003). Third, calibration and discrimination were assessed in the test set (N = 1,000) to assess the performance of the models.

Results

Neuropsychiatric symptoms were noted in the electronic health record of 58% of patients. The area under the receiver operating curve reached 0.80 for the psychotic cluster model and 0.74 for the depressive cluster model. The Kappa index and accuracy also showed better discrimination in the psychotic model. Calibration plots indicated that both types of model had less predictive accuracy when the probability of neuropsychiatric symptoms was <25%. The most important variables in the psychotic cluster model were use of risperidone, level of sedation, use of quetiapine and haloperidol and the number of antipsychotics prescribed. In the depressive cluster model, the most important variables were number of antidepressants prescribed, escitalopram use, level of sedation, and age.

Conclusion

Given their relatively good performance, the predictive models can be used to estimate prevalence of NPS in population databases.

SUBMITTER: Mar J

PROVIDER: S-EPMC7592688 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundInternal carotid artery stenosis (ICAS) can cause stroke and cognitive decline. Associated hemodynamic impairments, which are most pronounced within individual watershed areas (iWSA) between vascular territories, can be assessed with hemodynamic-oxygenation-sensitive MRI and may help to detect severely affected patients. We aimed to identify the most sensitive parameters and volumes of interest (VOI) to predict high-grade ICAS with random forest machine learning. We hypothesized an increased predictive ability considering iWSAs and a decreased cognitive performance in correctly classified patients.Materials and methodsTwenty-four patients with asymptomatic, unilateral, high-grade carotid artery stenosis and 24 age-matched healthy controls underwent MRI comprising pseudo-continuous arterial spin labeling (pCASL), breath-holding functional MRI (BH-fMRI), dynamic susceptibility contrast (DSC), T2 and T2* mapping, MPRAGE and FLAIR. Quantitative maps of eight perfusion, oxygenation and microvascular parameters were obtained. Mean values of respective parameters within and outside of iWSAs split into gray (GM) and white matter (WM) were calculated for both hemispheres and for interhemispheric differences resulting in 96 features. Random forest classifiers were trained on whole GM/WM VOIs, VOIs considering iWSAs and with additional feature selection, respectively.ResultsThe most sensitive features in decreasing order were time-to-peak (TTP), cerebral blood flow (CBF) and cerebral vascular reactivity (CVR), all of these inside of iWSAs. Applying iWSAs combined with feature selection yielded significantly higher receiver operating characteristics areas under the curve (AUC) than whole GM/WM VOIs (AUC: 0.84 vs. 0.90, p = 0.039). Correctly predicted patients presented with worse cognitive performances than frequently misclassified patients (Trail-making-test B: 152.5s vs. 94.4s, p = 0.034).ConclusionRandom forest classifiers trained on multiparametric MRI data allow identification of the most relevant parameters and VOIs to predict ICAS, which may improve personalized treatments.

Project description:Since 2017, we have used IonTorrent NGS platform in our hospital to diagnose and treat cancer. Analyzing variants at each run requires considerable time, and we are still struggling with some variants that appear correct on the metrics at first, but are found to be negative upon further investigation. Can any machine learning algorithm (ML) help us classify NGS variants? This has led us to investigate which ML can fit our NGS data and to develop a tool that can be routinely implemented to help biologists. Currently, one of the greatest challenges in medicine is processing a significant quantity of data. This is particularly true in molecular biology with the advantage of next-generation sequencing (NGS) for profiling and identifying molecular tumors and their treatment. In addition to bioinformatics pipelines, artificial intelligence (AI) can be valuable in helping to analyze mutation variants. Generating sequencing data from patient DNA samples has become easy to perform in clinical trials. However, analyzing the massive quantities of genomic or transcriptomic data and extracting the key biomarkers associated with a clinical response to a specific therapy requires a formidable combination of scientific expertise, biomolecular skills and a panel of bioinformatic and biostatistic tools, in which artificial intelligence is now successful in developing future routine diagnostics. However, cancer genome complexity and technical artifacts make identifying real variants challenging. We present a machine learning method for classifying pathogenic single nucleotide variants (SNVs), single nucleotide polymorphisms (SNPs), multiple nucleotide variants (MNVs), insertions, and deletions detected by NGS from different types of tumor specimens, such as: colorectal, melanoma, lung and glioma cancer. We compared our NGS data to different machine learning algorithms using the k-fold cross-validation method and to neural networks (deep learning) to measure the performance of the different ML algorithms and determine which one is a valid model for confirming NGS variant calls in cancer diagnosis. We trained our machine learning with 70% of our data samples, extracted from our local database (our data structure had 7 parameters: chromosome, position, exon, variant allele frequency, minor allele frequency, coverage and protein description) and validated it with the 30% remaining data. The model offering the best accuracy was chosen and implemented in the NGS analysis routine. Artificial intelligence was developed with the R script language version 3.6.0. We trained our model on 70% of 102,011 variants. Our best error rate (0.22%) was found with random forest machine learning (ntree = 500 and mtry = 4), with an AUC of 0.99. Neural networks achieved some good scores. The final trained model with the neural network achieved an accuracy of 98% and an ROC-AUC of 0.99 with validation data. We tested our RF model to interpret more than 2000 variants from our NGS database: 20 variants were misclassified (error rate < 1%). The errors were nomenclature problems and false positives. After adding false positives to our training database and implementing our RF model routinely, our error rate was always < 0.5%. The RF model shows excellent results for oncosomatic NGS interpretation and can easily be implemented in other molecular biology laboratories. AI is becoming increasingly important in molecular biomedical analysis and can be very helpful in processing medical data. Neural networks show a good capacity in variant classification, and in the future, they may be useful in predicting more complex variants.