Dataset Information

Machine Learning-based Derivation and External Validation of a Tool to Predict Death and Development of Organ Failure in Hospitalized Patients with COVID-19.

ABSTRACT: BackgroundCOVID-19 mortality risk stratification tools could improve care, inform accurate and rapid triage decisions, and guide family discussions regarding goals of care. A minority of COVID-19 prognostic tools have been tested in external cohorts. Our objective was to compare machine learning algorithms and develop a tool for predicting subsequent clinical outcomes in COVID-19. MethodsWe conducted a retrospective cohort study that included hospitalized patients with COVID-19 from March 2020 to March 2021. 712 consecutive patients from University of Washington (UW) and 345 patients from Tongji Hospital in China were included. We applied three different machine learning algorithms to clinical and laboratory data collected within the initial 24 hours of hospital admission to determine the risk of in-hospital mortality, transfer to the intensive care unit (ICU), shock requiring vasopressors, and receipt of renal replacement therapy (RRT). Mortality risk models were derived, internally validated in UW and externally validated in Tongji Hospital. The risk models for ICU transfer, shock and RRT were derived and internally validated in the UW dataset. ResultsAmong the UW dataset, 122 patients died (17%) during hospitalization and the mean days to hospital mortality was 15.7 +/- 21.5 (mean +/- SD). Elastic net logistic regression resulted in a C-statistic for in-hospital mortality of 0.72 (95% CI, 0.64 to 0.81) in the internal validation and 0.85 (95% CI, 0.81 to 0.89) in the external validation set. Age, platelet count, and white blood cell count were the most important predictors of mortality. In the sub-group of patients > 50 years of age, the mortality prediction model continued to perform with a C-statistic of 0.82 (95% CI:0.76,0.87). Mortality prediction models also performed well for shock and RRT in the UW dataset but functioned with lower accuracy for ICU transfer. ConclusionsWe trained, internally and externally validated a prediction model using data collected within 24 hours of hospital admission to predict in-hospital mortality on average two weeks prior to death. We also developed models to predict RRT and shock with high accuracy. These models could be used to improve triage decisions, resource allocation, and support clinical trial enrichment.

SUBMITTER: Xu Y

PROVIDER: S-EPMC8609901 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:AbstractAs severe acute respiratory syndrome coronavirus 2 continues to spread, easy-to-use risk models that predict hospital mortality can assist in clinical decision making and triage. We aimed to develop a risk score model for in-hospital mortality in patients hospitalized with 2019 novel coronavirus (COVID-19) that was robust across hospitals and used clinical factors that are readily available and measured standardly across hospitals.In this retrospective observational study, we developed a risk score model using data collected by trained abstractors for patients in 20 diverse hospitals across the state of Michigan (Mi-COVID19) who were discharged between March 5, 2020 and August 14, 2020. Patients who tested positive for severe acute respiratory syndrome coronavirus 2 during hospitalization or were discharged with an ICD-10 code for COVID-19 (U07.1) were included. We employed an iterative forward selection approach to consider the inclusion of 145 potential risk factors available at hospital presentation. Model performance was externally validated with patients from 19 hospitals in the Mi-COVID19 registry not used in model development. We shared the model in an easy-to-use online application that allows the user to predict in-hospital mortality risk for a patient if they have any subset of the variables in the final model.Two thousand one hundred and ninety-three patients in the Mi-COVID19 registry met our inclusion criteria. The derivation and validation sets ultimately included 1690 and 398 patients, respectively, with mortality rates of 19.6% and 18.6%, respectively. The average age of participants in the study after exclusions was 64 years old, and the participants were 48% female, 49% Black, and 87% non-Hispanic. Our final model includes the patient's age, first recorded respiratory rate, first recorded pulse oximetry, highest creatinine level on day of presentation, and hospital's COVID-19 mortality rate. No other factors showed sufficient incremental model improvement to warrant inclusion. The area under the receiver operating characteristics curve for the derivation and validation sets were .796 (95% confidence interval, .767-.826) and .829 (95% confidence interval, .782-.876) respectively.We conclude that the risk of in-hospital mortality in COVID-19 patients can be reliably estimated using a few factors, which are standardly measured and available to physicians very early in a hospital encounter.

Project description:Background Opioid misuse screening in hospitals is resource-intensive and rarely done. Many hospitalized patients are never offered opioid treatment. An automated approach leveraging routinely captured electronic health record (EHR) data may be easier for hospitals to institute. We previously derived and internally validated an opioid classifier in a separate hospital setting. The aim is to externally validate our previously published and open-source machine-learning classifier at a different hospital for identifying cases of opioid misuse. Methods An observational cohort of 56,227 adult hospitalizations was examined between October 2017 and December 2019 during a hospital-wide substance use screening program with manual screening. Manually completed Drug Abuse Screening Test served as the reference standard to validate a convolutional neural network (CNN) classifier with coded word embedding features from the clinical notes of the EHR. The opioid classifier utilized all notes in the EHR and sensitivity analysis was also performed on the first 24 h of notes. Calibration was performed to account for the lower prevalence than in the original cohort. Results Manual screening for substance misuse was completed in 67.8% (n?=?56,227) with 1.1% (n?=?628) identified with opioid misuse. The data for external validation included 2,482,900 notes with 67,969 unique clinical concept features. The opioid classifier had an AUC of 0.99 (95% CI 0.99–0.99) across the encounter and 0.98 (95% CI 0.98–0.99) using only the first 24 h of notes. In the calibrated classifier, the sensitivity and positive predictive value were 0.81 (95% CI 0.77–0.84) and 0.72 (95% CI 0.68–0.75). For the first 24 h, they were 0.75 (95% CI 0.71–0.78) and 0.61 (95% CI 0.57–0.64). Conclusions Our opioid misuse classifier had good discrimination during external validation. Our model may provide a comprehensive and automated approach to opioid misuse identification that augments current workflows and overcomes manual screening barriers. Supplementary Information The online version contains supplementary material available at 10.1186/s13722-021-00229-7.

Project description:BackgroundWith higher adoption of electronic health records at health-care centers, electronic search algorithms (computable phenotype) for identifying acute decompensated heart failure (ADHF) among hospitalized patients can be an invaluable tool to enhance data abstraction accuracy and efficacy in order to improve clinical research accrual and patient centered outcomes. We aimed to derive and validate a computable phenotype for ADHF in hospitalized patients.MethodsWe screened 256, 443 eligible (age > 18 years and with prior research authorization) individuals who were admitted to Mayo Clinic Hospital in Rochester, MN, from January 1, 2006, through December 31, 2014. Using a randomly selected derivation cohort of 938 patients, several iterations of a free-text electronic search were developed and refined. The computable phenotype was subsequently validated in an independent cohort 100 patients. The sensitivity and specificity of the computable phenotype were compared to the gold standard (expert review of charts) and International Classification of Diseases-9 (ICD-9) codes for Acute Heart Failure.ResultsIn the derivation cohort, the computable phenotype achieved a sensitivity of 97.5%, and specificity of 100%, whereas ICD-9 codes for Acute Heart Failure achieved a sensitivity of 47.5% and specificity of 96.7%. When all Heart Failure codes (ICD-9) were used, sensitivity and specificity were 97.5 and 86.6%, respectively. In the validation cohort, the sensitivity and specificity of the computable phenotype were 100 and 98.5%. The sensitivity and specificity for the ICD-9 codes (Acute Heart Failure) were 42 and 98.5%. Upon use of all Heart Failure codes (ICD-9), sensitivity and specificity were 96.8 and 91.3%.ConclusionsOur results suggest that using computable phenotype to ascertain ADHF from the clinical notes contained within the electronic medical record are feasible and reliable. Our computable phenotype outperformed ICD-9 codes for the detection of ADHF.

Project description:PurposeFall prevention is a safety goal in many hospitals. The performance of the Johns Hopkins Fall Risk Assessment Tool (JHFRAT) in older inpatients is largely unknown. We aimed to assess the JHFRAT performance in a large sample of Dutch older inpatients, including its trend over time.MethodsWe used an Electronic Health Records (EHR) dataset with hospitalized patients (≥ 70), admitted for ≥ 24 h between 2016 and 2021. Inpatient falls were extracted from structured and free-text data. We assessed the association between JHFRAT and falls using logistic regression. For test accuracy, we calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Discrimination was measured by the AUC. For calibration, we plotted the predicted fall probability with the actual probability of falls. For time-related effects, we calculated the AUC per 6 months (using data of patients admitted during the 6 months' time interval) and plotted these different AUC values over time. Furthermore, we compared the model (JHFRAT and falls) with and without adjusting for seasonal influenza, COVID-19, spring, summer, fall or winter periods.ResultsData included 17,263 admissions with at least 1 JHFRAT measurement, a median age of 76 and a percentage female of 47%. The in-hospital fall prevalence was 2.5%. JHFRAT [OR = 1.11 (1.03-1.20)] and its subcategories were significantly associated with falls. For medium/high risk of falls (JHFRAT > 5), sensitivity was 73%, specificity 51%, PPV 4% and NPV 99%. The overall AUC was 0.67, varying over time between 0.62 and 0.71 (for 6 months' time intervals). Seasonal influenza did affect the association between JHFRAT and falls. COVID-19, spring, summer, fall or winter did not affect the association.ConclusionsOur results show an association between JHFRAT and falls, a low discrimination by JHFRAT for older inpatients and over-prediction in the calibration. Improvements in the fall-risk assessment are warranted to improve efficiency.

Project description:Background and aimsUnhealthy alcohol use (UAU) is one of the leading causes of global morbidity. A machine learning approach to alcohol screening could accelerate best practices when integrated into electronic health record (EHR) systems. This study aimed to validate externally a natural language processing (NLP) classifier developed at an independent medical center.DesignRetrospective cohort study.SettingThe site for validation was a midwestern United States tertiary-care, urban medical center that has an inpatient structured universal screening model for unhealthy substance use and an active addiction consult service.Participants/casesUnplanned admissions of adult patients between October 23, 2017 and December 31, 2019, with EHR documentation of manual alcohol screening were included in the cohort (n = 57 605).MeasurementsThe Alcohol Use Disorders Identification Test (AUDIT) served as the reference standard. AUDIT scores ≥5 for females and ≥8 for males served as cases for UAU. To examine error in manual screening or under-reporting, a post hoc error analysis was conducted, reviewing discordance between the NLP classifier and AUDIT-derived reference. All clinical notes excluding the manual screening and AUDIT documentation from the EHR were included in the NLP analysis.FindingsUsing clinical notes from the first 24 hours of each encounter, the NLP classifier demonstrated an area under the receiver operating characteristic curve (AUCROC) and precision-recall area under the curve (PRAUC) of 0.91 (95% CI = 0.89-0.92) and 0.56 (95% CI = 0.53-0.60), respectively. At the optimal cut point of 0.5, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were 0.66 (95% CI = 0.62-0.69), 0.98 (95% CI = 0.98-0.98), 0.35 (95% CI = 0.33-0.38), and 1.0 (95% CI = 1.0-1.0), respectively.ConclusionsExternal validation of a publicly available alcohol misuse classifier demonstrates adequate sensitivity and specificity for routine clinical use as an automated screening tool for identifying at-risk patients.

Dataset Information

Machine Learning-based Derivation and External Validation of a Tool to Predict Death and Development of Organ Failure in Hospitalized Patients with COVID-19.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets