Dataset Information

Development and External Validation of a Delirium Prediction Model for Hospitalized Patients With Coronavirus Disease 2019.

ABSTRACT:

Background

The coronavirus disease 2019 pandemic has placed unprecedented stress on health systems and has been associated with elevated risk for delirium. The convergence of pandemic resource limitation and clinical demand associated with delirium requires careful risk stratification for targeted prevention efforts.

Objectives

To develop an incident delirium predictive model among coronavirus disease 2019 patients.

Methods

We applied supervised machine learning to electronic health record data for inpatients with coronavirus disease 2019 at three hospitals to build an incident delirium diagnosis prediction model. We validated this model in three different hospitals. Both hospital cohorts included academic and community settings.

Results

Among 2907 patients across 6 hospitals, 488 (16.8%) developed delirium. Applying the predictive model in the external validation cohort of 755 patients, the c-index was 0.75 (0.71-0.79) and the lift in the top quintile was 2.1. At a sensitivity of 80%, the specificity was 56%, negative predictive value 92%, and positive predictive value 30%. Equivalent model performance was observed in subsamples stratified by age, sex, race, need for critical care and care at community vs. academic hospitals.

Conclusion

Machine learning applied to electronic health records available at the time of inpatient admission can be used to risk-stratify patients with coronavirus disease 2019 for incident delirium. Delirium is common among patients with coronavirus disease 2019, and resource constraints during a pandemic demand careful attention to the optimal application of predictive models.

SUBMITTER: Castro VM

PROVIDER: S-EPMC7933786 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe coronavirus disease 2019 (COVID-19) pandemic dramatically increased the number of patients requiring treatment in an intensive care unit or invasive mechanical ventilation worldwide. Delirium is a well-known neuropsychiatric complication of patients with acute respiratory diseases, representing the most frequent clinical expression of acute brain dysfunction in critically ill patients, especially in those undergoing invasive mechanical ventilation. Among hospitalized patients with COVID-19, delirium incidence ranges from 11% to 80%, depending on the studied population and hospital setting.ObjectiveTo determine risk factors for the development of delirium in hospitalized patients with COVID-19 pneumonia.MethodsWe retrospectively studied consecutive hospitalized adult (≥18 y) patients with confirmed COVID-19 pneumonia from March 15 to July 15, 2020, in a tertiary-care hospital in Mexico City. Delirium was assessed by the attending physician or trained nurse, with either the Confusion Assessment Method for the Intensive Care Unit or the Confusion Assessment Method brief version, according to the appropriate diagnostic tool for each hospital setting. Consultation-liaison psychiatrists and neurologists confirmed all diagnoses. We calculated adjusted hazard ratios (aHR) with 95% confidence interval (CI) using a Cox proportional-hazards regression model.ResultsWe studied 1017 (64.2% men; median age, 54 y; interquartile range 44-64), of whom 166 (16.3%) developed delirium (hyperactive in 75.3%); 78.9% of our delirium cases were detected in patients under invasive mechanical ventilation. The median of days from admission to diagnosis was 14 (interquartile range 8-21) days. Unadjusted mortality rates between delirium and no delirium groups were similar (23.3% vs. 24.1; risk ratio 0.962, 95% CI 0.70-1.33). Age (aHR 1.02, 95% CI 1.01-1.04; P = 0.006), an initial neutrophil-to-lymphocyte ratio ≥9 (aHR 1.81, 95% CI 1.23-2.65; P = 0.003), and requirement of invasive mechanical ventilation (aHR 3.39, 95% CI 1.47-7.84; P = 0.004) were independent risk factors for in-hospital delirium development.ConclusionsDelirium is a common in-hospital complication of patients with COVID-19 pneumonia, associated with disease severity; given the extensive number of active COVID-19 cases worldwide, it is essential to detect patients who are most likely to develop delirium during hospitalization. Improving its preventive measures may reduce the risk of the long-term cognitive and functional sequelae associated with this neuropsychiatric complication.

Project description:ImportanceThe Epic Sepsis Model (ESM), a proprietary sepsis prediction model, is implemented at hundreds of US hospitals. The ESM's ability to identify patients with sepsis has not been adequately evaluated despite widespread use.ObjectiveTo externally validate the ESM in the prediction of sepsis and evaluate its potential clinical value compared with usual care.Design, setting, and participantsThis retrospective cohort study was conducted among 27 697 patients aged 18 years or older admitted to Michigan Medicine, the academic health system of the University of Michigan, Ann Arbor, with 38 455 hospitalizations between December 6, 2018, and October 20, 2019.ExposureThe ESM score, calculated every 15 minutes.Main outcomes and measuresSepsis, as defined by a composite of (1) the Centers for Disease Control and Prevention surveillance criteria and (2) International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnostic codes accompanied by 2 systemic inflammatory response syndrome criteria and 1 organ dysfunction criterion within 6 hours of one another. Model discrimination was assessed using the area under the receiver operating characteristic curve at the hospitalization level and with prediction horizons of 4, 8, 12, and 24 hours. Model calibration was evaluated with calibration plots. The potential clinical benefit associated with the ESM was assessed by evaluating the added benefit of the ESM score compared with contemporary clinical practice (based on timely administration of antibiotics). Alert fatigue was evaluated by comparing the clinical value of different alerting strategies.ResultsWe identified 27 697 patients who had 38 455 hospitalizations (21 904 women [57%]; median age, 56 years [interquartile range, 35-69 years]) meeting inclusion criteria, of whom sepsis occurred in 2552 (7%). The ESM had a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% CI, 0.62-0.64). The ESM identified 183 of 2552 patients with sepsis (7%) who did not receive timely administration of antibiotics, highlighting the low sensitivity of the ESM in comparison with contemporary clinical practice. The ESM also did not identify 1709 patients with sepsis (67%) despite generating alerts for an ESM score of 6 or higher for 6971 of all 38 455 hospitalized patients (18%), thus creating a large burden of alert fatigue.Conclusions and relevanceThis external validation cohort study suggests that the ESM has poor discrimination and calibration in predicting the onset of sepsis. The widespread adoption of the ESM despite its poor performance raises fundamental concerns about sepsis management on a national level.

Project description:BackgroundThe worldwide pandemic of coronavirus disease 2019 (COVID-19) has still been an overwhelming public health challenge, and it is vital to identify determinants early to forecast the risk of severity using indicators easily available at admission. The current multicenter retrospective study aimed to derive and validate a user-friendly and effective nomogram to address this issue.MethodsA training cohort consisting of 437 confirmed COVID-19 cases from three hospitals in Hubei province (Tongji Hospital affiliated with Huazhong University of Science and Technology, Wuhan Third Hospital of Wuhan University and Wuhan Jinyintan Hospital in Hubei province) was retrospectively analyzed to construct a predicting model, and another cohort of 161 hospitalized patients from Public Health Clinical Center of Shanghai was selected as an external validation cohort from January 1, 2020 to March 8, 2020. Determinants of developing into severe COVID-19 were probed using univariate regression together with a multivariate stepwise regression model. The risk of progression to severe COVID-19 was forecasted using the derived nomogram. The performances of the nomogram regarding the discrimination and calibration were assessed in the cohort of training as well as the cohort of external validation, respectively.ResultsA total of 144 (32.95%) and 54 (33.54%) patients, respectively, in cohorts of training and validation progressed to severe COVID-19 during hospitalization. Multivariable analyses showed determinants of severity consisted of hypertension, shortness of breath, platelet count, alanine aminotransferase (ALT), potassium, cardiac troponin I (cTnI), myohemoglobin, procalcitonin (PCT) and intervals from onset to diagnosis. The nomogram had good discrimination with concordance indices being 0.887 (95% CI: 0.854-0.919) and 0.850 (95% CI: 0.815-0.885) in internal and external validation, respectively. Calibration curves exhibited excellent concordance between the predictions by nomogram and actual observations in two cohorts.ConclusionsWe have established and validated an early predicting nomogram model, which can contribute to determine COVID-19 cases at risk of progression to severe illness.

Project description:ObjectivesTo evaluate factors predictive of clinical progression among coronavirus disease 2019 patients following admission, and whether continuous, automated assessments of patient status may contribute to optimal monitoring and management.DesignRetrospective cohort for algorithm training, testing, and validation.SettingEight hospitals across two geographically distinct regions.PatientsTwo-thousand fifteen hospitalized coronavirus disease 2019-positive patients.InterventionsNone.Measurements and main resultsAnticipating Respiratory failure in Coronavirus disease (ARC), a clinically interpretable, continuously monitoring prognostic model of acute respiratory failure in hospitalized coronavirus disease 2019 patients, was developed and validated. An analysis of the most important clinical predictors aligns with key risk factors identified by other investigators but contributes new insights regarding the time at which key factors first begin to exhibit aberrency and distinguishes features predictive of acute respiratory failure in coronavirus disease 2019 versus pneumonia caused by other types of infection. Departing from prior work, ARC was designed to update continuously over time as new observations (vitals and laboratory test results) are recorded in the electronic health record. Validation against data from two geographically distinct health systems showed that the proposed model achieved 75% specificity and 77% sensitivity and predicted acute respiratory failure at a median time of 32 hours prior to onset. Over 80% of true-positive alerts occurred in non-ICU settings.ConclusionsPatients admitted to non-ICU environments with coronavirus disease 2019 are at ongoing risk of clinical progression to severe disease, yet it is challenging to anticipate which patients will develop acute respiratory failure. A continuously monitoring prognostic model has potential to facilitate anticipatory rather than reactive approaches to escalation of care (e.g., earlier initiation of treatments for severe disease or structured monitoring and therapeutic interventions for high-risk patients).

Project description:Background: In patients with chronic obstructive pulmonary disease (COPD), acute exacerbations affect patients' health and can lead to death. This study was aimed to develop a prediction model for in-hospital mortality in patients with acute exacerbations of COPD (AECOPD). Method: A retrospective study was performed in patients hospitalized for AECOPD between 2015 and 2019. Patients admitted between 2015 and 2017 were included to develop model and individuals admitted in the following 2 years were included for external validation. We analyzed variables that were readily available in clinical practice. Given that death was a rare outcome in this study, we fitted Firth penalized logistic regression. C statistic and calibration plot quantified the model performance. Optimism-corrected C statistic and slope were estimated by bootstrapping. Accordingly, the prediction model was adjusted and then transformed into risk score. Result: Between 2015 and 2017, 1,096 eligible patients were analyzed, with a mean age of 73 years and 67.8% male. The in-hospital mortality was 2.6%. Compared to survivors, non-survivors were older, more admitted from emergency, more frequently concomitant with respiratory failure, pneumothorax, hypoxic-hypercarbic encephalopathy, and had longer length of stay (LOS). Four variables were included into the final model: age, respiratory failure, pneumothorax, and LOS. In internal validation, C statistic was 0.9147, and the calibration slope was 1.0254. Their optimism-corrected values were 0.90887 and 0.9282, respectively, indicating satisfactory discrimination and calibration. When externally validated in 700 AECOPD patients during 2018 and 2019, the model demonstrated good discrimination with a C statistic of 0.8176. Calibration plot illustrated a varying discordance between predicted and observed mortality. It demonstrated good calibration in low-risk patients with predicted mortality rate ≤10% (P = 0.3253) but overestimated mortality in patients with predicted rate >10% (P < 0.0001). The risk score of 20 was regarded as a threshold with an optimal Youden index of 0.7154. Conclusion: A simple prediction model for AECOPD in-hospital mortality has been developed and externally validated. Based on available data in clinical setting, the model could serve as an easily used instrument for clinical decision-making. Complications emerged as strong predictors, underscoring an important role of disease management in improving patients' prognoses during exacerbation episodes.