Dataset Information

External validation of a prognostic model for intensive care unit mortality: a retrospective study using the Ontario Critical Care Information System.

ABSTRACT: PURPOSE:To externally validate an intensive care unit (ICU) mortality prediction model that was created using the Ontario Critical Care Information System (CCIS), which includes the Multiple Organ Dysfunction Score (MODS). METHODS:We applied the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) recommendations to a prospective longitudinal cohort of patients discharged between 1 July 2015 and 31 December 31 2016 from 90 adult level-3 critical care units in Ontario. We used multivariable logistic regression with measures of discrimination, calibration-in-the-large, calibration slope, and flexible calibration plots to compare prediction model performance of the entire data set and for each ICU subtype. RESULTS:Among 121,201 CCIS records with ICU mortality of 11.3%, the C-statistic for the validation data set was 0.805. The C-statistic ranged from 0.775 to 0.846 among the ICU subtypes. After intercept recalibration to adjust the baseline risk, the mean predicted risk of death matched actual ICU mortality. The calibration slope was close to 1 with all CCIS data and ICU subtypes of cardiovascular and community hospitals with low ventilation rates. Calibration slopes significantly less than 1 were found for ICUs in teaching hospitals and community hospitals with high ventilation rates whereas coronary care units had a calibration slope significantly higher than 1. Calibration plots revealed over-prediction in high risk groups to a varying degree across all cohorts. CONCLUSIONS:A risk prediction model primarily based on the MODS shows reproducibility and transportability after intercept recalibration. Risk adjusting models that use existing and feasible data collection can support performance measurement at the individual ICU level.

SUBMITTER: Priestap F

PROVIDER: S-EPMC7223438 | biostudies-literature | 2020 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

External validation of a prognostic model for intensive care unit mortality: a retrospective study using the Ontario Critical Care Information System.

Priestap Fran F Kao Raymond R Martin Claudio M CM

Canadian journal of anaesthesia = Journal canadien d'anesthesie 20200507 8

<h4>Purpose</h4>To externally validate an intensive care unit (ICU) mortality prediction model that was created using the Ontario Critical Care Information System (CCIS), which includes the Multiple Organ Dysfunction Score (MODS).<h4>Methods</h4>We applied the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) recommendations to a prospective longitudinal cohort of patients discharged between 1 July 2015 and 31 December 31 2016 from 90 adult ...[more]

PMID: 32383124

Similar Datasets

Project description:BackgroundEarly prediction model of hemodynamic instability has the potential to improve the critical care, whereas limited external validation on the generalizability. We aimed to independently validate the Hemodynamic Stability Index (HSI), a multi-parameter machine learning model, in predicting hemodynamic instability in Asian patients.MethodHemodynamic instability was marked by using inotropic, vasopressor, significant fluid therapy, and/or blood transfusions. This retrospective study included among 15,967 ICU patients who aged 20 years or older (not included 20 years) and stayed in ICU for more than 6 h admitted to Taipei Veteran General Hospital (TPEVGH) between January 1, 2010, and March 31, 2020, of whom hemodynamic instability occurred in 3053 patients (prevalence = 19%). These patients in unstable group received at least one intervention during their ICU stays, and the HSI score of both stable and unstable group was calculated in every hour before intervention. The model performance was assessed using the area under the receiver operating characteristic curve (AUROC) and was compared to single indicators like systolic blood pressure (SBP) and shock index. The hemodynamic instability alarm was set by selecting optimal threshold with high sensitivity, acceptable specificity, and lead time before intervention was calculated to indicate when patients were firstly identified as high risk of hemodynamic instability.ResultsThe AUROC of HSI was 0.76 (95% CI, 0.75-0.77), which performed significantly better than shock Index (0.7; 95% CI, 0.69-0.71) and SBP (0.69; 95% CI, 0.68-0.70). By selecting 0.7 as a threshold, HSI predicted 72% of all 3053 patients who received hemodynamic interventions with 67% in specificity. Time-varying results also showed that HSI score significantly outperformed single indicators even up to 24 h before intervention. And 95% unstable patients can be identified more than 5 h in advance.ConclusionsThe HSI has acceptable discrimination but underestimates the risk of stable patients in predicting the onset of hemodynamic instability in an external cohort.

Project description:BackgroundNo data is available on delirium prediction models in the cardiac intensive care unit (CICU), although preexisting delirium prediction models [PREdiction of DELIRium in ICu patients (PRE-DELIRIC) and Early PREdiction of DELIRium in ICu patients (E-PRE-DELIRIC)] were developed and validated based on a population admitted to the general intensive care unit (ICU). Therefore, we externally validated the usefulness of the PRE-DELIRIC and E-PRE-DELIRIC models and compared their predictive performance in patients admitted to the CICU.MethodsA total of 2,724 patients admitted to the CICU were enrolled between September 2012 and December 2018. Delirium was defined as at least one positive Confusion Assessment Method for the ICU (CAM-ICU) which was screened at least once every 8 h. The PRE-DELIRIC value was calculated within 24 h of CICU admission, and the E-PRE-DELIRIC value was calculated at CICU admission. The predictive performance of the models was evaluated by using the area under the receiver operating characteristic (AUROC) curve, and the calibration slope was assessed graphically by plotting.ResultsDelirium occurred in 677 patients (24.8%) when the patients were assessed thrice daily until 7 days of the CICU stay. The AUROC curve for the prediction of delirium was significantly greater for PRE-DELIRIC values [0.84, 95% confidence interval (CI): 0.82-0.86] than for E-PRE-DELIRIC values (0.79, 95% CI: 0.77-0.80) [z score of -6.24 (p < 0.001)]. Net reclassification improvement for the prediction of delirium increased by 0.27 (95% CI: 0.21-0.32, p < 0.001). Calibration was acceptable in the PRE-DELIRIC model (Hosmer-Lemeshow p = 0.170) but not in the E-PRE-DELIRIC model (Hosmer-Lemeshow p < 0.001).ConclusionAlthough both models have good predictive performance for the development of delirium, even in critically ill cardiac patients, the performance of the PRE-DELIRIC model might be superior to that of the E-PRE-DELIRIC model. Further studies are required to confirm our results and design a specific delirium prediction model for CICU patients.

Project description:Objectives backgroundTo externally validate clinical prediction models that aim to predict progression to invasive ventilation or death on the ICU in patients admitted with confirmed COVID-19 pneumonitis.DesignSingle-center retrospective external validation study.Data sourcesRoutinely collected healthcare data in the ICU electronic patient record. Curated data recorded for each ICU admission for the purposes of the U.K. Intensive Care National Audit and Research Centre (ICNARC).SettingThe ICU at Manchester Royal Infirmary, Manchester, United Kingdom.PatientsThree hundred forty-nine patients admitted to ICU with confirmed COVID-19 Pneumonitis, older than 18 years, from March 1, 2020, to February 28, 2022. Three hundred two met the inclusion criteria for at least one model. Fifty-five of the 349 patients were admitted before the widespread adoption of dexamethasone for the treatment of severe COVID-19 (pre-dexamethasone patients).OutcomesAbility to be externally validated, discriminate, and calibrate.MethodsArticles meeting the inclusion criteria were identified, and those that gave sufficient details on predictors used and methods to generate predictions were tested in our cohort of patients, which matched the original publications' inclusion/exclusion criteria and endpoint.ResultsThirteen clinical prediction articles were identified. There was insufficient information available to validate models in five of the articles; a further three contained predictors that were not routinely measured in our ICU cohort and were not validated; three had performance that was substantially lower than previously published (range C-statistic = 0.483-0.605 in pre-dexamethasone patients and C = 0.494-0.564 among all patients). One model retained its discriminative ability in our cohort compared with previously published results (C = 0.672 and 0.686), and one retained performance among pre-dexamethasone patients but was poor in all patients (C = 0.793 and 0.596). One model could be calibrated but with poor performance.ConclusionsOur findings, albeit from a single center, suggest that the published performance of COVID-19 prediction models may not be replicated when translated to other institutions. In light of this, we would encourage bedside intensivists to reflect on the role of clinical prediction models in their own clinical decision-making.

Project description:Objectives: Multiple risk factors have been described to be related to external ventricular drain (EVD) associated infections, with results varying between studies. Former studies were limited by a non-uniform definition of EVD associated infection, thus complicating a comparison between studies. In this regard, we assessed risk factors promoting EVD associated infections and propose a modified practice-oriented definition of EVD associated infections. Methods: We performed a retrospective, single-center study on patients who were treated with an EVD, at the neurosurgical intensive care unit (ICU) at a tertiary center between 2008 and 2019. Based on microbiological findings and laboratory results, patients were assigned into an infection and a non-infection group. Patient characteristics and potential risk factors were compared between the two groups (p < 0.05). Receiver operating characteristics (ROC) for significant clinical, serum laboratory and cerebrospinal fluid (CSF) parameters were calculated. Results: In total, 396 patients treated with an EVD were included into the study with a mean age of 54.3 (range: 18-89) years. EVD associated infections were observed in 32 (8.1%) patients. EVD insertion at another hospital (OR 3.86), and an increased CSF sampling frequency of more than every third day (OR 12.91) were detected as major risk factors for an EVD associated infection. The indication for EVD insertion, surgeon's experience, the setting of EVD insertion (ICU vs. operating room) and the operating time did not show any significant differences between the two groups. Furthermore, ROC analysis showed that clinical, serum laboratory and CSF parameters did not provide specific prediction of EVD associated infections (specificity 44.4%). This explains the high overtreatment rate in our cohort with the majority of our patients who received intrathecal vancomycin (63.3%), having either negative microbiological results (n = 12) or were defined as contaminations (n = 7). Conclusions: Since clinical parameters and blood analyzes are not very predictive to detect EVD associated infections in neurosurgical patients, sequential but not too frequent microbiological and laboratory analysis of CSF are still necessary. Furthermore, we propose a uniform classification for EVD associated infections to allow comparability between studies and to sensitize the treating physician in determining the right treatment.

Project description:BackgroundThis study aimed to develop prognostic models for predicting the need for invasive mechanical ventilation (IMV) in intensive care unit (ICU) patients with COVID-19 and compare their performance with the Respiratory rate-OXygenation (ROX) index.MethodsA retrospective cohort study was conducted using data collected between March 2020 and August 2021 at three hospitals in Rio de Janeiro, Brazil. ICU patients aged 18 years and older with a diagnosis of COVID-19 were screened. The exclusion criteria were patients who received IMV within the first 24 h of ICU admission, pregnancy, clinical decision for minimal end-of-life care and missing primary outcome data. Clinical and laboratory variables were collected. Multiple logistic regression analysis was performed to select predictor variables. Models were based on the lowest Akaike Information Criteria (AIC) and lowest AIC with significant p values. Assessment of predictive performance was done for discrimination and calibration. Areas under the curves (AUC)s were compared using DeLong's algorithm. Models were validated externally using an international database.ResultsOf 656 patients screened, 346 patients were included; 155 required IMV (44.8%), 191 did not (55.2%), and 207 patients were male (59.8%). According to the lowest AIC, arterial hypertension, diabetes mellitus, obesity, Sequential Organ Failure Assessment (SOFA) score, heart rate, respiratory rate, peripheral oxygen saturation (SpO2), temperature, respiratory effort signals, and leukocytes were identified as predictors of IMV at hospital admission. According to AIC with significant p values, SOFA score, SpO2, and respiratory effort signals were the best predictors of IMV; odds ratios (95% confidence interval): 1.46 (1.07-2.05), 0.81 (0.72-0.90), 9.13 (3.29-28.67), respectively. The ROX index at admission was lower in the IMV group than in the non-IMV group (7.3 [5.2-9.8] versus 9.6 [6.8-12.9], p < 0.001, respectively). In the external validation population, the area under the curve (AUC) of the ROX index was 0.683 (accuracy 63%), the AIC model showed an AUC of 0.703 (accuracy 69%), and the lowest AIC model with significant p values had an AUC of 0.725 (accuracy 79%).ConclusionsIn the development population of ICU patients with COVID-19, SOFA score, SpO2, and respiratory effort signals predicted the need for IMV better than the ROX index. In the external validation population, although the AUCs did not differ significantly, the accuracy was higher when using SOFA score, SpO2, and respiratory effort signals compared to the ROX index. This suggests that these variables may be more useful in predicting the need for IMV in ICU patients with COVID-19.ClinicaltrialsGov identifierNCT05663528.

Dataset Information

External validation of a prognostic model for intensive care unit mortality: a retrospective study using the Ontario Critical Care Information System.

Publications

External validation of a prognostic model for intensive care unit mortality: a retrospective study using the Ontario Critical Care Information System.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets