Dataset Information

Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance.

ABSTRACT: There are few comparisons among the most recent versions of the major adult ICU prognostic systems (APACHE [Acute Physiology and Chronic Health Evaluation] IV, Simplified Acute Physiology Score [SAPS] 3, Mortality Probability Model [MPM]0III). Only MPM0III includes resuscitation status as a predictor.We assessed the discrimination, calibration, and overall performance of the models in 2,596 patients in three ICUs at our tertiary referral center in 2006. For APACHE and SAPS, the analyses were repeated with and without inclusion of resuscitation status as a predictor variable.Of the 2,596 patients studied, 283 (10.9%) died before hospital discharge. The areas under the curve (95% CI) of the models for prediction of hospital mortality were 0.868 (0.854-0.880), 0.861 (0.847-0.874), 0.801 (0.785-0.816), and 0.721 (0.704-0.738) for APACHE III, APACHE IV, SAPS 3, and MPM0III, respectively. The Hosmer-Lemeshow statistics for the models were 33.7, 31.0, 36.6, and 21.8 for APACHE III, APACHE IV, SAPS 3, and MPM0III, respectively. Each of the Hosmer-Lemeshow statistics generated P values < .05, indicating poor calibration. Brier scores for the models were 0.0771, 0.0749, 0.0890, and 0.0932, respectively. There were no significant differences between the discriminative ability or the calibration of APACHE or SAPS with and without “do not resuscitate” status.APACHE III and IV had similar discriminatory capability and both were better than SAPS 3, which was better than MPM0III. The calibrations of the models studied were poor. Overall, models with more predictor variables performed better than those with fewer. The addition of resuscitation status did not improve APACHE III or IV or SAPS 3 prediction.

SUBMITTER: Keegan MT

PROVIDER: S-EPMC3465106 | biostudies-other | 2012 Oct

REPOSITORIES: biostudies-other

ACCESS DATA

Similar Datasets

Project description:BackgroundFunctional status (FS) before intensive care unit (ICU) admission is associated with short-term and long-term outcomes among critically ill patients. However, measures of FS are generally not integrated into ICU-specific mortality prediction models.MethodsThis retrospective cohort study used prospectively collected data from 9638 consecutive patients admitted to a single ICU between 1 October 2005 and 30 September 2015. For each ICU admission, FS was prospectively determined and classified into three discrete categories based on performance of basic daily living activities (FS1 - fully independent; FS2 - partly dependent; FS3 - completely dependent). We prospectively calculated Acute Physiology and Chronic Health Evaluation (APACHE) IV predicted mortality percentage (APIV PM) for each admission and calculated observed-expected mortality ratios (OEMR), stratified by FS category and APIV PM. We calculated area under the receiver operator characteristic curve (AUC) for APIV PM and mortality for the entire cohort and the three FS categories.ResultsPatients had a median (IQR) age of 67 (52-80) years and mean (SD) APIV PM was 18.3% (24.3%). Of these, 7714 (80.0%) were classified as FS1, 1728 (17.9%) as FS2 and 196 (2.0%) as FS3. FS1 patients were younger, had less comorbid disease, and lower APIV PM compared to FS2 and FS3. The OEMR were significantly lower for FS1 (0.67) than FS2 (0.93) or FS3 (0.90) (p < 0.0001 for both comparisons). Among patients with APIV PM 0-10%, 10-25%, 25-50% and ≥50% the OEMR for FS1 were 0.33, 0.49, 0.61 and 0.86. The AUC (95% CI) for APIV PM and mortality for FS1, FS2 and FS3 were 0.924 (0.914-0.933), 0.837 (0.816-0.858) and 0.775 (0.705-0.8456), respectively (p < 0.001 for each comparison). Multivariable analysis demonstrated that FS2 (OR 2.18 (1.84-2.57) (p < 0.0001)) and FS3 (OR 1.99 (1.34-2.96) (p = 0.0006)) were independently associated with increased risk of mortality.ConclusionsBaseline FS prior to critical illness is a strong independent predictor of mortality and impacts the relationship between observed and APIV PM in those with lower illness severity. Future iterations of mortality prediction models should integrate a baseline measure of FS to improve performance.

Project description:BackgroundSimulation is increasingly being used in postgraduate medical education as an opportunity for competency assessment. However, there is limited direct evidence that supports performance in the simulation lab as a surrogate of workplace-based clinical performance for non-procedural tasks such as resuscitation in the emergency department (ED). We sought to directly compare entrustment scoring of resident performance in the simulation environment to clinical performance in the ED.MethodsThe resuscitation assessment tool (RAT) was derived from the previously implemented and studied Queen's simulation assessment tool (QSAT) via a modified expert review process. The RAT uses an anchored global assessment scale to generate an entrustment score and narrative comments. Emergency medicine (EM) residents were assessed using the RAT on cases in simulation-based examinations and in the ED during resuscitation cases from July 2016 to June 2017. Resident mean entrustment scores were compared using Pearson's correlation coefficient to determine the relationship between entrustment in simulation cases and in the ED. Inductive thematic analysis of written commentary was conducted to compare workplace-based with simulation-based feedback.ResultsThere was a moderate, positive correlation found between mean entrustment scores in the simulated and workplace-based settings, which was statistically significant (r = 0.630, n = 17, p < 0.01). Further, qualitative analysis demonstrated overall management and leadership themes were more common narratives in the workplace, while more specific task-based feedback predominated in the simulation-based assessment. Both workplace-based and simulation-based narratives frequently commented on communication skills.ConclusionsIn this single-center study with a limited sample size, assessment of residents using entrustment scoring in simulation settings was demonstrated to have a moderate positive correlation with assessment of resuscitation competence in the workplace. This study suggests that resuscitation performance in simulation settings may be an indicator of competence in the clinical setting. However, multiple factors contribute to this complicated and imperfect relationship. It is imperative to consider narrative comments in supporting the rationale for numerical entrustment scores in both settings and to include both simulation and workplace-based assessment in high-stakes decisions of progression.

Project description:BackgroundSeverity scoring systems are inherent to ICU practice for multiple purposes. Acute Physiology and Chronic Health Evaluation (APACHE) scoring systems are designed for ICU mortality prediction. This study aims to validate APACHE IV in COVID-19 patients admitted to the ICU.MethodsAll COVID-19 patients admitted to the ICU between March 13, 2020, and October 17, 2020, were retrospectively analyzed. APACHE II and APACHE IV scores as well as SOFA scores were calculated within 24 hours after admission. Discrimination for mortality of all three scoring systems was assessed by receiver operating characteristic curves. Youden index was determined for the scoring system with the best discriminative performance. The Hosmer-Lemeshow goodness-of-fit test was used to assess calibration. All analyses were performed for both the overall population as in a subgroup treated with anti-Xa adjusted dosages of LMWHs.Results116 patients were admitted to our ICU during the study period. 13 were excluded for various reasons, leaving 103 patients in the statistical analysis of the overall population. 57 patients were treated with anti-Xa adjusted prophylactic dosages of LMWH and were supplementary analyzed in a subgroup analysis. APACHE IV had the best discriminative power of the three scoring systems, both in the overall population (APACHE IV ROC AUC 0.67 vs. APACHE II ROC AUC 0.63) as in the subgroup (APACHE IV ROC AUC 0.82 vs. APACHE II ROC AUC 0.7). This model exhibits good calibration. Hosmer-Lemeshow p values for APACHE IV were 0.9234 for the overall population and 0.8017 for the subgroup. Calibration p values of the APACHE II score were 0.1394 and 0.6475 for the overall versus subgroup, respectively.ConclusionsAPACHE IV provided the best discrimination and calibration of the considered scoring systems in critically ill COVID-19 patients, both in the overall group and in the subgroup with anti-Xa adjusted LMWH doses. Only in the subgroup analysis, discriminative abilities of APACHE IV were very good. This trial is registered with NCT04713852.

Dataset Information

Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets