Dataset Information

Process monitoring in intensive care with the use of cumulative expected minus observed mortality and risk-adjusted P charts.

ABSTRACT: INTRODUCTION: A health care system is a complex adaptive system. The effect of a single intervention, incorporated into a complex clinical environment, may be different from that expected. A national database such as the Intensive Care National Audit & Research Centre (ICNARC) Case Mix Programme in the UK represents a centralised monitoring, surveillance and reporting system for retrospective quality and comparative audit. This can be supplemented with real-time process monitoring at a local level for continuous process improvement, allowing early detection of the impact of both unplanned and deliberately imposed changes in the clinical environment. METHODS: Demographic and UK Acute Physiology and Chronic Health Evaluation II (APACHE II) data were prospectively collected on all patients admitted to a UK regional hospital between 1 January 2003 and 30 June 2004 in accordance with the ICNARC Case Mix Programme. We present a cumulative expected minus observed (E-O) plot and the risk-adjusted p chart as methods of continuous process monitoring. We describe the construction and interpretation of these charts and show how they can be used to detect planned or unplanned organisational process changes affecting mortality outcomes. RESULTS: Five hundred and eighty-nine adult patients were included. The overall death rate was 0.78 of predicted. Calibration showed excess survival in ranges above 30% risk of death. The E-O plot confirmed a survival above that predicted. Small transient variations were seen in the slope that could represent random effects, or real but transient changes in the quality of care. The risk-adjusted p chart showed several observations below the 2 SD control limits of the expected mortality rate. These plots provide rapid analysis of risk-adjusted performance suitable for local application and interpretation. The E-O chart provided rapid easily visible feedback of changes in risk-adjusted mortality, while the risk-adjusted p chart allowed statistical evaluation. CONCLUSION: Local analysis of risk-adjusted mortality data with an E-O plot and a risk-adjusted p chart is feasible and allows the rapid detection of changes in risk-adjusted outcome of intensive care patients. This complements the centralised national database, which is more archival and comparative in nature.

SUBMITTER: Cockings JG

PROVIDER: S-EPMC1550849 | biostudies-other | 2006 Feb

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Process monitoring in intensive care with the use of cumulative expected minus observed mortality and risk-adjusted P charts.

Cockings Jerome G L JG Cook David A DA Iqbal Rehana K RK

Critical care (London, England) 20060201 1

<h4>Introduction</h4>A health care system is a complex adaptive system. The effect of a single intervention, incorporated into a complex clinical environment, may be different from that expected. A national database such as the Intensive Care National Audit & Research Centre (ICNARC) Case Mix Programme in the UK represents a centralised monitoring, surveillance and reporting system for retrospective quality and comparative audit. This can be supplemented with real-time process monitoring at a lo ...[more]

PMID: 16507152

Similar Datasets

Project description:ObjectivesTo determine whether an innovative graphical tool for accurate measurement of individual surgeon performance metrics, adjusted for both surgeon-specific and patient-specific factors, significantly alters interpretation of performance data.DesignRetrospective analysis of all total knee replacements (TKRs) conducted at the host institution between 1996 and 2009. The database was randomly divided into training and testing datasets. Using multivariate generalised estimating equation regression models, the training dataset enabled generation of patient-risk and surgeon-experience adjustment factors. To simulate prospective monitoring of individual surgeon outcomes, the testing dataset was mapped on control charts. Weighted ? statistics were calculated to measure the agreement between patient-risk adjusted and fully adjusted control charts.SettingTertiary care academic hospital.ParticipantsAll patients undergoing TKR at the host institution 1996-2009.Main outcome measureOperative efficiency.Results5313 procedures were analysed. Adjusted control charts were generated using a training dataset comprising 3756 procedures performed by 13 surgeons. The operative time gradually declined by 121 min with 25 years of experience (p<0.0001). Charts were tested by monitoring four other surgeons, performing an average of 389 procedures each. Adjustment for surgeon experience significantly altered the interpretation of operative efficiency (?=0.29 (95% CI 0.11 to 0.47)), and enhanced assessment of a surgeon's improvement or diminishment in efficiency over time. Specifically, experience adjustment inverted the interpretation of surgeon efficiency from above average to below average, or from improving to declining performance.ConclusionsAdjustment for surgeon experience is necessary for accurate interpretation of metrics over the course of a surgeon's career. Patient-adjusted and surgeon-adjusted control charts provide an accurate method of monitoring individual operative efficiency.

Project description:ImportanceNational surgical quality improvement programs lack tools for early detection of quality or safety concerns, which risks patient safety because of delayed recognition of poor performance.ObjectiveTo compare the risk-adjusted cumulative sum (CUSUM) with episodic evaluation for early detection of hospitals with excess perioperative mortality.Design, setting, and participantsNational, observational, hospital-level, comparative effectiveness study of 697 566 patients. Identification of hospitals with excess, risk-adjusted, quarterly 30-day mortality using observed to expected ratios (ie, current criterion standard in the Veterans Affairs Surgical Quality Improvement Program) was compared with the risk-adjusted CUSUM. Patients included in the study underwent a noncardiac operation at a Veterans Affairs hospital, had a record in the Veterans Affairs Surgical Quality Improvement Program (January 1, 2011, through December 31, 2016), and were aged 18 years or older.Main outcome and measureNumber of hospitals identified as having excess risk-adjusted 30-day mortality.ResultsThe cohort included 697 566 patients treated at 104 hospitals across 24 quarters. The mean (SD) age was 60.9 (13.2) years, 91.4% were male, and 8.6% were female. For each hospital, the median number of quarters detected with observed to expected ratios, at least 1 CUSUM signal, and more than 1 CUSUM signal was 2 quarters (IQR, 1-4 quarters), 8 quarters (IQR, 4-11 quarters), and 3 quarters (IQR, 1-4 quarters), respectively. During 2496 total quarters of data, outlier hospitals were identified 33.3% of the time (830 quarters) with at least 1 CUSUM signal within a quarter, 12.5% (311 quarters) with more than 1 CUSUM signal, and 11.0% (274 quarters) with observed to expected ratios at the end of the quarter. The CUSUM detection occurred a median of 49 days (IQR, 25-63 days) before observed to expected ratio reporting (1 signal, 35 days [IQR, 17-54 days]; 2 signals, 49 days [IQR, 26-61 days]; 3 signals, 58 days [IQR, 44-69 days]; ≥4 signals, 49 days [IQR, 42-69 days]; trend test, P < .001). Of 274 hospital quarters detected with observed to expected ratios, 72.6% (199) were concurrently detected by at least 1 CUSUM signal vs 42.7% (117) by more than 1 CUSUM signal. There was a dose-response relationship between the number of CUSUM signals in a quarter and the median observed to expected ratio (0 signals, 0.63; 1 signal, 1.28; 2 signals, 1.58; 3 signals, 2.08; ≥4 signals, 2.49; trend test, P < .001).ConclusionsThis study found that with CUSUM, hospitals with excess perioperative mortality can be identified well in advance of standard end-of-quarter reporting, which suggests episodic evaluation strategies fail to detect out-of-control processes and place patients at risk. Continuous performance evaluation tools should be adopted in national quality improvement programs to prevent avoidable patient harm.

Project description:IntroductionA common quality indicator for monitoring and comparing hospitals is based on death within 30 days of admission. An important use is to determine whether a hospital has higher or lower mortality than other hospitals. Thus, the ability to identify such outliers correctly is essential. Two approaches for detection are: 1) calculating the ratio of observed to expected number of deaths (OE) per hospital and 2) including all hospitals in a logistic regression (LR) comparing each hospital to a form of average over all hospitals. The aim of this study was to compare OE and LR with respect to correctly identifying 30-day mortality outliers. Modifications of the methods, i.e., variance corrected approach of OE (OE-Faris), bias corrected LR (LR-Firth), and trimmed mean variants of LR and LR-Firth were also studied.Materials and methodsTo study the properties of OE and LR and their variants, we performed a simulation study by generating patient data from hospitals with known outlier status (low mortality, high mortality, non-outlier). Data from simulated scenarios with varying number of hospitals, hospital volume, and mortality outlier status, were analysed by the different methods and compared by level of significance (ability to falsely claim an outlier) and power (ability to reveal an outlier). Moreover, administrative data for patients with acute myocardial infarction (AMI), stroke, and hip fracture from Norwegian hospitals for 2012-2014 were analysed.ResultsNone of the methods achieved the nominal (test) level of significance for both low and high mortality outliers. For low mortality outliers, the levels of significance were increased four- to fivefold for OE and OE-Faris. For high mortality outliers, OE and OE-Faris, LR 25% trimmed and LR-Firth 10% and 25% trimmed maintained approximately the nominal level. The methods agreed with respect to outlier status for 94.1% of the AMI hospitals, 98.0% of the stroke, and 97.8% of the hip fracture hospitals.ConclusionWe recommend, on the balance, LR-Firth 10% or 25% trimmed for detection of both low and high mortality outliers.

Dataset Information

Process monitoring in intensive care with the use of cumulative expected minus observed mortality and risk-adjusted P charts.

Publications

Process monitoring in intensive care with the use of cumulative expected minus observed mortality and risk-adjusted P charts.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets