Dataset Information

Patient- and surgeon-adjusted control charts for monitoring performance.

ABSTRACT:

Objectives

To determine whether an innovative graphical tool for accurate measurement of individual surgeon performance metrics, adjusted for both surgeon-specific and patient-specific factors, significantly alters interpretation of performance data.

Design

Retrospective analysis of all total knee replacements (TKRs) conducted at the host institution between 1996 and 2009. The database was randomly divided into training and testing datasets. Using multivariate generalised estimating equation regression models, the training dataset enabled generation of patient-risk and surgeon-experience adjustment factors. To simulate prospective monitoring of individual surgeon outcomes, the testing dataset was mapped on control charts. Weighted ? statistics were calculated to measure the agreement between patient-risk adjusted and fully adjusted control charts.

Setting

Tertiary care academic hospital.

Participants

All patients undergoing TKR at the host institution 1996-2009.

Main outcome measure

Operative efficiency.

Results

5313 procedures were analysed. Adjusted control charts were generated using a training dataset comprising 3756 procedures performed by 13 surgeons. The operative time gradually declined by 121 min with 25 years of experience (p<0.0001). Charts were tested by monitoring four other surgeons, performing an average of 389 procedures each. Adjustment for surgeon experience significantly altered the interpretation of operative efficiency (?=0.29 (95% CI 0.11 to 0.47)), and enhanced assessment of a surgeon's improvement or diminishment in efficiency over time. Specifically, experience adjustment inverted the interpretation of surgeon efficiency from above average to below average, or from improving to declining performance.

Conclusions

Adjustment for surgeon experience is necessary for accurate interpretation of metrics over the course of a surgeon's career. Patient-adjusted and surgeon-adjusted control charts provide an accurate method of monitoring individual operative efficiency.

SUBMITTER: Maruthappu M

PROVIDER: S-EPMC3902330 | biostudies-literature | 2014 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Patient- and surgeon-adjusted control charts for monitoring performance.

Maruthappu Mahiben M Carty Matthew J MJ Lipsitz Stuart R SR Wright John J Orgill Dennis D Duclos Antoine A

BMJ open 20140116 1

<h4>Objectives</h4>To determine whether an innovative graphical tool for accurate measurement of individual surgeon performance metrics, adjusted for both surgeon-specific and patient-specific factors, significantly alters interpretation of performance data.<h4>Design</h4>Retrospective analysis of all total knee replacements (TKRs) conducted at the host institution between 1996 and 2009. The database was randomly divided into training and testing datasets. Using multivariate generalised estimati ...[more]

PMID: 24440796

Similar Datasets

Project description:INTRODUCTION: A health care system is a complex adaptive system. The effect of a single intervention, incorporated into a complex clinical environment, may be different from that expected. A national database such as the Intensive Care National Audit & Research Centre (ICNARC) Case Mix Programme in the UK represents a centralised monitoring, surveillance and reporting system for retrospective quality and comparative audit. This can be supplemented with real-time process monitoring at a local level for continuous process improvement, allowing early detection of the impact of both unplanned and deliberately imposed changes in the clinical environment. METHODS: Demographic and UK Acute Physiology and Chronic Health Evaluation II (APACHE II) data were prospectively collected on all patients admitted to a UK regional hospital between 1 January 2003 and 30 June 2004 in accordance with the ICNARC Case Mix Programme. We present a cumulative expected minus observed (E-O) plot and the risk-adjusted p chart as methods of continuous process monitoring. We describe the construction and interpretation of these charts and show how they can be used to detect planned or unplanned organisational process changes affecting mortality outcomes. RESULTS: Five hundred and eighty-nine adult patients were included. The overall death rate was 0.78 of predicted. Calibration showed excess survival in ranges above 30% risk of death. The E-O plot confirmed a survival above that predicted. Small transient variations were seen in the slope that could represent random effects, or real but transient changes in the quality of care. The risk-adjusted p chart showed several observations below the 2 SD control limits of the expected mortality rate. These plots provide rapid analysis of risk-adjusted performance suitable for local application and interpretation. The E-O chart provided rapid easily visible feedback of changes in risk-adjusted mortality, while the risk-adjusted p chart allowed statistical evaluation. CONCLUSION: Local analysis of risk-adjusted mortality data with an E-O plot and a risk-adjusted p chart is feasible and allows the rapid detection of changes in risk-adjusted outcome of intensive care patients. This complements the centralised national database, which is more archival and comparative in nature.

Project description:BACKGROUND:Continuous monitoring of surgical outcomes after joint replacement is needed to detect which brands' components have a higher than expected failure rate and are therefore no longer recommended to be used in surgical practice. We developed a monitoring method based on cumulative sum (CUSUM) chart specifically for this application. METHODS:Our method entails the use of the competing risks model with the Weibull and the Gompertz hazard functions adjusted for observed covariates to approximate the baseline time-to-revision and time-to-death distributions, respectively. The correlated shared frailty terms for competing risks, corresponding to the operating unit, are also included in the model. A bootstrap-based boundary adjustment is then required for risk-adjusted CUSUM charts to guarantee a given probability of the false alarm rates. We propose a method to evaluate the CUSUM scores and the adjusted boundary for a survival model with the shared frailty terms. We also introduce a unit performance quality score based on the posterior frailty distribution. This method is illustrated using the 2003-2012 hip replacement data from the UK National Joint Registry (NJR). RESULTS:We found that the best model included the shared frailty for revision but not for death. This means that the competing risks of revision and death are independent in NJR data. Our method was superior to the standard NJR methodology. For one of the two monitored components, it produced alarms four years before the increased failure rate came to the attention of the UK regulatory authorities. The hazard ratios of revision across the units varied from 0.38 to 2.28. CONCLUSIONS:An earlier detection of failure signal by our method in comparison to the standard method used by the NJR may be explained by proper risk-adjustment and the ability to accommodate time-dependent hazards. The continuous monitoring of hip replacement outcomes should include risk adjustment at both the individual and unit level.

Project description:BackgroundRecurrent mood episodes and subsyndromal mood instability cause substantial disability in patients with bipolar disorder. Early identification of mood episodes enabling timely mood stabilization is an important clinical goal. This study investigates the ability of control chart methodology to predict manic and/or depressive episodes by applying Shewhart's control rules to weekly self-reported scores from mania and depression questionnaires.MethodsShewhart's control rules were applied to weekly self-reported scores from the Altman Self-Rating Mania Scale (ASRM) and the Quick Inventory of Depressive Symptomatology-Self-Report (QIDS) collected from 2001 to 2012 as part of the OXTEXT programme. Manic and depressive episodes were defined as an ASRM score ≥ 10 or a QIDS score ≥ 15, respectively. An episode-free run-in period of eight consecutive weeks without an episode of either type was used to calibrate control charts. Shewhart's rules were then applied to follow-up data. Their sensitivity and positive predictive value for predicting manic or depressive episodes within the next 4 weeks were calculated focusing on the first episode. Secondary analyses varying control chart type, length of episode-free run-in period, time frames to evaluate diagnostic accuracy, thresholds defining either manic or depressive episodes, and missing data methods were performed.ResultsData from 146 participants (37% men) were included. The mean age was 43.4 (SD = 13.3) years. The median follow-up was 10 (IQR 5-40) weeks for mania and 10 (IQR 5-23) weeks for depression. A total of 53 (36%) participants had a manic episode and 67 (46%) had a depressive episode. For manic episodes, the sensitivity and positive predictive value of Shewhart's control rules were 30% (95% CI 19-45%) and 7% (95% CI 5-9%), and for depressive episodes, 33% (95% CI 22-46%) and 9% (95% CI 6-12%), respectively. Results from secondary analyses were similar to these.ConclusionsTele-monitoring with control rules has the potential to predict about one-third of manic or depressive episodes before they occur, at the cost of a high false positive rate. Given the severe consequences of manic and depressive episodes, this trade-off may be desirable.

Project description:ObjectiveTo determine the effect of introducing prospective monitoring of outcomes using control charts and regular feedback on indicators to surgical teams on major adverse events in patients.DesignNational, parallel, cluster randomised trial embedding a difference-in-differences analysis.Setting40 surgical departments of hospitals across France.Participants155 362 adults who underwent digestive tract surgery. 20 of the surgical departments were randomised to prospective monitoring of outcomes using control charts with regular feedback on indicators (intervention group) and 20 to usual care only (control group).InterventionsProspective monitoring of outcomes using control charts, provided in sets quarterly, with regular feedback on indicators (intervention hospitals). To facilitate implementation of the programme, study champion partnerships were established at each site, comprising a surgeon and another member of the surgical team (surgeon, anaesthetist, or nurse), and were trained to conduct team meetings, display posters in operating rooms, maintain a logbook, and devise an improvement plan.Main outcome measuresThe primary outcome was a composite of major adverse events (inpatient death, intensive care stay, reoperation, and severe complications) within 30 days after surgery. Changes in surgical outcomes were compared before and after implementation of the programme between intervention and control hospitals, with adjustment for patient mix and clustering.Results75 047 patients were analysed in the intervention hospitals (37 579 before and 37 468 after programme implementation) versus 80 315 in the control hospitals (41 548 and 38 767). After introduction of the control chart, the absolute risk of a major adverse event was reduced by 0.9% (95% confidence interval 0.4% to 1.4%) in intervention compared with control hospitals, corresponding to 114 patients (70 to 280) who needed to receive the intervention to prevent one major adverse event. A significant decrease in major adverse events (adjusted ratio of odds ratios 0.89, 95% confidence interval 0.83 to 0.96), patient death (0.84, 0.71 to 0.99), and intensive care stay (0.85, 0.76 to 0.94) was found in intervention compared with control hospitals. The same trend was observed for reoperation (0.91, 0.82 to 1.00), whereas severe complications remained unchanged (0.96, 0.87 to 1.07). Among the intervention hospitals, the effect size was proportional to the degree of control chart implementation witnessed. Highly compliant hospitals experienced a more important reduction in major adverse events (0.84, 0.77 to 0.92), patient death (0.78, 0.63 to 0.97), intensive care stay (0.76, 0.67 to 0.87), and reoperation (0.84, 0.74 to 0.96).ConclusionsThe implementation of control charts with feedback on indicators to surgical teams was associated with concomitant reductions in major adverse events in patients. Understanding variations in surgical outcomes and how to provide safe surgery is imperative for improvements.Trial registrationClinicalTrials.gov NCT02569450.

Project description:BackgroundAt the end of December 2019, the world in general and Wuhan, the industrial hub of China, in particular, experienced the COVID-19 pandemic. Massive increment of cases and deaths occurred in China and 209 countries in Europe, America, Australia, Asia and Pakistan. Pakistan was first hit by COVID-19 when a case was reported in Karachi on 26 February 2020. Several methods were presented to model the death rate due to the COVID-19 pandemic and to forecast the pinnacle of reported deaths. Still, these methods were not used in identifying the first day when Pakistan enters or exits the early exponential growth phase.ObjectiveThe present study intends to monitor variations in deaths and identify the growth phases such as pre-growth, growth, and post-growth phases in Pakistan due to the COVID-19 pandemic.MethodsNew approaches are needed that display the death patterns and signal an alarming situation so that corrective actions can be taken before the condition worsens. To meet this purpose, secondary data on daily reported deaths due to the COVID-19 pandemic have been considered, and the $c$ and exponentially weighted moving average (EWMA) control charts are used To meet this purpose, secondary data on daily reported deaths in Pakistan due to the COVID-19 pandemic have been considered. The $ c$ and exponentially weighted moving average (EWMA) control charts have been used for monitoring variations.ResultsThe chart shows that Pakistan switches from the pre-growth to the growth phase on 31 March 2020. The EWMA chart demonstrates that Pakistan remains in the growth phase from 31 March 2020 to 17 August 2020, with some indications signaling a decrease in deaths. It is found that Pakistan moved to a post-growth phase for a brief period from 27 July 2020 to 28 July 2020. Pakistan switches to re-growth phase with an alarm on 31/7/2020, right after the short-term post-growth phase. The number of deaths starts decreasing in August in that Pakistan may approach the post-growth phase shortly.ConclusionThis amalgamation of control charts illustrates a systematic implementation of the charts for government leaders and forefront medical teams to facilitate the rapid detection of daily reported deaths due to COVID-19. Besides government and public health officials, it is also the public's responsibility to follow the enforced standard operating procedures as a temporary remedy of this pandemic in ensuring public safety while awaiting a suitable vaccine to be discovered.

Dataset Information

Patient- and surgeon-adjusted control charts for monitoring performance.

Objectives

Design

Setting

Participants

Main outcome measure

Results

Conclusions

Publications

Patient- and surgeon-adjusted control charts for monitoring performance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets