Dataset Information

Comparing and Monitoring Risk-Adjusted Hospital Performance Measures: A Weighted Estimating Equations Approach.

ABSTRACT: Background. There is a great deal of interest in evaluating hospital performance in order to monitor and improve health care quality. Increasingly, risk-adjusted performance measures are available to the public and statistical approaches for estimating these measures are considered. Some methods in use currently are based on 3-year aggregates of data since a small number of cases may lead to imprecise estimates and make it hard for stakeholders to detect differences across hospitals over time. However, if quality changes over time, a measure based on these data is a biased estimate of present performance. Methods. We present an alternative approach (weighted estimating equations [WEE]) for combining historical data in estimation that regulates the tradeoff between bias and precision in the measure of present performance. The WEE approach uses all available historical data through estimating functions that down-weight past data. Results. We compare the WEE approach to two current practices using a realistic dataset of the mortality of patients following an elective percutaneous coronary intervention procedure in New York State who meet certain criteria. The width of the uncertainty interval in the realistic example is up to 65% smaller and the difference is more pronounced for hospitals with a small number of cases. Conclusions. The advantage of this approach extends from the example dataset to other datasets. The WEE approach uses all available data rather than data from an arbitrary 3-year window. The effect of borrowing strength from historical data is a more precise estimate of present performance than current practices. Its advantages are important for the comparison of other aspects of medical performance, including surgical or medical practitioner performance.

SUBMITTER: Barfoot PC

PROVIDER: S-EPMC6125076 | biostudies-literature | 2018 Jan-Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparing and Monitoring Risk-Adjusted Hospital Performance Measures: A Weighted Estimating Equations Approach.

Barfoot Patricia Cooper PC MacKay R Jock RJ Steiner Stefan H SH

MDM policy & practice 20180101 1

<b>Background.</b> There is a great deal of interest in evaluating hospital performance in order to monitor and improve health care quality. Increasingly, risk-adjusted performance measures are available to the public and statistical approaches for estimating these measures are considered. Some methods in use currently are based on 3-year aggregates of data since a small number of cases may lead to imprecise estimates and make it hard for stakeholders to detect differences across hospitals over ...[more]

PMID: 30288439

Similar Datasets

Project description:ObjectivesTo determine whether an innovative graphical tool for accurate measurement of individual surgeon performance metrics, adjusted for both surgeon-specific and patient-specific factors, significantly alters interpretation of performance data.DesignRetrospective analysis of all total knee replacements (TKRs) conducted at the host institution between 1996 and 2009. The database was randomly divided into training and testing datasets. Using multivariate generalised estimating equation regression models, the training dataset enabled generation of patient-risk and surgeon-experience adjustment factors. To simulate prospective monitoring of individual surgeon outcomes, the testing dataset was mapped on control charts. Weighted ? statistics were calculated to measure the agreement between patient-risk adjusted and fully adjusted control charts.SettingTertiary care academic hospital.ParticipantsAll patients undergoing TKR at the host institution 1996-2009.Main outcome measureOperative efficiency.Results5313 procedures were analysed. Adjusted control charts were generated using a training dataset comprising 3756 procedures performed by 13 surgeons. The operative time gradually declined by 121 min with 25 years of experience (p<0.0001). Charts were tested by monitoring four other surgeons, performing an average of 389 procedures each. Adjustment for surgeon experience significantly altered the interpretation of operative efficiency (?=0.29 (95% CI 0.11 to 0.47)), and enhanced assessment of a surgeon's improvement or diminishment in efficiency over time. Specifically, experience adjustment inverted the interpretation of surgeon efficiency from above average to below average, or from improving to declining performance.ConclusionsAdjustment for surgeon experience is necessary for accurate interpretation of metrics over the course of a surgeon's career. Patient-adjusted and surgeon-adjusted control charts provide an accurate method of monitoring individual operative efficiency.

Project description:Accurate assessment of kidney function is important for the management of solid-organ transplant recipients. In other clinical populations, glomerular filtration rate (GFR) most commonly is estimated using the CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) creatinine or the 4-variable MDRD (Modification of Diet in Renal Disease) Study equation. The accuracy of these equations compared with other GFR estimating equations in transplant recipients has not been carefully studied.Diagnostic test study.Solid-organ transplant recipients longer than 6 months posttransplantation from 5 clinical populations (N=3,622, including recipients of kidney [53%], liver [35%], and other or multiple organs [12%]).Estimated GFR (eGFR) using creatinine-based GFR estimating equations identified from a systematic review of the literature. Performance of the CKD-EPI creatinine and the MDRD Study equations was compared with alternative equations.Measured GFR (mGFR) from urinary clearance of iothalamate or plasma clearance of iohexol.Error (difference between mGFR and eGFR) expressed as P30 (proportion of absolute percent error <30%) and mean absolute error.We identified 26 GFR estimating equations. Mean mGFR was 55.1±22.7 (SD) mL/min/1.73 m(2). P30 and mean absolute error for the CKD-EPI and the MDRD Study equations were 78.9% (99.6% CI, 76.9%-80.8%) for both and 10.6 (99.6% CI, 10.1-11.1) versus 11.0 (99.6% CI, 10.5-11.5) mL/min/1.73 m(2), respectively; these equations were more accurate than any of the alternative equations (P <0.001 for all pairwise comparisons for both measures). They performed better than or as well as the alternative equations in most subgroups defined by demographic and clinical characteristics, including type of transplanted organ.Study population included few nonwhites and people with solid-organ transplants other than liver and kidneys.The CKD-EPI creatinine and the MDRD Study equations perform better than the alternative creatinine-based estimating equations in solid-organ transplant recipients. They can be used for clinical management.

Project description:Rationale & objectiveEvaluation of glomerular filtration rate (GFR) is challenging in adults undergoing bariatric surgery because creatinine and cystatin C levels are influenced by changes in muscle and fat mass. Additionally, indexing of GFR by body surface area (BSA) may by affected by decreases in BSA.Study designProspective observational study.Setting & participants27 adults with body mass index (BMI) ≥ 35 kg/m2 who underwent measurement of GFR before and after bariatric surgery.OutcomesIndexed and nonindexed GFRs measured (mGFRs) using plasma iohexol clearance, indexed and nonindexed estimated GFR (eGFR) based on levels of creatinine, cystatin C, or both from Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equations.Analytic approachBias and percent of estimates within 20% and 30% of mGFR (P20 and P30) for estimating equations were examined.ResultsMean presurgery BMI was 49.5 (SD, 9.4) kg/m2, BSA was 2.42 (SD, 0.27) m2, nonindexed mGFR was 117.3 (SD, 34.1) mL/min, and indexed mGFR was 84.1 (SD, 22.0) mL/min/1.73 m2. After 6 months, mean BMI changed by -13.8 (95% CI, -15.9 to -11.8) kg/m2, BSA by -0.30 (95% CI, -0.33 to -0.27) m2, and nonindexed mGFR by -9.2 (95% CI, -17.2 to -1.1) mL/min, while indexed mGFR was unchanged at 5.1 (95% CI, -0.1 to 10.4) mL/min/1.73 m2. Nonindexed eGFRcr was unbiased (median bias, 5.0 [95% CI, -4.3 to 11.6] mL/min) before surgery, but overestimated mGFR (8.8 [95% CI, 1.8 to 16.9] mL/min) after surgery. Nonindexed eGFRcys underestimated mGFR before (median bias, -12.1 [95% CI, -21.4 to -1.2] mL/min) and after surgery (-11.2 [95% CI, -21.8 to -7.3] mL/min). Nonindexed eGFRcr-cys was unbiased before (median bias, -6.0 [95% CI, -11.0 to 1.0] mL/min) and after surgery (-2.0 [95% CI, -8.8 to 4.9] mL/min). Findings were similar for indexed eGFR compared with indexed mGFR.LimitationsSmall, mostly white sample.ConclusionsChanges in indexed and nonindexed GFRs may be discordant after bariatric surgery in adults because of decreases in BSA. Indexed and nonindexed eGFRcr-cys may be less biased than indexed or nonindexed eGFRcr or eGFRcys because of opposite biases in estimating mGFR.

Dataset Information

Comparing and Monitoring Risk-Adjusted Hospital Performance Measures: A Weighted Estimating Equations Approach.

Publications

Comparing and Monitoring Risk-Adjusted Hospital Performance Measures: A Weighted Estimating Equations Approach.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure