Ranking hospitals on surgical mortality: the importance of reliability adjustment.
Ontology highlight
ABSTRACT: Objective
We examined the implications of reliability adjustment on hospital mortality with surgery.Data source
We used national Medicare data (2003-2006) for three surgical procedures: coronary artery bypass grafting (CABG), abdominal aortic aneurysm (AAA) repair, and pancreatic resection.Study design
We conducted an observational study to evaluate the impact of reliability adjustment on hospital mortality rankings. Using hierarchical modeling, we adjusted hospital mortality for reliability using empirical Bayes techniques. We assessed the implication of this adjustment on the apparent variation across hospitals and the ability of historical hospital mortality rates (2003-2004) to forecast future mortality (2005-2006).Principal findings
The net effect of reliability adjustment was to greatly diminish apparent variation for all three operations. Reliability adjustment was also particularly important for identifying hospitals with the lowest future mortality. Without reliability adjustment, hospitals in the "best" quintile (2003-2004) with pancreatic resection had a mortality of 7.6 percent in 2005-2006; with reliability adjustment, the "best" hospital quintile had a mortality of 2.7 percent in 2005-2006. For AAA repair, reliability adjustment also improved the ability to identify hospitals with lower future mortality. For CABG, the benefits of reliability adjustment were limited to the lowest volume hospitals.Conclusion
Reliability adjustment results in more stable estimates of mortality that better forecast future performance. This statistical technique is crucial for helping patients select the best hospitals for specific procedures, particularly uncommon ones, and should be used for public reporting of hospital mortality.
SUBMITTER: Dimick JB
PROVIDER: S-EPMC2976775 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND:The conceptualization of hospital quality indicators usually includes some form of risk adjustment to account for hospital differences in case mix. For binary outcome variables like in-hospital mortality, frequently utilized risk adjusted measures include the standardized mortality ratio (SMR), the risk standardized mortality rate (RSMR), and excess risk (ER). All of these measures require the estimation of expected hospital mortality, which is often based on logistic regression models. In this context, an issue that is often neglected is correlation between hospital performance (e.g. care quality) and patient-specific risk factors. The objective of this study was to investigate the impact of such correlation on the adequacy of hospital rankings based on different measures and methods. METHODS:Using Monte Carlo simulation, the impact of correlation between hospital care quality and patient-specific risk factors on the adequacy of hospital rankings was assessed for SMR/RSMR, and ER based on logistic regression and random effects logistic regression. As an alternative method, fixed effects logistic regression with Firth correction was considered. The adequacies of the resulting hospital rankings were assessed by the shares of hospitals correctly classified into quintiles according to their true (unobserved) care qualities. RESULTS:The performance of risk adjustment approaches based on logistic regression and random effects logistic regression declined when correlation between care quality and a risk factor was induced. In contrast, fixed-effects-based estimations proved to be more robust. This was particularly true for fixed-effects-logistic-regression-based ER. In the absence of correlation between risk factors and care quality, all approaches showed similar performance. CONCLUSIONS:Correlation between risk factors and hospital performance may severely bias hospital rankings based on logistic regression and random effects logistic regression. ER based on fixed effects logistic regression with Firth correction should be considered as an alternative approach to assess hospital performance.
| S-EPMC6892499 | biostudies-literature
Project description:OBJECTIVES:To extend reliability of WHO Behaviourally Anchored Rating Scale (WHOBARS) to measure the quality of WHO Surgical Safety Checklist administration using generalisability theory. In this context, extending reliability refers to establishing generalisability of the tool scores across populations of teams and raters by accounting for the relevant sources of measurement errors. DESIGN:Cross-sectional random effect measurement design assessing surgical teams by the five items on the three Checklist phases, and at three sites by two trained raters simultaneously. SETTING:The data were collected in three tertiary hospitals in Auckland, New Zealand in 2016 and included 60 teams observed in 60 different cases with an equal number of teams (n=20) per site. All elective and acute cases (adults and children) involving surgery under general anaesthesia during normal working hours were eligible. PARTICIPANTS:The study included 243 surgical staff members, 138 (50.12%) women. MAIN OUTCOME MEASURE:Absolute generalisability coefficient that accounts for variance due to items, phases, sites and raters for the WHOBARS measure of the quality of WHO Surgical Safety Checklist administration. RESULTS:The WHOBARS in its present form has demonstrated good generalisability of scores across teams and raters (G absolute=0.83). The largest source of measurement error was the interaction between the surgical team and the rater, accounting for 16.7% (95% CI 16.4 to 16.9) of the total variance in the data. Removing any items from the WHOBARS led to a decrease in the overall reliability of the instrument. CONCLUSIONS:Assessing checklist administration quality is important for promoting improvement in its use, and WHOBARS offers a reliable approach for doing this.
| S-EPMC6340010 | biostudies-literature
Project description:BackgroundThe Centers for Medicare and Medicaid Services (CMS) use colon surgical site infection (SSI) rates to rank hospitals and apply financial penalties. The CMS' risk-adjustment model omits potentially impactful variables that might disadvantage hospitals with complex surgical populations.MethodsWe analyzed adult patients who underwent colon surgery within facilities associated with HCA Healthcare from 2014 to 2016. SSIs were identified from National Health Safety Network (NHSN) reporting. We trained and validated 3 SSI prediction models, using (1) current CMS model variables, including hospital-specific random effects (HCA-adapted CMS model); (2) demographics and claims-based comorbidities (expanded-claims model); and (3) demographics, claims-based comorbidities, and NHSN variables (claims-plus-electronic health record [EHR] model). Discrimination, calibration, and resulting rankings were compared among all models and the current CMS model with published coefficient values.ResultsWe identified 39 468 colon surgeries in 149 hospitals, resulting in 1216 (3.1%) SSIs. Compared to the HCA-adapted CMS model, the expanded-claims model had similar performance (c-statistic, 0.65 vs 0.67, respectively), while the claims-plus-EHR model was more accurate (c-statistic, 0.70; 95% confidence interval, .67-.73; P = .004). The sampling variation, due to the low surgical volume and small number of infections, contributed 74% of the total variation in observed SSI rates between hospitals. When CMS model rankings were compared to those from the expanded-claims and claims-plus-EHR models, 18 (15%) and 26 (22%) hospitals changed quartiles, respectively, and 10 (8.3%) and 12 (10%) hospitals changed into or out of the lowest-performing quartile, respectively.ConclusionsAn expanded set of variables improved colon SSI risk predictions and quartile assignments, but low procedure volumes and SSI events remain a barrier to effectively comparing hospitals.
| S-EPMC7823072 | biostudies-literature
Project description:ObjectiveDespite a growing volume of surgical procedures in low-income and middle-income countries, the costs of these procedures are not well understood. We estimated the costs of 12 surgical procedures commonly conducted in five different types of hospitals in India from the provider perspective, using a microcosting method.DesignCost and utilisation data were collected retrospectively from April 2010 to March 2011 to avoid seasonal variability.SettingFor this study, we chose five hospitals of different types: a 57-bed charitable hospital, a 200-bed private hospital, a 400-bed district hospital, a 655-bed private teaching hospital and a 778-bed tertiary care teaching hospital based on their willingness to cooperate and data accessibility. The hospitals were from four states in India. The private, charitable and tertiary care hospitals serve urban populations, the district hospital serves a semiurban area and the private teaching hospital serves a rural population.ResultsCosts of conducting lower section caesarean section ranged from rupees 2469 to 41 087; hysterectomy rupees 4124 to 57 622 and appendectomy rupees 2421 to 3616 (US$1=rupees 52). We computed the costs of conducting lap and open cholecystectomy (rupees 27 732 and 44 142, respectively); hernia repair (rupees 13 204); external fixation (rupees 8406); intestinal obstruction (rupees 6406); amputation (rupees 5158); coronary artery bypass graft (rupees 177 141); craniotomy (rupees 75 982) and functional endoscopic sinus surgery (rupees 53 398).ConclusionsEstimated costs are roughly comparable with rates of reimbursement provided by the Rashtriya Swasthya Bima Yojana (RSBY)-India's government-financed health insurance scheme that covers 32.4 million poor families. Results from this type of study can be used to set and revise the reimbursement rates.
| S-EPMC3686227 | biostudies-literature
Project description:Globally sclerotic glomeruli (GSG) occur with both normal aging and kidney disease. However, it is unknown whether any GSG or only GSG exceeding that expected for age is clinically important. To evaluate this, we identified patients with a glomerulopathy that often presents with nephrotic syndrome (focal segmental glomerulosclerosis, membranous nephropathy, or minimal change disease) in the setting of the Nephrotic Syndrome Study Network (NEPTUNE), China-Digital Kidney Pathology (DiKiP), and the Southeast Minnesota cohorts. Age-based thresholds (95th percentile) for GSG based on normotensive living kidney donors were used to classify each patient into one of three groups; no GSG, GSG normal for age, or GSG abnormal for age. The risk of end-stage renal disease or a 40% decline in glomerular filtration rate during follow-up was then compared between groups. Among the 425 patients studied, 170 had no GSG, 107 had GSG normal for age, and 148 had GSG abnormal for age. Compared to those with no GSG, the risk of kidney disease progression with GSG normal for age was similar but was significantly higher with GSG abnormal for age. This increased risk with GSG abnormal for age remained significant after adjustment for interstitial fibrosis, arteriosclerosis, age, hypertension, diabetes, body mass index, glomerulopathy type, glomerular filtration rate, and proteinuria. Thus, in patients with glomerulopathy that often presents with nephrotic syndrome, global glomerulosclerosis is clinically important only if it exceeds that expected for age.
| S-EPMC5911429 | biostudies-literature
Project description:BACKGROUND:Although there is evidence that hospitals recognized for nursing excellence--Magnet hospitals--are successful in attracting and retaining nurses, it is uncertain whether Magnet recognition is associated with better patient outcomes than non-Magnets, and if so why. OBJECTIVES:To determine whether Magnet hospitals have lower risk-adjusted mortality and failure-to-rescue compared with non-Magnet hospitals, and to determine the most likely explanations. METHOD AND STUDY DESIGN: Analysis of linked patient, nurse, and hospital data on 56 Magnet and 508 non-Magnet hospitals. Logistic regression models were used to estimate differences in the odds of mortality and failure-to-rescue for surgical patients treated in Magnet versus non-Magnet hospitals, and to determine the extent to which differences in outcomes can be explained by nursing after accounting for patient and hospital differences. RESULTS:Magnet hospitals had significantly better work environments and higher proportions of nurses with bachelor's degrees and specialty certification. These nursing factors explained much of the Magnet hospital effect on patient outcomes. However, patients treated in Magnet hospitals had 14% lower odds of mortality (odds ratio 0.86; 95% confidence interval, 0.76-0.98; P=0.02) and 12% lower odds of failure-to-rescue (odds ratio 0.88; 95% confidence interval, 0.77-1.01; P=0.07) while controlling for nursing factors as well as hospital and patient differences. CONCLUSIONS:The lower mortality we find in Magnet hospitals is largely attributable to measured nursing characteristics but there is a mortality advantage above and beyond what we could measure. Magnet recognition identifies existing quality and stimulates further positive organizational behavior that improves patient outcomes.
| S-EPMC3568449 | biostudies-literature
Project description:BackgroundThe US News and World Report reputation score correlates strongly with overall rank in adult and pediatric hospital rankings. Social media affects how information is disseminated to physicians and is used by hospitals as a marketing tool to recruit patients. It is unclear whether the reputation score for adult and children's hospitals relates to social media presence.ObjectiveThe objective of our study was to analyze the association between a hospital's social media metrics and the US News 2017-2018 Best Hospital Rankings for adult and children's hospitals.MethodsWe conducted a cross-sectional analysis of the reputation score, total score, and social media metrics (Twitter, Facebook, and Instagram) of hospitals who received at least one subspecialty ranking in the 2017-2018 US News publicly available annual rankings. Regression analysis was employed to analyze the partial correlation coefficients between social media metrics and a hospital's total points (ie, rank) and reputation score for both adult and children's hospitals while controlling for the bed size and time on Twitter.ResultsWe observed significant correlations for children's hospitals' reputation score and total points with the number of Twitter followers (total points: r=.465, P<.001; reputation: r=.524, P<.001) and Facebook followers (total points: r=.392, P=.002; reputation: r=.518, P<.001). Significant correlations for the adult hospitals reputation score were found with the number of Twitter followers (r=.848, P<.001), number of tweets (r=.535, P<.001), Klout score (r=.242, P=.02), and Facebook followers (r=.743, P<.001). In addition, significant correlations for adult hospitals total points were found with Twitter followers (r=.548, P<.001), number of tweets (r=.358, P<.001), Klout score (r=.203, P=.05), Facebook followers (r=.500, P<.001), and Instagram followers (r=.692, P<.001).ConclusionsA statistically significant correlation exists between multiple social media metrics and both a hospital's reputation score and total points (ie, overall rank). This association may indicate that a hospital's reputation may be influenced by its social media presence or that the reputation or rank of a hospital drives social media followers.
| S-EPMC6250955 | biostudies-literature
Project description:BACKGROUND:Coronary atherosclerosis raises the risk of acute myocardial infarction (AMI), and is usually included in AMI risk-adjustment models. Percutaneous coronary intervention (PCI) does not cause atherosclerosis, but may contribute to the notation of atherosclerosis in administrative claims. We investigated how adjustment for atherosclerosis affects rankings of hospitals that perform PCI. METHODS AND RESULTS:This was a retrospective cohort study of 414 715 Medicare beneficiaries hospitalized for AMI between 2009 and 2011. The outcome was 30-day mortality. Regression models determined the association between patient characteristics and mortality. Rankings of the 100 largest PCI and non-PCI hospitals were assessed with and without atherosclerosis adjustment. Patients admitted to PCI hospitals or receiving interventional cardiology more frequently had an atherosclerosis diagnosis. In adjustment models, atherosclerosis was associated, implausibly, with a 42% reduction in odds of mortality (odds ratio=0.58, P<0.0001). Without adjustment for atherosclerosis, the number of expected lives saved by PCI hospitals increased by 62% (P<0.001). Hospital rankings also changed: 72 of the 100 largest PCI hospitals had better ranks without atherosclerosis adjustment, while 77 of the largest non-PCI hospitals had worse ranks (P<0.001). CONCLUSIONS:Atherosclerosis is almost always noted in patients with AMI who undergo interventional cardiology but less often in medically managed patients, so adjustment for its notation likely removes part of the effect of interventional treatment. Therefore, hospitals performing more extensive imaging and more PCIs have higher atherosclerosis diagnosis rates, making their patients appear healthier and artificially reducing the expected mortality rate against which they are benchmarked. Thus, atherosclerosis adjustment is detrimental to hospitals providing more thorough AMI care.
| S-EPMC6015352 | biostudies-literature
Project description:OBJECTIVES:Comprehensive reporting of surgical disease burden and outcomes are vital components of resilient health systems but remain under-reported. The primary objective was to identify the Victorian surgical burden of disease necessitating treatment in a hospital or day centre, including a thorough epidemiology of surgical procedures and their respective perioperative mortality rates (POMR). DESIGN:Retrospective population-level observational study. SETTING:The study was conducted in Victoria, Australia. Access to data from the Victorian Admitted Episodes Dataset was obtained using the Dr Foster Quality Investigator tool. The study included public and private facilities, including day-case facilities. PARTICIPANTS:From January 2014 to December 2016, all admissions with an International Statistical Classification of Diseases-10 code matched to the Global Health Estimates (GHE) disease categories were included. PRIMARY AND SECONDARY OUTCOME MEASURES:Admissions were assigned a primary disease category according to the 23 GHE disease categories. Surgical procedures during hospitalisations were identified using the Australian Refined Diagnosis Related Groups (AR-DRG). POMR were calculated for GHE disease categories and AR-DRG procedures. RESULTS:A total of 4 865 226 admitted episodes were identified over the 3-year period. 1 715 862 (35.3%) of these required a surgical procedure. The mortality rate for those undergoing a procedure was 0.42%, and 1.47% for those without. The top five procedures performed per GHE category were lens procedures (162 835 cases, POMR 0.001%), caesarean delivery (76 032 cases, POMR 0.01%), abortion with operating room procedure (65 451 cases, POMR 0%), hernia procedures (52 499 cases, POMR 0.05%) and other knee procedures (47 181 cases, POMR 0.004%). CONCLUSIONS:Conditions requiring surgery were responsible for 35.3% of the hospital admitted disease burden in Victoria, a rate higher than previously published from Sweden, New Zealand and the USA. POMR is comparable to other studies reporting individual procedures and conditions, but has been reported comprehensively across all GHE disease categories for the first time.
| S-EPMC6549668 | biostudies-other
Project description:ObjectiveTo examine the benefits of a high-performance work environment (HPWE) for employees, patients, and hospitals.Study settingForty-five adult, medical-surgical units in nine hospitals in upstate New York.Study designCross-sectional study.Data collectionSurveys were collected from 1,527 unit-based hospital providers (68.5 percent response rate). Hospitals provided unit turnover and patient data (16,459 discharge records and 2,920 patient surveys).Principal findingsHPWE, as perceived by multiple occupational groups on a unit, is significantly associated with desirable work processes, retention indicators, and care quality.ConclusionOur findings underscore the potential benefits for providers, patients, and health care organizations of designing work environments that value and support a broad range of employees as having essential contributions to make to the care process and their organizations.
| S-EPMC3589968 | biostudies-literature