Performance Characteristics of Profiling Methods and the Impact of Inadequate Case-mix Adjustment.
Ontology highlight
ABSTRACT:
SUBMITTER: Chen Y
PROVIDER: S-EPMC7731974 | biostudies-literature | 2019
REPOSITORIES: biostudies-literature
Publications
Performance Characteristics of Profiling Methods and the Impact of Inadequate Case-mix Adjustment.
Chen Yanjun Y Şentürk Damla D Estes Jason P JP Campos Luis F LF Rhee Connie M CM Dalrymple Lorien S LS Kalantar-Zadeh Kamyar K Nguyen Danh V DV
Communications in statistics: Simulation and computation 20190328
Profiling or evaluation of health care providers involves the application of statistical models to compare each provider's performance with respect to a patient outcome, such as unplanned 30-day hospital readmission, adjusted for patient case-mix characteristics. The nationally adopted method is based on random effects (RE) hierarchical logistic regression models. Although RE models are sensible for modeling hierarchical data, novel high dimensional fixed effects (FE) models have been proposed w ...[more]
Similar Datasets
Project description:ObjectiveTo assess the effect of case-mix adjustment on community health center (CHC) performance on patient experience measures.Data sourcesA Medicaid-managed care plan in Washington State collected patient survey data from 33 CHCs over three fiscal quarters during 2007-2008. The survey included three composite patient experience measures (6-month reports) and two overall ratings of care. The analytic sample includes 2,247 adult patients and 2,859 adults reporting for child patients.Study designWe compared the relative importance of patient case-mix adjusters by calculating each adjuster's predictive power and variability across CHCs. We then evaluated the impact of case-mix adjustment on the relative ranking of CHCs.Principal findingsImportant case-mix adjusters included adult self-reported health status or parent-reported child health status, adult age, and educational attainment. The effects of case-mix adjustment on patient reports and ratings were different in the adult and child samples. Adjusting for race/ethnicity and language had a greater impact on parent reports than adult reports, but it impacted ratings similarly across the samples. The impact of adjustment on composites and ratings was modest, but it affected the relative ranking of CHCs.ConclusionsTo ensure equitable comparison of CHC performance on patient experience measures, reports and ratings should be adjusted for adult self-reported health status or parent-reported child health status, adult age, education, race/ethnicity, and survey language. Because of the differential impact of case-mix adjusters for child and adult surveys, initiatives should consider measuring and reporting adult and child scores separately.
| S-EPMC2875754 | biostudies-literature
Project description:To compare hospitals' hip fracture patient mortality in a quality of care registry, correction for patient characteristics is needed. This study evaluates in 39,374 patients which characteristics are associated with 30 and 90-day mortality, and showed how using these characteristics in a case mix-model changes hospital comparisons within the Netherlands.PurposeMortality rates after hip fracture surgery are considerable and may be influenced by patient characteristics. This study aims to evaluate hospital variation regarding patient demographics and disease burden, to develop a case-mix adjustment model to analyse differences in hip fracture patients' mortality to calculate case-mix adjusted hospital-specific mortality rates.MethodsData were derived from 64 hospitals participating in the Dutch Hip Fracture Audit (DHFA). Adult hip fracture patients registered in 2017-2019 were included. Variation of case-mix factors between hospitals was analysed, and the association between case-mix factors and mortality at 30 and 90 days was determined through regression models.ResultsThere were 39,374 patients included. Significant variation in case-mix factors amongst hospitals was found for age ≥ 80 (range 25.8-72.1% p < 0.001), male gender (12.0-52.9% p < 0.001), nursing home residents (42.0-57.9% p < 0.001), pre-fracture mobility aid use (9.9-86.7% p < 0,001), daily living dependency (27.5-96.5% p < 0,001), ASA-class ≥ 3 (25.8-83.3% p < 0.001), dementia (3.6-28.6% p < 0.001), osteoporosis (0.0-57.1% p < 0.001), risk of malnutrition (0.0-29.2% p < 0.001) and fracture types (all p < 0.001). All factors were associated with 30- and 90-day mortality. Eight hospitals showed higher and six showed lower 30-day mortality than expected based on their case-mix. Six hospitals showed higher and seven lower 90-day mortality than expected. The specific outlier hospitals changed when correcting for case-mix factors.ConclusionsDutch hospitals show significant case-mix variation regarding hip fracture patients. Case-mix adjustment is a prerequisite when comparing hospitals' 30-day and 90-day hip fracture patients' mortality. Adjusted mortality may serve as a starting point for improving hip fracture care.
| S-EPMC9046354 | biostudies-literature
Project description:Quantitative metrics are used to develop profiles of health care institutions, including hospitals, nursing homes, and dialysis clinics. These profiles serve as measures of quality of care, which are used to compare institutions and determine reimbursement, as a part of a national effort led by the Center for Medicare and Medicaid Services in the United States. However, there is some concern about how misclassification in case-mix factors, which are typically accounted for in profiling, impacts results. We evaluated the potential effect of misclassification on profiling results, using 20 744 patients from 2740 dialysis facilities in the US Renal Data System. In this case study, we compared 30-day readmission as the profiling outcome measure, using comorbidity data from either the Center for Medicare and Medicaid Services Medical Evidence Report (error-prone) or Medicare claims (more accurate). Although the regression coefficient of the error-prone covariate demonstrated notable bias in simulation, the outcome measure-standardized readmission ratio-and profiling results were quite robust; for example, correlation coefficient of 0.99 in standardized readmission ratio estimates. Thus, we conclude that misclassification on case-mix did not meaningfully impact overall profiling results. We also identified both extreme degree of case-mix factor misclassification and magnitude of between-provider variability as 2 factors that can potentially exert enough influence on profile status to move a clinic from one performance category to another (eg, normal to worse performer).
| S-EPMC7265077 | biostudies-literature
Project description:BackgroundTo evaluate the performance of the patient clinical complexity level (PCCL) mechanism, which is the patient-level complexity adjustment factor within the Korean Diagnosis-Related Groups (KDRG) patient classification system, in explaining the variation in resource consumption within age adjacent diagnosis-related groups (AADRGs).MethodsWe used the inpatient claims data from a public hospital in Korea from 1 January 2017 to 30 June 2019, with 18 846 claims and 138 AADRGs. The differences in the total average payment between the four PCCL levels for each AADRG was tested using ANOVA and Duncan's post hoc test. The three patterns of differences with R-squared were as follows: the PCCL reflected the complexity well (valid); the average payment for PCCL 2, 3, and 4 was greater than PCCL 0 (partially valid); the PCCL did not reflect the complexity (not valid).ResultsThere were 9 (6.52%), 26 (18.84%), and 103 (74.64%) ADRGs included in the valid, partially valid, and not valid categories, respectively. The average R-squared values were 32.18, 40.81, and 35.41%, respectively, with an average R-squared for all patterns of 36.21%.ConclusionsAdjustment using the PCCL in the KDRG classification system exhibited low performance in explaining the variation in resource consumption within AADRGs. As the KDRG classification system is used for reimbursement under the new DRG-based prospective payment system (PPS) pilot project, with plans for expansion, there should be an overall review of the validity of the complexity and rationality of using the KDRG classification system.
| S-EPMC8243480 | biostudies-literature
Project description:Profiling analysis aims to evaluate health care providers by modeling each provider's performance with respect to a patient outcome, such as unplanned hospital readmission. High-dimensional regression models are used in profiling to risk-adjust for patient case-mix covariates. Case-mix covariates typically ascertained from administrative databases are inherently error-prone. We examine the impact of case-mix measurement error (ME) on profiling models. The results show that even though the models' coefficient estimates are biased, this does not affect the estimation of standardized readmission ratio (SRR). However, ME leads to increased variation in SRR estimates and degrades the ability to identify under-performing providers.
| S-EPMC7731965 | biostudies-literature
Project description:ObjectiveTo develop a model for adjusting patients' reports of behavioral health care experiences on the Experience of Care and Health Outcomes (ECHO) survey to allow for fair comparisons across health plans.Data sourceSurvey responses from 4,068 individuals enrolled in 21 managed behavioral health plans who received behavioral health care within the previous year (response rate = 48 percent).Study designPotential case-mix adjustors were evaluated by combining information about their predictive power and the amount of within- and between-plan variability. Changes in plan scores and rankings due to case-mix adjustment were quantified.Principal findingsThe final case-mix adjustment model included self-reported mental health status, self-reported general health status, alcohol/drug treatment, age, education, and race/ethnicity. The impact of adjustment on plan report scores was modest, but large enough to change some plan rankings.ConclusionsAdjusting plan report scores on the ECHO survey for differences in patient characteristics had modest effects, but still may be important to maintain the credibility of patient reports as a quality metric. Differences between those with self-reported fair/poor health compared with those in excellent/very good health varied by plan, suggesting quality differences associated with health status and underscoring the importance of collecting quality information.
| S-EPMC2613989 | biostudies-literature
Project description:Along with the development of information and technology, the quality characteristics of a product cannot be monitored separately in the different types of control charts. In the past, conventional control charts were developed to monitor only one type of quality characteristic. The variable control charts are used to observe the variable or metric quality characteristics. Meanwhile, in monitoring non-metric characteristics or categorical data, attribute control charts are employed. To accommodate these two types of data, the PCA Mix control chart is suggested to simultaneously monitor these two types of data in one chart. However, some drawbacks occur when this chart is applied to monitor non-metric data which has an imbalanced proportion. Therefore, the Kernel PCA Mix control chart is created to overcome the gaps that occurred in the PCA Mix chart. Similar to the previous chart, this chart is also constructed using Hotelling's T2 statistics with Kernel Density Estimation control limit. Several simulations are used to evaluate the performance of the proposed control charts. The simulation results show that the proposed chart has a better result than the previous control chart, especially for a small mean shift with an imbalanced proportion of non-metric data. However, the PCA Mix chart has a similar performance to the proposed chart when it is applied to monitor the balanced proportion of categorical data with a large mean shift. The application with simulated data with various scenarios and the real-world case also shows that the Kernel PCA Mix chart performs better compared to the performance of the PCA Mix chart.
| S-EPMC9489795 | biostudies-literature
Project description:BACKGROUND:In stroke studies, ordinal logistic regression (OLR) is often used to analyze outcome on the modified Rankin Scale (mRS), whereas the non-parametric Mann-Whitney measure of superiority (MWS) has also been suggested. It is unclear how these perform comparatively when confounding adjustment is warranted. AIMS:Our aim is to quantify the performance of OLR and MWS in different confounding variable settings. METHODS:We set up a simulation study with three different scenarios; (1) dichotomous confounding variables, (2) continuous confounding variables, and (3) confounding variable settings mimicking a study on functional outcome after stroke. We compared adjusted ordinal logistic regression (aOLR) and stratified Mann-Whitney measure of superiority (sMWS), and also used propensity scores to stratify the MWS (psMWS). For comparability, OLR estimates were transformed to a MWS. We report bias, the percentage of runs that produced a point estimate deviating by more than 0.05 points (point estimate variation), and the coverage probability. RESULTS:In scenario 1, there was no bias in both sMWS and aOLR, with similar point estimate variation and coverage probabilities. In scenario 2, sMWS resulted in more bias (0.04 versus 0.00), and higher point estimate variation (41.6% versus 3.3%), whereas coverage probabilities were similar. In scenario 3, there was no bias in both methods, point estimate variation was higher in the sMWS (6.7%) versus aOLR (1.1%), and coverage probabilities were 0.98 (sMWS) versus 0.95 (aOLR). With psMWS, bias remained 0.00, with less point estimate variation (1.5%) and a coverage probability of 0.95. CONCLUSIONS:The bias of both adjustment methods was similar in our stroke simulation scenario, and the higher point estimate variation in the MWS improved with propensity score based stratification. The stratified MWS is a valid alternative for adjusted OLR only when the ratio of number of strata versus number of observations is relatively low, but propensity score based stratification extends the application range of the MWS.
| S-EPMC7162480 | biostudies-literature
Project description:BackgroundCase mix adjustment is a pre-requisite for valid measurement of healthcare performance and socioeconomic status (SES) is important to account for. Lack of information on individual-level SES has led to investigations into using a proxy for SES based on patient area of residence. The objective of this study was to use neighbourhood SES for case mix adjustment of performance indicators in total hip replacement (THR) in Sweden, and to compare with use of individual SES.MethodsData from patient administrative systems and the Swedish Hip Arthroplasty Register were extracted for all patients undergoing THR in four Swedish regions. For each subject, individual data and neighbourhood data on country of birth, educational level, and income were provided by Statistics Sweden. Three variables were selected for analysis of performance; EQ-5D, hip pain and length of stay (LoS). In addition to socioeconomic information, several important clinical characteristics were used as case mix factors. Regression analysis was used to study each variable's impact on the three outcome variables and model fit was evaluated using mean squared error.ResultsA total of 27,121 patients operated between 2010 and 2016 were included in the study. Both educational level and income were higher when based on neighbourhood information than individual information, while proportion born in Sweden was similar. Higher SES was generally found to be associated with better outcomes and lower LoS, albeit with certain differences between the different measures of SES. The predictive ability of the models was increased when adding information on SES to the clinical characteristics. The increase in predictive ability was higher for individual SES compared to neighbourhood SES. When analysing performance for the two providers with most diverging case mix in terms of SES, the inclusion of SES altered the relative performance using individual as well as neighbourhood SES.ConclusionsIncorporating SES improves case mix adjustment marginally compared to using only clinical information. In this patient group, geographically derived SES was found to improve case mix adjustment compared to only clinical information but not to the same extent as actual individual-level SES.
| S-EPMC7353710 | biostudies-literature
Project description:BackgroundTreatment switching in randomised controlled trials (RCTs) is a problem for health technology assessment when substantial proportions of patients switch onto effective treatments that would not be available in standard clinical practice. Often statistical methods are used to adjust for switching: these can be applied in different ways, and performance has been assessed in simulation studies, but not in real-world case studies. We assessed the performance of adjustment methods described in National Institute for Health and Care Excellence Decision Support Unit Technical Support Document 16, applying them to an RCT comparing panitumumab to best supportive care (BSC) in colorectal cancer, in which 76% of patients randomised to BSC switched onto panitumumab. The RCT resulted in intention-to-treat hazard ratios (HR) for overall survival (OS) of 1.00 (95% confidence interval [CI] 0.82-1.22) for all patients, and 0.99 (95% CI 0.75-1.29) for patients with wild-type KRAS (Kirsten rat sarcoma virus).MethodsWe tested several applications of inverse probability of censoring weights (IPCW), rank preserving structural failure time models (RPSFTM) and simple and complex two-stage estimation (TSE) to estimate treatment effects that would have been observed if BSC patients had not switched onto panitumumab. To assess the performance of these analyses we ascertained the true effectiveness of panitumumab based on: (i) subsequent RCTs of panitumumab that disallowed treatment switching; (ii) studies of cetuximab that disallowed treatment switching, (iii) analyses demonstrating that only patients with wild-type KRAS benefit from panitumumab. These sources suggest the true OS HR for panitumumab is 0.76-0.77 (95% CI 0.60-0.98) for all patients, and 0.55-0.73 (95% CI 0.41-0.93) for patients with wild-type KRAS.ResultsSome applications of IPCW and TSE provided treatment effect estimates that closely matched the point-estimates and CIs of the expected truths. However, other applications produced estimates towards the boundaries of the expected truths, with some TSE applications producing estimates that lay outside the expected true confidence intervals. The RPSFTM performed relatively poorly, with all applications providing treatment effect estimates close to 1, often with extremely wide confidence intervals.ConclusionsAdjustment analyses may provide unreliable results. How each method is applied must be scrutinised to assess reliability.
| S-EPMC10802004 | biostudies-literature