Dataset Information

Evaluation of Reliability and Correlations of Quality Measures in Cancer Care.

ABSTRACT:

Importance

Measurement of the quality of care is important for alternative payment models in oncology, yet the ability to distinguish high-quality from low-quality care across oncology practices remains uncertain.

Objective

To assess the reliability of cancer care quality measures across oncology practices using registry and claims-based measures of process, utilization, end-of-life (EOL) care, and survival, and to assess the correlations of practice-level performance across measure and cancer types.

Design, setting, and participants

This cross-sectional study used the Surveillance, Epidemiology, and End Results (SEER) Program registry linked to Medicare administrative data to identify individuals with lung cancer, breast cancer, or colorectal cancer (CRC) that was newly diagnosed between January 1, 2011, and December 31, 2015, and who were treated in oncology practices with 20 or more patients. Data were analyzed from January 2018 to December 2020.

Main outcomes and measures

Receipt of guideline-recommended treatment and surveillance, hospitalizations or emergency department visits during 6-month chemotherapy episodes, care intensity in the last month of life, and 12-month survival were measured. Summary measures for each domain in each cohort were calculated. Practice-level rates for each measure were estimated from hierarchical linear models with practice-level random effects; practice-level reliability (reproducibility) for each measure based on the between-measure variance, within-measure variance, and distribution of patients treated in each practice; and correlations of measures across measure and cancer types.

Results

In this study of SEER registry data linked to Medicare administrative data from 49 715 patients with lung cancer treated in 502 oncology practices, 21 692 with CRC treated in 347 practices, and 52 901 with breast cancer treated in 492 practices, few practices had 20 or more patients who were eligible for most process measures during the 5-year study period. Patients were 65 years or older; approximately 50% of the patients with lung cancer and CRC and all of the patients with breast cancer were women. Most measures had limited variability across practices. Among process measures, 0 of 6 for lung cancer, 0 of 6 for CRC, and 3 of 11 for breast cancer had a practice-level reliability of 0.75 or higher for the median-sized practice. No utilization, EOL care, or survival measure had reliability across practices of 0.75 or higher. Correlations across measure types were low (r ≤ 0.20 for all) except for a correlation between the CRC process and 1-year survival summary measures (r = 0.35; P < .001). Summary process measures had limited or no correlation across lung cancer, breast cancer, and CRC (r ≤ 0.16 for all).

Conclusions and relevance

This study found that quality measures were limited by the small numbers of Medicare patients with newly diagnosed cancer treated in oncology practices, even after pooling 5 years of data. Measures had low reliability and had limited to no correlation across measure and cancer types, suggesting the need for research to identify reliable quality measures for practice-level quality assessments.

SUBMITTER: Keating NL

PROVIDER: S-EPMC7985722 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:Patient experience measures are widely used to compare performance at the individual physician level. OBJECTIVE:To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level. DESIGN:Repeated cross-sectional design. SETTING:Academic family medicine practice in California. PARTICIPANTS:One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians). MEASUREMENTS:Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status). RESULTS:The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0-25.2%)]. CONCLUSIONS:Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.

Project description:BackgroundSystemic lupus erythematosus (SLE) affects 1 in 2500 Americans and is associated with significant morbidity and mortality. The recent development of SLE quality measures provides an opportunity to understand gaps in clinical care and to identify modifiable factors associated with variations in quality.ObjectiveTo evaluate performance on SLE quality measures as well as differences in quality of care by demographic, socioeconomic, disease, and health system characteristics.Design and patientsCross-sectional analysis of data derived from the Lupus Outcomes Study, a prospective, longitudinal study of 814 individuals. Principal data collection was through annual structured telephone surveys between 2009-2010. Data on 13 SLE quality measures was collected. We used regression models to estimate demographic, socioeconomic, disease, and health system characteristics associated with performance on individual and overall quality measures.Outcome measuresPerformance on each quality measure and overall performance on all measures for which participants were eligible (pass rate).ResultsParticipants were eligible for a mean of five measures (range 2-12). Performance varied from 29 % (assessment of cardiovascular risk factors) to 90 % (sun avoidance counseling). The overall pass rate was 65 % (95 % CI 64 %, 65 %). In unadjusted analyses, younger age, minority race/ethnicity, poverty, shorter disease duration, fewer physician visits, and lack of health insurance, were associated with lower pass rates. Receiving care in public sector managed care organizations was associated with higher pass rates. After adjustment, younger age, having fewer physician visits and lacking health insurance remained significantly associated with lower performance; receiving care in public sector managed care organizations remained associated with higher performance.ConclusionsWe identified a number of gaps in clinical care for SLE. Factors associated with the health care system, including presence and type of health insurance, were the primary determinants of performance on quality measures in SLE.