Dataset Information

Do Differential Response Rates to Patient Surveys Between Organizations Lead to Unfair Performance Comparisons?: Evidence From the English Cancer Patient Experience Survey.

ABSTRACT: BACKGROUND:Patient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of performance comparisons. OBJECTIVE:To explore the size and likely sources of associations between hospital-level survey response rates and patient experience. RESEARCH DESIGN, SUBJECTS, AND MEASURES:Cross-sectional mail survey including 60 patient experience items sent to 101,771 cancer survivors recently treated by 158 English NHS hospitals. Age, sex, race/ethnicity, socioeconomic status, clinical diagnosis, hospital type, and region were available for respondents and nonrespondents. RESULTS:The overall response rate was 67% (range, 39% to 77% between hospitals). Hospitals with higher response rates had higher scores for all items (Spearman correlation range, 0.03-0.44), particularly questions regarding hospital-level administrative processes, for example, procedure cancellations or medical note availability.From multivariable analysis, associations between individual patient experience and hospital-level response rates were statistically significant (P<0.05) for 53/59 analyzed questions, decreasing to 37/59 after adjusting for case-mix, and 25/59 after further adjusting for hospital-level characteristics.Predicting responses of nonrespondents, and re-estimating hypothetical hospital scores assuming a 100% response rate, we found that currently low performing hospitals would have attained even lower scores. Overall nationwide attainment would have decreased slightly to that currently observed. CONCLUSIONS:Higher response rate hospitals have more positive experience scores, and this is only partly explained by patient case-mix. High response rates may be a marker of efficient hospital administration, and higher quality that should not, therefore, be adjusted away in public reporting. Although nonresponse may result in slightly overestimating overall national levels of performance, it does not appear to meaningfully bias comparisons of case-mix-adjusted hospital results.

SUBMITTER: Saunders CL

PROVIDER: S-EPMC4674144 | biostudies-literature | 2016 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Do Differential Response Rates to Patient Surveys Between Organizations Lead to Unfair Performance Comparisons?: Evidence From the English Cancer Patient Experience Survey.

Saunders Catherine L CL Elliott Marc N MN Lyratzopoulos Georgios G Abel Gary A GA

Medical care 20160101 1

<h4>Background</h4>Patient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of performance comparisons.<h4>Objective</h4>To explore the size and likely sources of associations between hospital-level survey response rates and patient experience.<h4>Research design, subjects, and measures</h4>Cross-sectional mail survey including 60 patient experience items sent to 101,771 cancer survivors recently treated by 15 ...[more]

PMID: 26595223

Similar Datasets

Project description:BackgroundPatient experience surveys are important tools for improving the quality of cancer services, but the representativeness of responders is a concern. Increasingly, patient surveys that traditionally used postal questionnaires are incorporating an online response option. However, the characteristics and experience ratings of online responders are poorly understood.ObjectiveWe sought to examine predictors of postal or online response mode, and associations with patient experience in the (English) Cancer Patient Experience Survey.MethodsWe analyzed data from 71,186 patients with cancer recently treated in National Health Service hospitals who responded to the Cancer Patient Experience Survey 2015. Using logistic regression, we explored patient characteristics associated with greater probability of online response and whether, after adjustment for patient characteristics, the online response was associated with a more or less critical evaluation of cancer care compared to the postal response.ResultsOf the 63,134 patients included in the analysis, 4635 (7.34%) responded online. In an adjusted analysis, male (women vs men: odds ratio [OR] 0.50, 95% confidence interval [CI] 0.46-0.54), younger (<55 vs 65-74 years: OR 3.49, 95% CI 3.21-3.80), least deprived (most vs least deprived quintile: OR 0.57, 95% CI 0.51-0.64), and nonwhite (nonwhite vs white ethnic group: OR 1.37, 95% CI 1.24-1.51) patients were more likely to respond online. Compared to postal responders, after adjustment for patient characteristics, online responders had a higher likelihood of reporting an overall satisfied experience of care (OR 1.24, 95% CI 1.16-1.32). For 34 of 49 other items, online responders more frequently reported a less than positive experience of care (8 reached statistical significance), and the associations were positive for the remaining 15 of 49 items (2 reached statistical significance).ConclusionsIn the context of a national survey of patients with cancer, online and postal responders tend to differ in their characteristics and rating of satisfaction. Associations between online response and reported experience were generally small and mostly nonsignificant, but with a tendency toward less than positive ratings, although not consistently. Whether the observed associations between response mode and reported experience were causal needs to be examined using experimental survey designs.

Project description:ObjectivesUncertainties exist about when and how best to adjust performance measures for case mix. Our aims are to quantify the impact of case-mix adjustment on practice-level scores in a national survey of patient experience, to identify why and when it may be useful to adjust for case mix, and to discuss unresolved policy issues regarding the use of case-mix adjustment in performance measurement in health care.Design/settingSecondary analysis of the 2009 English General Practice Patient Survey. Responses from 2 163 456 patients registered with 8267 primary care practices. Linear mixed effects models were used with practice included as a random effect and five case-mix variables (gender, age, race/ethnicity, deprivation, and self-reported health) as fixed effects.Main outcome measuresPrimary outcome was the impact of case-mix adjustment on practice-level means (adjusted minus unadjusted) and changes in practice percentile ranks for questions measuring patient experience in three domains of primary care: access; interpersonal care; anticipatory care planning, and overall satisfaction with primary care services.ResultsDepending on the survey measure selected, case-mix adjustment changed the rank of between 0.4% and 29.8% of practices by more than 10 percentile points. Adjusting for case-mix resulted in large increases in score for a small number of practices and small decreases in score for a larger number of practices. Practices with younger patients, more ethnic minority patients and patients living in more socio-economically deprived areas were more likely to gain from case-mix adjustment. Age and race/ethnicity were the most influential adjustors.ConclusionsWhile its effect is modest for most practices, case-mix adjustment corrects significant underestimation of scores for a small proportion of practices serving vulnerable patients and may reduce the risk that providers would 'cream-skim' by not enrolling patients from vulnerable socio-demographic groups.

Project description:BackgroundThe routine use of patient reported outcome measures (PROMs) aims to compare providers as regards the clinical need of their patients and their outcome. Simple methods of estimating recruitment rates based on aggregated data may be inaccurate. Our objectives were to: use patient-level linked data to evaluate these estimates; produce revised estimates of national and providers' recruitment rates; and explore whether or not recruitment bias exists.MethodsCase study based on patients who were eligible to participate in the English National PROMs Programme for elective surgery (hip and knee replacement, groin hernia repair, varicose vein surgery) using data from pre-operative questionnaires and Hospital Episode Statistics. Data were linked to determine: the eligibility for including operations; eligibility of date of surgery; duplicate questionnaires; cancelled operations; correct assignment to provider. Influence of patient characteristics on recruitment rates were investigated.ResultsNational recruitment rates based on aggregated data over-estimated the true rate because of the inclusion of ineligible operations (from 1.9% - 7.0% depending on operation) and operations being cancelled (1.9% - 3.6%). Estimates of national recruitment rates using inclusion criteria based on patient-level linked data were lower than those based on simple methods (eg hip replacement was 73% rather than 78%).Estimates of provider's recruitment rates based on aggregated data were also adversely affected by attributing patients to the wrong provider (2.4% - 4.9%). Use of linked data eliminated all estimates of over 100% recruitment, though providers still showed a wide range of rates.While the principal determinant of recruitment rates was the provider, some patients' socio-demographic characteristics had an influence on non-recruitment: non-white (Adjusted Odds Ratio 1.25-1.67, depending on operation); most deprived socio-economic group (OR 1.11-1.23); aged over 75 years (OR 1.28-1.79). However, there was no statistically significant association between providers' recruitment rates and patients' pre-operative clinical need.ConclusionsAccurate recruitment rates require the use of linked data to establish consistent inclusion criteria for numerators and denominators. Non-recruitment will bias comparisons of providers' pre-operative case-mix and may bias comparisons of outcomes if unmeasured confounders are not evenly distributed between providers. It is important, therefore, to strive for high recruitment rates.

Dataset Information

Do Differential Response Rates to Patient Surveys Between Organizations Lead to Unfair Performance Comparisons?: Evidence From the English Cancer Patient Experience Survey.

Publications

Do Differential Response Rates to Patient Surveys Between Organizations Lead to Unfair Performance Comparisons?: Evidence From the English Cancer Patient Experience Survey.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets