Dataset Information

Delta Studies: Expanding the Concept of Deviance Studies to Design More Effective Improvement Interventions.

ABSTRACT:

Background

The effects of improvement (implementation and de-implementation) interventions are often modest. Although positive and negative deviance studies have been extensively used in improvement science and quality improvement efforts, conceptual and methodological innovations are needed to improve our ability to use information about variation in quality to design more effective interventions.

Objective

We describe a novel mixed methods extension of the deviance study we term "delta studies." Delta studies seek to quantitatively identify sites that have recently changed from low performers to high performers, or vice versa, in order to qualitatively learn about active strategies that produced recent change, challenges change agents faced and how they overcame them, and where applicable, the causes of recent deterioration in performance-information intended to inform the design of improvement interventions for deployment in low performing sites. We provide examples of lessons learned from this method that may have been missed with traditional positive or negative deviance designs.

Design

Considerations for quantitatively identifying delta sites are described including which quality metrics to track, over what timeframe to observe change, how to account for reliability of observed change, consideration of patient volume and initial performance as implementation context factors, and how to define clinically meaningful change. Methods to adapt qualitative protocols by integrating quantitative information about change in performance are also presented. We provide sample data and R code that can be used to graphically display distributions of initial status, change, and volume that are essential to delta studies.

Participants

Patients and facilities of the US Veterans Health Administration.

Key results

As an example, we discuss what decisions we made regarding the delta study design considerations in a funded study of low-value preoperative testing. The method helped us find sites that had recently reduced the burden of low-value testing, and learn about the strategies they employed and challenges they faced.

Conclusions

The delta study concept is a promising mixed methods innovation to efficiently and effectively identify improvement strategies and other factors that have actually produced change in real-world settings.

SUBMITTER: Harris AHS

PROVIDER: S-EPMC7878588 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:STUDY DESIGN:Markov model. OBJECTIVE:Examine the 1-year effectiveness and cost-effectiveness (societal and payer perspectives) of adding nonpharmacologic interventions for chronic low back pain (CLBP) to usual care using a decision analytic model-based approach. SUMMARY OF BACKGROUND DATA:Treatment guidelines now recommend many safe and effective nonpharmacologic interventions for CLBP. However, little is known regarding their effectiveness in subpopulations (e.g., high-impact chronic pain patients), nor about their cost-effectiveness. METHODS:The model included four health states: high-impact chronic pain (substantial activity limitations); no pain; and two others without activity limitations, but with higher (moderate-impact) or lower (low-impact) pain. We estimated intervention-specific transition probabilities for these health states using individual patient-level data from 10 large randomized trials covering 17 nonpharmacologic therapies. The model was run for nine 6-week cycles to approximate a 1-year time horizon. Quality-adjusted life-year weights were based on six-dimensional health state short form scores; healthcare costs were based on 2003 to 2015 Medical Expenditure Panel Survey data; and lost productivity costs used in the societal perspective were based on reported absenteeism. Results were generated for two target populations: (1) a typical baseline mix of patients with CLBP (25% low-impact, 35% moderate-impact, and 40% high-impact chronic pain) and (2) high-impact chronic pain patients. RESULTS:From the societal perspective, all but two of the therapies were cost effective (<$50,000/quality-adjusted life-year) for a typical patient mix and most were cost saving. From the payer perspective fewer were cost saving, but the same number was cost-effective. Assuming all patients in the model have high-impact chronic pain increases the effectiveness and cost-effectiveness of most, but not all, therapies indicating that substantial benefits are possible in this subpopulation. CONCLUSION:Modeling leverages the evidence produced from clinical trials to provide more information than is available in the published studies. We recommend modeling for all existing studies of nonpharmacologic interventions for CLBP. LEVEL OF EVIDENCE:4.