Dataset Information

A process evaluation of the improving wisely intervention: a peer-to-peer data intervention to reduce overuse in surgery.

ABSTRACT:

Background

The Improving Wisely intervention is a peer-to-peer audit and feedback intervention to reduce overuse of Mohs Micrographic Surgery (MMS). The objective of this study was to conduct a process evaluation to evaluate Mohs surgeons' perceptions of the implementation quality and perceived impact of the Improving Wisely intervention.

Methods

Surgeons in the Improving Wisely intervention arm, comprised of members of the American College of Mohs Surgeons (ACMS) who co-led the intervention, were invited to complete surveys and key informant interviews. Participants described perceptions of implementation quality (evaluated via dose, quality of implementation, reach and participant responsiveness), perceived impact of the Improving Wisely intervention (evaluated on a 1-5 Likert and qualitatively), and barriers and facilitators to changing surgeons' clinical practice patterns to reduce Mohs overuse.

Results

Seven hundred thirty-seven surgeons participated in the survey. 89% were supportive of the intervention. Participants agreed that the intervention would improve patient care and reduce the annual costs of Mohs surgery. Thirty surgeons participated in key informant interviews. 93% were interested in receiving additional data reports in the future. Participants recommended the reports be disseminated annually, that the reports be expanded to include appropriateness data, and that the intervention be extended to non ACMS members. Six themes identifying factors impacting potential MMS overuse were identified.

Conclusions

Participants were strongly supportive of the intervention. We present the template used to design and implement the Improving Wisely intervention and provide suggestions for specialty societies interested in leading similar quality improvement interventions among their members.

SUBMITTER: Fahim C

PROVIDER: S-EPMC7845024 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A process evaluation of the improving wisely intervention: a peer-to-peer data intervention to reduce overuse in surgery.

Fahim Christine C Bruhn William E WE Albertini John G JG Makary Marty A MA

BMC health services research 20210129 1

<h4>Background</h4>The Improving Wisely intervention is a peer-to-peer audit and feedback intervention to reduce overuse of Mohs Micrographic Surgery (MMS). The objective of this study was to conduct a process evaluation to evaluate Mohs surgeons' perceptions of the implementation quality and perceived impact of the Improving Wisely intervention.<h4>Methods</h4>Surgeons in the Improving Wisely intervention arm, comprised of members of the American College of Mohs Surgeons (ACMS) who co-led the i ...[more]

PMID: 33514362

Similar Datasets

Project description:ImportanceMohs micrographic surgery (MMS) is a skin cancer treatment that uses staged excisions based on margin status. Wide surgeon-level variation exists in the mean number of staged resections used to treat a tumor, resulting in a cost disparity and question of appropriateness.ObjectiveTo evaluate the effectiveness of a behavioral intervention aimed at reducing extreme overuse in MMS, as defined by the specialty society, by confidentially sharing stages-per-case performance data with individual surgeons benchmarked to their peers nationally.Design, setting, and participantsThis nonrandomized controlled intervention study included 2329 US surgeons who performed MMS procedures from January 1, 2016, to March 31, 2018. Physicians were identified using a 100% capture of Medicare Part B claims. The intervention group included physicians affiliated with the American College of Mohs Surgery, and the control group included physicians not affiliated with the American College of Mohs Surgery.InterventionsIndividualized performance reports were delivered to all outlier surgeons, defined by the specialty society as those with mean stages per case 2 SDs above the mean, and inlier surgeons in the intervention group.Main outcomes and measuresThe primary outcome was surgeon-level change in mean stages per case between the prenotification (January 2016 to January 2017) and postnotification (March 2017 to March 2018) periods. A multivariable linear regression model was used to evaluate the association of notification with this surgeon-level outcome. The surgeon-level metric of mean stages per case was not risk adjusted. The mean Medicare cost savings associated with changes in practice patterns were calculated.ResultsOf the 2329 included surgeons, 1643 (70.5%) were male and 2120 (91.0%) practiced in metropolitan areas. In the intervention group (n = 1045), 53 surgeons (5.1%) were outliers; in the control group (n = 1284), 87 surgeons (6.8%) were outliers. Among the outliers in the intervention group, 44 (83%) demonstrated a reduction in mean stages per case compared with 60 outliers in the control group (69%; difference, 14%; 95% CI of difference, -1 to 27; P = .07). There was a mean stages-per-case reduction of 12.6% among outliers in the intervention group compared with 9.0% among outliers in the control group, and outliers in the intervention group had an adjusted postintervention differential decrease of 0.14 stages per case (95% CI, -0.19 to -0.09; P = .002). The total administrative cost of the intervention program was $150 000, and the estimated reduction in Medicare spending was $11.1 million.Conclusions and relevanceSharing personalized practice pattern data with physicians benchmarked to their peers can reduce overuse of MMS among outlier physicians.

Project description:BACKGROUND:Improving the quality and safety of perioperative care is a global priority. The Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial was a stepped-wedge cluster randomised trial of a quality improvement (QI) programme to improve 90-day survival for patients undergoing emergency abdominal surgery in 93 hospitals in the UK National Health Service. METHODS:The aim of this process evaluation is to describe how the EPOCH intervention was planned, delivered and received, at both cluster and local hospital levels. The QI programme comprised of two interventions: a care pathway and a QI intervention to aid pathway implementation, focussed on stakeholder engagement, QI teamwork, data analysis and feedback and applying the model for improvement. Face-to-face training and online resources were provided to support senior clinicians in each hospital (QI leads) to lead improvement. For this evaluation, we collated programme activity data, administered an exit questionnaire to QI leads and collected ethnographic data in six hospitals. Qualitative data were analysed with thematic or comparative analysis; quantitative data were analysed using descriptive statistics. RESULTS:The EPOCH trial did not demonstrate any improvement in survival or length of hospital stay. Whilst the QI programme was delivered as planned at the cluster level, self-assessed intervention fidelity at the hospital level was variable. Seventy-seven of 93 hospitals responded to the exit questionnaire (60 from a single QI lead response on behalf of the team); 33 respondents described following the QI intervention closely (35%) and there were only 11 of 37 care pathway processes that >?50% of respondents reported attempting to improve. Analysis of qualitative data suggests QI leads were often attempting to deliver the intervention in challenging contexts: the social aspects of change such as engaging colleagues were identified as important but often difficult and clinicians frequently attempted to lead change with limited time or organisational resources. CONCLUSIONS:Significant organisational challenges faced by QI leads shaped their choice of pathway components to focus on and implementation approaches taken. Adaptation causing loss of intervention fidelity was therefore due to rational choices made by those implementing change within constrained contexts. Future large-scale QI programmes will need to focus on dedicating local time and resources to improvement as well as on training to develop QI capabilities. EPOCH TRIAL REGISTRATION:ISRCTN80682973 https://doi.org/10.1186/ISRCTN80682973 Registered 27 February 2014 and Lancet protocol 13PRT/7655.

Project description:BackgroundSmoking in pregnancy causes harm to mother and baby. Despite evidence from trials of what helps women quit, implementation in the real world has been hard to achieve. An evidence-based intervention, babyClear©, involving staff training, universal carbon monoxide monitoring, opt-out referral to smoking cessation services, enhanced follow-up protocols and a risk perception tool was introduced across North East England. This paper presents the results of the qualitative analyses, reporting acceptability of the system changes to staff, as well as aids and hindrances to implementation and normalization of this complex intervention.MethodsProcess evaluation was used to complement an effectiveness study. Interviews with maternity and smoking cessation services staff and observations of training were undertaken. Normalization Process Theory (NPT) was used to frame the interview guides and analysis. NPT is an empirically-derived theory, developed by sociologists, that uses four concepts to understand the process of routinising new practices.ResultsStaff interviews took place across eight National Health Service trusts at a time of widespread restructuring in smoking cessation services. Principally interviewees worked in maternity (n = 63) and smoking cessation services (n = 35). Five main themes, identified inductively, influenced the implementation: 1) initial preparedness of the organisations; 2) staff training; 3) managing partnership working; 4) resources; 5) review and planning for sustainability.ConclusionsNPT was used to show that the babyClear© package was acceptable to staff in a range of organisations. Illustrated in Themes 1, 2 & 3, staff welcomed ways to approach pregnant women about their smoking, without damaging their professional relationship with them. Predicated on producing individual behaviour change in women, the intervention does this largely through reorganising and standardising healthcare systems that are required to implement best practice guidelines. Changing organisational systems requires belief and commitment from staff, so that they set up and maintain practical adjustments to their practice and are reflective about adapting themselves and the work context as new challenges are encountered. The ongoing challenge is to identify and maintain the elements of the intervention package which are essential for its effectiveness and how to tailor them to local circumstances and resources without compromising its core ingredients.

Project description:ProblemMedical school can be a socially isolating experience, particularly for students underrepresented in medicine. Social isolation and perceptions of not belonging can negatively impact students' academic performance and well-being. Therefore, interventions are needed to support students and these efforts should be appealing, brief, and low-burden.InterventionGuided by evidence-based approaches, we developed the Build & Belong intervention for medical students as a brief peer-to-peer approach that consisted of four components. First, M3 and M4 students wrote reflections on belonging in medical school. Second, M3 and M4 students video recorded messages for M1 and M2 students using their written reflections. Third, M1 and M2 students watched and discussed the videos in small groups. Fourth, the M1 and M2 students wrote letters to future students. Our intervention differs from previous student belonging interventions in the peer delivery of messages.ContextThe Build & Belong intervention aimed to improve medical students' social belongingness. Using a longitudinal observational study design, the intervention was piloted at a medical school in the Mid-Atlantic United States in 2017-2018. Students completed surveys before and after the intervention. Paired samples tests (t-tests and Wilcoxon) assessed pre- to post-intervention changes in social isolation, social connectedness, and social assurance.ImpactAmong 63 medical students, with 25.9% from backgrounds underrepresented in medicine, we assessed follow-up outcomes in 38 students. Social isolation scores significantly decreased from baseline (M = 54.8, SD = 7.06) to follow-up (M = 51.3, SD = 6.67; p < .001). Social isolation changes were evident regardless of sex, although males reported a greater reduction (M Δ = -5.32, p < .001) than females (M Δ = -2.79, p = .014). Black/African American students had the largest reduction in social isolation (M Δ = -7.24, p = .010). Social assurance and connectedness scores did not change significantly between baseline and follow-up. Medical students appeared to resonate with messages delivered by more experienced peers (M3s and M4s), particularly messages that normalized feelings of not belonging and strategies to reduce those feelings.Lessons learnedThe Build & Belong intervention appears to reduce social isolation scores among medical students. This pilot test of the Build & Belong intervention provides initial evidence of the effectiveness of a brief, low-cost intervention. Build & Belong may provide a scalable strategy to reduce medical students' social isolation. Our peer-based approach is distinct from administrator-led strategies; peers were seen as trusted and reliable sources of information about belonging and ways to overcome the challenges experienced during medical school.

Project description:BackgroundResearch suggests that collaborative and tailored approaches with external expertise are important to process implementations. We therefore performed a process evaluation of an intervention using participatory action research, tailored information provision, and external coaching to reduce inappropriate psychotropic drug use among nursing home residents with dementia. The process evaluation was conducted alongside a randomized controlled trial assessing the utility of this approach.MethodsWe used Leontjevas' model of process evaluation to guide data collection and analysis, focusing on the relevance and feasibility, extent of performance, and barriers and facilitators to implementation. Data on the relevance and feasibility and on the extent of performance were collected using a questionnaire targeting internal project leaders at nursing homes and our external coaches. Implementation barriers and facilitators were identified by individual semi-structured interviews. The Consolidated Framework for Implementation Research was used to structure and describe the identified barriers and facilitators.ResultsThe intervention was viewed positively, but it was also considered time consuming due to the involvement of many people and designing a tailored action and implementation plan was viewed as complex. The extent of performance differed between nursing homes. Delays in implementation and suboptimal execution of actions may have reduced effectiveness of the RID intervention in some nursing homes. Barriers to implementation were reorganizations, staff turnover, communication issues, unclear expectations, and perceived time pressures. Implementation also depended on the involvement and skills of key stakeholders, and organizations' readiness to change. Although external coaches stimulated implementation, their additional value was rated variably across organizations.ConclusionsBarriers to implementation occurred on several levels and some barriers appear to be inherent to the nursing home environment and could be points of leverage of future implementation trajectories. This underlines the importance of assessing and supporting organizations in their readiness to change. Sensitivity analyses, taking into account the week in which nursing homes started with implementation and the degree to which actions were implemented as intended, will be appropriate in the effect analyses of the trial.

Project description:Purpose: Psychological interventions targeting fear of cancer recurrence (FCR) are effective in reducing fear and distress. Process evaluations are an important, yet scarce adjunct to published intervention trials, despite their utility in guiding the interpretation of study outcomes and optimizing intervention design for broader implementation. Accordingly, this paper reports the findings of a process evaluation conducted alongside a randomized controlled trial of a psychological intervention for melanoma patients. Methods: Men and women with a history of Stage 0-II melanoma at high-risk of developing new primary disease were recruited via High Risk Melanoma Clinics across Sydney, Australia and randomly allocated to receive the psychological intervention (n = 80) or usual care (n = 84). Intervention participants received a tailored psycho-educational resource and three individual psychotherapeutic sessions delivered via telehealth. Qualitative and quantitative data on intervention context, processes, and delivery (reach, dose, and fidelity), and mechanisms of impact (participant responses, moderators of outcome) were collected from a range of sources, including participant surveys, psychotherapeutic session audio-recordings, and clinical records. Results: Almost all participants reported using the psycho-educational resource (97%), received all intended psychotherapy sessions (96%), and reported high satisfaction with both intervention components. Over 80% of participants would recommend the intervention to others, and a small proportion (4%) found discussion of melanoma-related experiences confronting. Perceived benefits included enhanced doctor-patient communication, talking more openly with family members about melanoma, and improved coping. Of potential moderators, only higher FCR severity at baseline (pre-intervention) was associated with greater reductions in FCR severity (primary outcome) at 6-month follow-up (primary endpoint). Conclusions: Findings support the acceptability and feasibility of a psychological intervention to reduce FCR amongst individuals at high risk of developing another melanoma. Implementation into routine melanoma care is an imperative next step, with FCR screening recommended to identify those most likely to derive the greatest psychological benefit.

Project description:BackgroundThe Targeted Intervention Interdisciplinary Model for Evaluation and Treatment of Neuropsychiatric Symptoms (TIME) has recently in a three-month cluster randomised controlled trial demonstrated reduction in agitation in nursing home residents with dementia. To ease replication and future implementation, and to clarify possible causal mechanisms, we performed a process evaluation of the intervention based on the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, Maintenance).MethodsAn exploratory and a quasi-experimental design with mixed methods were used. The RE-AIM dimensions were explored by questionnaires to 807 staff members and 46 leading ward nurses in both the intervention nursing homes (INH) and the control nursing homes (CNH), before the start of the trial (baseline), and six and 12 months later. These questionnaires assessed data regarding the reach, effectiveness (staff level) and adoption dimensions. To assess implementation, we used a checklist for performance of the main components in TIME and analysed the minutes from 84 case conferences in the INH. To explore adoption and maintenance, five focus group interviews with 32 participants from the staff in the INH were conducted three to 6 months after the end of the trial.ResultsReach: On average 61% (SD 22) of the staff in each ward in the INH attended the training sessions. Effectiveness at staff level: There were no between-group differences throughout the study period for attitudes towards dementia, perceived competence or perception of mastery and social interaction. Adoption: 16 of the 17 INH completed the intervention.Implementation75% or more of the components of TIME were performed for 91% of the included residents. Maintenance: Most of the nursing homes used TIME three to 6 months after the end of the trial. An easy to grasp model and an engaged leadership facilitated the intervention and maintenance.ConclusionsA high degree of reach, adoption, implementation and maintenance contributed to the effectiveness of TIME at resident level. One other causal assumption of the effectiveness of TIME is the development in the staff of a new, shared and situated knowledge about each individual resident, not reflected by measurements in general knowledge and attitudes.Trial registrationThe trial was registered January 6, 2016 with ClinicalTrials.gov ( NCT02655003 ).

Dataset Information

A process evaluation of the improving wisely intervention: a peer-to-peer data intervention to reduce overuse in surgery.

Background

Methods

Results

Conclusions

Publications

A process evaluation of the improving wisely intervention: a peer-to-peer data intervention to reduce overuse in surgery.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets