Dataset Information

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care.

ABSTRACT: BACKGROUND:Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention's effectiveness when compared with A&F only. OBJECTIVE:To understand the mechanisms through which A&F with action implementation toolbox facilitates action planning by ICUs to increase A&F effectiveness. METHODS:We extracted all individual actions from action plans developed by ICUs that received A&F with (n?=?10) and without (n?=?11) toolbox for 6 months and classified them using Clinical Performance Feedback Intervention Theory. We held semi-structured interviews with participants during the trial. We compared the number and type of planned and completed actions between study groups and explored barriers and facilitators to effective action planning. RESULTS:ICUs with toolbox planned more actions directly aimed at improving practice (p?=?0.037) and targeted a wider range of practice determinants compared to ICUs without toolbox. ICUs with toolbox also completed more actions during the study period, but not significantly (p?=?0.142). ICUs without toolbox reported more difficulties in identifying what actions they could take. Regardless of the toolbox, all ICUs still experienced barriers relating to the feedback (low controllability, accuracy) and organisational context (competing priorities, resources, cost). CONCLUSIONS:The toolbox helped health professionals to broaden their mindset about actions they could take to change clinical practice. Without the toolbox, professionals tended to focus more on feedback verification and exploring solutions without developing intentions for actual change. All feedback recipients experienced organisational barriers that inhibited eventual completion of actions. TRIAL REGISTRATION:ClinicalTrials.gov, NCT02922101 . Registered on 26 September 2016.

SUBMITTER: Gude WT

PROVIDER: S-EPMC6751678 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care.

Gude Wouter T WT Roos-Blom Marie-José MJ van der Veer Sabine N SN Dongelmans Dave A DA de Jonge Evert E Peek Niels N de Keizer Nicolette F NF

Implementation science : IS 20190918 1

<h4>Background</h4>Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention's effectiveness when compared with A&F only.<h4>Objective< ...[more]

PMID: 31533841

Similar Datasets

Project description:Audit and feedback is often used as a strategy to improve quality of care, however, its effects are variable and often marginal. In order to learn how to design and deliver effective feedback, we need to understand their mechanisms of action. This theory-informed study will investigate how electronic audit and feedback affects improvement intentions (i.e. information-intention gap), and whether an action implementation toolbox with suggested actions and materials helps translating those intentions into action (i.e. intention-behaviour gap). The study will be executed in Dutch intensive care units (ICUs) and will be focused on pain management.We will conduct a laboratory experiment with individual ICU professionals to assess the impact of feedback on their intentions to improve practice. Next, we will conduct a cluster randomised controlled trial with ICUs allocated to feedback without or feedback with action implementation toolbox group. Participants will not be told explicitly what aspect of the intervention is randomised; they will only be aware that there are two variations of providing feedback. ICUs are eligible for participation if they submit indicator data to the Dutch National Intensive Care Evaluation (NICE) quality registry and agree to allocate a quality improvement team that spends 4 h per month on the intervention. All participating ICUs will receive access to an online quality dashboard that provides two functionalities: gaining insight into clinical performance on pain management indicators and developing action plans. ICUs with access to the toolbox can develop their action plans guided by a list of potential barriers in the care process, associated suggested actions, and supporting materials to facilitate implementation of the actions. The primary outcome measure for the laboratory experiment is the proportion of improvement intentions set by participants that are consistent with recommendations based on peer comparisons; for the randomised trial it is the proportion of patient shifts during which pain has been adequately managed. We will also conduct a process evaluation to understand how the intervention is implemented and used in clinical practice, and how implementation and use affect the intervention's impact.The results of this study will inform care providers and managers in ICU and other clinical settings how to use indicator-based performance feedback in conjunction with an action implementation toolbox to improve quality of care. Within the ICU context, this study will produce concrete and directly applicable knowledge with respect to what is or is not effective for improving pain management, and under which circumstances. The results will further guide future research that aims to understand the mechanisms behind audit and feedback and contribute to identifying the active ingredients of successful interventions.ClinicalTrials.gov NCT02922101 . Registered 26 September 2016.

Project description:BackgroundAudit and feedback (A&F) is a widely used implementation strategy. Understanding mechanisms of action of A&F increases the likelihood that the strategy will lead to implementation of an evidence-based practice. We therefore sought to understand one hospital's experience selecting and implementing an A&F intervention, to determine the implementation strategies that were used by staff and to specify the mechanism of action of those implementation strategies using causal pathway models, with the ultimate goal of improving acute stroke treatment practices.MethodsWe selected an A&F strategy in a hospital, initially based on implementation determinants and staff consideration of their performance on acute stroke treatment measures. After 7 months of A&F, we conducted semi-structured interviews of hospital providers and administrative staff to understand how it contributed to implementing guideline-concordant acute stroke treatment (medication named tissue plasminogen activator). We coded the interviews to identify the implementation strategies that staff used following A&F and to assess their mechanisms of action.ResultsWe identified five implementation strategies that staff used following the feedback intervention. These included (1) creating folders containing the acute stroke treatment protocol for the emergency department, (2) educating providers about the protocol for acute stroke, (3) obtaining computed tomography imaging of stroke patients immediately upon emergency department arrival, (4) increasing access to acute stroke medical treatment in the emergency department, and (5) providing additional staff support for implementation of the protocol in the emergency department. We identified enablement, training, and environmental restructuring as mechanisms of action through which the implementation strategies acted to improve guideline-concordant and timely acute stroke treatment.ConclusionsA&F of a hospital's acute stroke treatment practices generated additional implementation strategies that acted through various mechanisms of action. Future studies should focus on how initial implementation strategies can be amplified through internal mechanisms.

Project description:BackgroundThere is a significant variation among individual primary care providers in prescribing of potentially problematic, low-value medicines which cause avoidable patient harm. Audit and feedback is generally effective at improving prescribing. However, progress has been hindered by research waste, leading to unanswered questions about how to include audit and feedback for specific problems and circumstances. Trials of different ways of providing audit and feedback in implementation laboratories have been proposed as a way of improving population healthcare while generating robust evidence on feedback effects. However, there is limited experience in their design and delivery.AimTo explore priorities, feasibility, and ethical challenges of establishing a primary care prescribing audit and feedback implementation laboratory.Design and settingTwo-stage Delphi consensus process involving primary care pharmacy leads, audit and feedback researchers, and patient and public.MethodParticipants initially scored statements relating to priorities, feasibility, and ethical considerations for an implementation laboratory. These covered current feedback practice, priority topics for feedback, usefulness of feedback in improving prescribing and different types of prescribing data, acceptability and desirability of different organization levels of randomization, options for trial consent, different methods of delivering feedback, and interest in finding out how effective different ways of presenting feedback would be. After receiving collated results, participants then scored the items again. The consensus was defined using the GRADE criteria. The results were analyzed by group and overall score.ResultsFourteen participants reached consensus for 38 out of 55 statements. Addressing antibiotic and opioid prescribing emerged as the highest priorities for action. The panel supported statements around addressing high-priority prescribing issues, taking an "opt-out" approach to practice consent if waiving consent was not permitted, and randomizing at lower rather than higher organizational levels. Participants supported patient-level prescribing data and further research evaluating most of the different feedback methods we presented them with.ConclusionsThere is a good level of support for evaluating a wide range of potential enhancements to improve the effects of feedback on prescribing. The successful design and delivery of a primary care audit and feedback implementation laboratory depend on identifying shared priorities and addressing practical and ethical considerations.

Project description:IntroductionExtensive antibiotic use makes the intensive care unit (ICU) an important focus for antibiotic stewardship programs. The aim of this study was to develop a set of actionable quality indicators for appropriate antibiotic use at ICUs and an implementation toolbox, which can be used to assess and improve the appropriateness of antibiotic use in the treatment of adult patients at an ICU.MethodsA four round modified-RAND Delphi procedure was used. Potential indicators were identified by a multidisciplinary panel of 15 Dutch experts, from international literature and guidelines. Using an online survey, the identified indicators were rated on three criteria: relevance, actionability and feasibility. Experts discussed and rated the indicators for the second time during a face-to-face consensus meeting. During a final consensus meeting the toolbox was developed, containing potential barriers and improvement strategies which were identified using a validated checklist by Flottorp et al., and if available also containing supporting material.ResultsThe first round resulted in 24 potential indicators. After the final meeting a set of three process indicators, one structure indicator and one quantity metric remained: 1) perform at least two sets of blood cultures before start of empirical systemic therapy; 2) perform therapeutic drug monitoring in patients treated with vancomycin or aminoglycosides; 3) perform surveillance cultures if selective digestive or oropharyngeal decontamination is applied at the ICU; 4) biannual face-to-face meetings between ICU and microbiology staff in which local resistance rates are discussed; and 5) quantitative antibiotic use at the ICU expressed in days of therapy (DOT). The toolbox contains 24 unique barriers and 37 improvement strategies.ConclusionsOur study identified a set of four actionable quality indicators and one quantity metric, together with an implementation toolbox, to improve appropriate antibiotic use at ICUs.

Project description:BackgroundAudit and feedback (A&F) often successfully enhances health professionals' intentions to improve quality of care but does not consistently lead to practice changes. Recipients often cite data credibility and limited resources as barriers impeding their ability to act upon A&F, suggesting the intention-to-action gap manifests while recipients are interacting with their data. While attention has been paid to the role feedback and contextual variables play in contributing to (or impeding) success, we lack a nuanced understanding of how healthcare professionals interact with and process clinical performance data.MethodsWe used qualitative, semi-structured interviews guided by Normalization Process Theory (NPT). Questions explored the role of data in quality improvement, experiences with the A&F report, perceptions of the data, and interpretations and reflections. Interviews were audio-recorded and transcribed verbatim. Data were analyzed using a combination of inductive and deductive strategies using reflexive thematic analysis informed by a constructivist paradigm.ResultsHealthcare professional characteristics (individual quality improvement capabilities and beliefs about data) seem to influence engagement with A&F to a greater degree than feedback variables (i.e., delivered by peers) and observed contextual factors (i.e., strong quality improvement culture). Most participants lacked the capabilities to interpret practice-level data in an actionable way despite a motivation to engage meaningfully. Reasons for the intention-to-action gap included challenges interpreting longitudinal data, appreciating the nuances of common data sources, understanding how aggregate data provides insights into individualized care, and identifying practice-level actions to improve quality. These factors limited effective cognitive participation and collective action, as outlined in NPT.ConclusionsA well-designed A&F intervention is necessary but not sufficient to inform practice changes. A&F initiatives must include co-interventions to address recipient characteristics (i.e., beliefs and capabilities) and context to optimize impact. Effective strategies to overcome the intention-to-action gap may include modelling how to use A&F to inform practice change, providing opportunities for social interaction relating to the A&F, and circulating examples of effective actions taken in response to A&F. More broadly, undergraduate medical education and post-graduate training must ensure physicians are equipped with QI capabilities, with an emphasis on the skills required to interpret and act on practice-level data.

Project description:Background To address low state physical education (PE) quantity and quality law implementation in elementary schools, the New York City Department of Education (NYCDOE) delivered a multilevel intervention (PE Works; 2015-2019), which included a district-led audit of school PE-law implementation, feedback, and coaching with principals. Using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) implementation science framework, we assessed the primary multilevel drivers of success for this approach in increasing adherence to PE quantity and quality law. Methods We conducted in-depth, semi-structured interviews with district-level personnel (n=17), elementary school administrators (n=18), and PE teachers (n=6) in 2020-21. Results Interview results suggested several key RE-AIM drivers of successful PE law implementation. Reach: Ensure higher-need schools receive the necessary initial support to improve PE and later focus on lower-need schools. Effectiveness: Provide support tailored to school needs, not penalties, to improve PE. Adoption: Increase the priority of PE at both district and school levels (e.g., audit and feedback, themselves, appear to elevate PE’s priority). Streamline data collection and feedback reports; collecting/reporting too much information is burdensome and leads to lack of focus. Involve qualified (i.e., skilled in both school administration and PE programming/pedagogy) district-level personnel to work collaboratively with schools. Implementation: Build strong, trusting district-school relationships. Maintenance: Provide ongoing district-level support to schools and involve parents to advocate for quality PE. Conclusions PE audits, feedback, and coaching (PEAFC) can guide schools in establishing long-term plans for successfully implementing PE-related law. Future research should examine the impact of PEAFC elsewhere (e.g., secondary schools, other districts). Supplementary Information The online version contains supplementary material available at 10.1186/s12966-023-01479-1.

Project description:BackgroundHigh quality goal setting in stroke rehabilitation is vital, but challenging to deliver. The G-AP framework (including staff training and a stroke survivor held G-AP record) guides patient centred goal setting with stroke survivors in community rehabilitation teams. We found G-AP was acceptable, feasible to deliver and clinically useful in one team. The aim of this study was to conduct a mixed methods investigation of G-AP implementation in diverse community teams prior to a large-scale evaluation.MethodsWe approached Scottish community rehabilitation teams to take part. Following training, G-AP was delivered to stroke survivors within participating teams for 6 months. We investigated staff experiences of G-AP training and its implementation using focus groups and a training questionnaire. We investigated fidelity of G-AP delivery through case note review. Focus group data were analysed using a Framework approach; identified themes were mapped into Normalisation Process Theory constructs. Questionnaire and case note data were analysed descriptively.ResultsWe recruited three teams comprising 55 rehabilitation staff. Almost all staff (93%, 51/55) participated in G-AP training; of those, 80% (n = 41/51) completed the training questionnaire. Training was rated as 'good' or 'very good' by almost all staff (92%, n = 37/41). G-AP was broadly implemented as intended in two teams. Implementation facilitators included - G-AP 'made sense'; repetitive use of G-AP in practice; flexible G-AP delivery and positive staff appraisals of G-AP impact. G-AP failed to gain traction in the third team. Implementation barriers included - delays between G-AP training and implementation; limited leadership engagement; a poor 'fit' between G-AP and the team organisational structure and simultaneous delivery of other goal setting methods. Staff recommended (i) development of training to include implementation planning; (ii) ongoing local implementation review and tailoring, and (iii) development of electronic and aphasia friendly G-AP records.ConclusionsThe interaction between G-AP and the practice setting is critical to implementation success or failure. Whilst facilitators support implementation success, barriers can collectively act as implementation "deal breakers". Local G-AP implementation efforts should be planned, monitored and tailored. These insights can inform implementation of other complex interventions in community rehabilitation settings.

Project description:BackgroundUsing patient audio recordings of medical visits to provide clinicians with feedback on their attention to patient life context in care planning can improve health care delivery and outcomes, and reduce costs. However, such an initiative can raise concerns across stakeholders about surveillance, intrusiveness and merit. This study examined the perspectives of patients, physicians and other clinical staff, and facility leaders over 3 years at six sites during the implementation of a patient-collected audio quality improvement program designed to improve patient-centered care in a non-threatening manner and with minimal effort required of patients and clinicians.MethodsPatients were invited during the first and third year to complete exit surveys when they returned their audio recorders following visits, and clinicians to complete surveys annually. Clinicians were invited to participate in focus groups in the first and third years. Facility leaders were interviewed individually during the last 6 months of the study.ResultsThere were a total of 12 focus groups with 89 participants, and 30 leadership interviews. Two hundred fourteen clinicians and 800 patients completed surveys. In a qualitative analysis of focus group data employing NVivo, clinicians initially expressed concerns that the program could be disruptive and/or burdensome, but these diminished with program exposure and were substantially replaced by an appreciation for the value of low stakes constructive feedback. They were also significantly more confident in the value of the intervention in the final year (p = .008), more likely to agree that leadership supports continuous improvement of patient care and gives feedback on outcomes (p = .02), and at a time that is convenient (p = .04). Patients who volunteered sometimes expressed concerns they were "spying" on their doctors, but most saw it as an opportunity to improve care. Leaders were supportive of the program but not yet prepared to commit to funding it exclusively with facility resources.ConclusionsA patient-collected audio program can be implemented when it is perceived as safe, not disruptive or burdensome, and as contributing to better health care.

Dataset Information

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care.

Publications

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets