How to Make Feedback More Effective? Qualitative Findings from Pilot Testing of an Audit and Feedback Report for Endoscopists.
Ontology highlight
ABSTRACT:
SUBMITTER: Webster F
PROVIDER: S-EPMC5045985 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundWe describe the creation and evaluation of a personal audit and feedback (A&F) tool for anesthesiologists.MethodsA survey aimed at capturing barriers for personal improvement efforts and feedback preferences was administered to attending anesthesiologists. The results informed the design and implementation of 4 dashboards that display information on individual practice characteristics as well as comparative performance on several quality metrics. The dashboards' usability was then tested using the human-centered design framework.ResultsAnesthesiologists listed lack of information on current practice as the main barrier for improvement. Regarding usability, participants gave the dashboards an average score of 3.8 (scale 1-5) on consistency, learnability, and information organization, and performed the assigned tasks well, with an average score of 89% (range 79-100%).ConclusionsWe describe the design, implementation, and usability testing of an innovative tool that utilizes data derived from the electronic health record (EHR) system to provide A&F to anesthesiology providers.
| S-EPMC6951966 | biostudies-literature
Project description:BackgroundAudit and feedback aims to improve patient care by comparing healthcare performance against explicit standards. It is used to monitor and improve patient care, including through National Clinical Audit (NCA) programmes in the UK. Variability in effectiveness of audit and feedback is attributed to intervention design; separate randomised trials to address multiple questions about how to optimise effectiveness would be inefficient. We evaluated different feedback modifications to identify leading candidates for further "real-world" evaluation.MethodsUsing an online fractional factorial screening experiment, we randomised recipients of feedback from five UK NCAs to different combinations of six feedback modifications applied within an audit report excerpt: use effective comparators, provide multimodal feedback, recommend specific actions, provide optional detail, incorporate the patient voice, and minimise cognitive load. Outcomes, assessed immediately after exposure to the online modifications, included intention to enact audit standards (primary outcome, ranked on a scale of -3 to +3, tailored to the NCA), comprehension, user experience, and engagement.ResultsWe randomised 1241 participants (clinicians, managers, and audit staff) between April and October 2019. Inappropriate repeated participant completion occurred; we conservatively excluded participant entries during the relevant period, leaving a primary analysis population of 638 (51.4%) participants. None of the six feedback modifications had an independent effect on intention across the five NCAs. We observed both synergistic and antagonistic effects across outcomes when modifications were combined; the specific NCA and whether recipients had a clinical role had dominant influences on outcome, and there was an antagonistic interaction between multimodal feedback and optional detail. Among clinical participants, predicted intention ranged from 1.22 (95% confidence interval 0.72, 1.72) for the least effective combination in which multimodal feedback, optional detail, and reduced cognitive load were applied within the audit report, up to 2.40 (95% CI 1.88, 2.93) for the most effective combination including multimodal feedback, specific actions, patient voice, and reduced cognitive load.ConclusionPotentially important synergistic and antagonistic effects were identified across combinations of feedback modifications, audit programmes, and recipients, suggesting that feedback designers must explicitly consider how different features of feedback may interact to achieve (or undermine) the desired effects.Trial registrationInternational Standard Randomised Controlled Trial Number: ISRCTN41584028.
| S-EPMC9137082 | biostudies-literature
Project description:AKT3 is one of the major therapeutic targets in melanoma but clinically targeting AKT3 alone seems to be an ineffective therapeutic approach. To identify unique strategies to enhance the efficacy of targeting AKT3, a screen was undertaken where AKT3 was co-targeted with a panel of kinases important in melanoma development. The screen identified WEE1 as the most potent target that when inhibited along with AKT3 would enhance the efficacy of targeting AKT3 in melanoma. RNAi mediated inhibition of AKT3 and WEE1 synergistically inhibited the viability of melanoma cells leading to a 65-75% decrease in tumor development. This approach was effective by mechanistically modulating pathways associated with the transcription factors p53 and FOXM1. Simultaneously regulating the activity of these two transcriptionally driven pathways, cooperatively deregulated cell cycle control and DNA damage repair to synergistically kill melanoma cells. This study uniquely identifies a potential approach to improve the efficacy of targeting AKT3 in melanoma.
| S-EPMC5790369 | biostudies-literature
Project description:BackgroundThe use of clinical performance feedback to support quality improvement (QI) activities is based on the sound rationale that measurement is necessary to improve quality of care. However, concerns persist about the reliability of this strategy, known as Audit and Feedback (A&F) to support QI. If successfully implemented, A&F should reflect an iterative, self-regulating QI process. Whether and how real-world A&F initiatives result in this type of feedback loop are scarcely reported. This study aimed to identify barriers or facilitators to implementation in a team-based primary care context.MethodsSemi-structured interviews were conducted with key informants from team-based primary care practices in Ontario, Canada. At the time of data collection, practices could have received up to three iterations of the voluntary A&F initiative. Interviews explored whether, how, and why practices used the feedback to guide their QI activities. The Consolidated Framework for Implementation Research was used to code transcripts and the resulting frameworks were analyzed inductively to generate key themes.ResultsTwenty-five individuals representing 18 primary care teams participated in the study. Analysis of how the A&F intervention was used revealed that implementation reflected an incomplete feedback loop. Participation was facilitated by the reliance on an external resource to facilitate the practice audit. The frequency of feedback, concerns with data validity, the design of the feedback report, the resource requirements to participate, and the team relationship were all identified as barriers to implementation of A&F.ConclusionsThe implementation of a real-world, voluntary A&F initiative did not lead to desired QI activities despite substantial investments in performance measurement. In small primary care teams, it may take long periods of time to develop capacity for QI and future evaluations may reveal shifts in the implementation state of the initiative. Findings from the present study demonstrate that the potential mechanism of action of A&F may be deceptively clear; in practice, moving from measurement to action can be complex.
| S-EPMC6591867 | biostudies-literature
Project description:BackgroundAudit and feedback interventions in healthcare have been found to be effective, but there has been little progress with respect to understanding their mechanisms of action or identifying their key 'active ingredients.'DiscussionGiven the increasing use of audit and feedback to improve quality of care, it is imperative to focus further research on understanding how and when it works best. In this paper, we argue that continuing the 'business as usual' approach to evaluating two-arm trials of audit and feedback interventions against usual care for common problems and settings is unlikely to contribute new generalizable findings. Future audit and feedback trials should incorporate evidence- and theory-based best practices, and address known gaps in the literature.SummaryWe offer an agenda for high-priority research topics for implementation researchers that focuses on reviewing best practices for designing audit and feedback interventions to optimize effectiveness.
| S-EPMC3896824 | biostudies-literature
Project description:BackgroundPatient decision aids support people to make informed decisions between healthcare options. Personal stories provide illustrative examples of others' experiences and are seen as a useful way to communicate information about health and illness. Evidence indicates that providing information within personal stories affects the judgments and values people have, and the choices they make, differentially from facts presented in non-narrative prose. It is unclear if including narrative communications within patient decision aids enhances their effectiveness to support people to make informed decisions.MethodsA survey of primary empirical research employing a systematic review method investigated the effect of patient decision aids with or without a personal story on people's healthcare judgements and decisions. Searches were carried out between 2005-2012 of electronic databases (Medline, PsycINFO), and reference lists of identified articles, review articles, and key authors. A narrative analysis described and synthesised findings.ResultsOf 734 citations identified, 11 were included describing 13 studies. All studies found participants' judgments and/or decisions differed depending on whether or not their decision aid included a patient story. Knowledge was equally facilitated when the decision aids with and without stories had similar information content. Story-enhanced aids may help people recall information over time and/or their motivation to engage with health information. Personal stories affected both "system 1" (e.g., less counterfactual reasoning, more emotional reactions and perceptions) and "system 2" (e.g., more perceived deliberative decision making, more stable evaluations over time) decision-making strategies. Findings exploring associations with narrative communications, decision quality measures, and different levels of literacy and numeracy were mixed. The pattern of findings was similar for both experimental and real-world studies.ConclusionsThere is insufficient evidence that adding personal stories to decision aids increases their effectiveness to support people's informed decision making. More rigorous research is required to elicit evidence about the type of personal story that a) encourages people to make more reasoned decisions, b) discourages people from making choices based on another's values, and c) motivates people equally to engage with healthcare resources.
| S-EPMC4044102 | biostudies-literature
Project description:BackgroundAudit and feedback (A&F) often successfully enhances health professionals' intentions to improve quality of care but does not consistently lead to practice changes. Recipients often cite data credibility and limited resources as barriers impeding their ability to act upon A&F, suggesting the intention-to-action gap manifests while recipients are interacting with their data. While attention has been paid to the role feedback and contextual variables play in contributing to (or impeding) success, we lack a nuanced understanding of how healthcare professionals interact with and process clinical performance data.MethodsWe used qualitative, semi-structured interviews guided by Normalization Process Theory (NPT). Questions explored the role of data in quality improvement, experiences with the A&F report, perceptions of the data, and interpretations and reflections. Interviews were audio-recorded and transcribed verbatim. Data were analyzed using a combination of inductive and deductive strategies using reflexive thematic analysis informed by a constructivist paradigm.ResultsHealthcare professional characteristics (individual quality improvement capabilities and beliefs about data) seem to influence engagement with A&F to a greater degree than feedback variables (i.e., delivered by peers) and observed contextual factors (i.e., strong quality improvement culture). Most participants lacked the capabilities to interpret practice-level data in an actionable way despite a motivation to engage meaningfully. Reasons for the intention-to-action gap included challenges interpreting longitudinal data, appreciating the nuances of common data sources, understanding how aggregate data provides insights into individualized care, and identifying practice-level actions to improve quality. These factors limited effective cognitive participation and collective action, as outlined in NPT.ConclusionsA well-designed A&F intervention is necessary but not sufficient to inform practice changes. A&F initiatives must include co-interventions to address recipient characteristics (i.e., beliefs and capabilities) and context to optimize impact. Effective strategies to overcome the intention-to-action gap may include modelling how to use A&F to inform practice change, providing opportunities for social interaction relating to the A&F, and circulating examples of effective actions taken in response to A&F. More broadly, undergraduate medical education and post-graduate training must ensure physicians are equipped with QI capabilities, with an emphasis on the skills required to interpret and act on practice-level data.
| S-EPMC7891166 | biostudies-literature
Project description:BACKGROUND:The GRADE-CERQual (Confidence in Evidence from Reviews of Qualitative research) approach has been developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group. The approach has been developed to support the use of findings from qualitative evidence syntheses in decision making, including guideline development and policy formulation. CERQual includes four components for assessing how much confidence to place in findings from reviews of qualitative research (also referred to as qualitative evidence syntheses): (1) methodological limitations, (2) coherence, (3) adequacy of data and (4) relevance. This paper is part of a series providing guidance on how to apply CERQual and focuses on making an overall assessment of confidence in a review finding and creating a CERQual Evidence Profile and a CERQual Summary of Qualitative Findings table. METHODS:We developed this guidance by examining the methods used by other GRADE approaches, gathering feedback from relevant research communities and developing consensus through project group meetings. We then piloted the guidance on several qualitative evidence syntheses before agreeing on the approach. RESULTS:Confidence in the evidence is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. Creating a summary of each review finding and deciding whether or not CERQual should be used are important steps prior to assessing confidence. Confidence should be assessed for each review finding individually, based on the judgements made for each of the four CERQual components. Four levels are used to describe the overall assessment of confidence: high, moderate, low or very low. The overall CERQual assessment for each review finding should be explained in a CERQual Evidence Profile and Summary of Qualitative Findings table. CONCLUSIONS:Structuring and summarising review findings, assessing confidence in those findings using CERQual and creating a CERQual Evidence Profile and Summary of Qualitative Findings table should be essential components of undertaking qualitative evidence syntheses. This paper describes the end point of a CERQual assessment and should be read in conjunction with the other papers in the series that provide information on assessing individual CERQual components.
| S-EPMC5791047 | biostudies-literature
Project description:BackgroundIn Ontario, an online audit and feedback tool that provides primary care physicians with detailed information about patients who are overdue for cancer screening is underused. In the present study, we aimed to examine the effect of messages operationalizing 3 behaviour change techniques on access to the audit and feedback tool and on cancer screening rates.MethodsDuring May-September 2017, a pragmatic 2×2×2 factorial experiment tested 3 behaviour change techniques: anticipated regret, material incentive, and problem-solving. Outcomes were assessed using routinely collected administrative data. A qualitative process evaluation explored how and why the e-mail messages did or did not support Screening Activity Report access.ResultsOf 5449 primary care physicians randomly allocated to 1 of 8 e-mail messages, fewer than half opened the messages and fewer than 1 in 10 clicked through the messages. Messages with problem-solving content were associated with a 12.9% relative reduction in access to the tool (risk ratio: 0.871; 95% confidence interval: 0.791 to 0.958; p = 0.005), but a 0.3% increase in cervical cancer screening (rate ratio: 1.003; 95% confidence interval: 1.001 to 1.006; p = 0.003). If true, that association would represent 7568 more patients being screened. No other significant effects were observed.ConclusionsFor audit and feedback to work, recipients must engage with the data; for e-mail messages to prompt activity, recipients must open and review the message content. This large factorial experiment demonstrated that small changes in the content of such e-mail messages might influence clinical behaviour. Future research should focus on strategies to make cancer screening more user-centred.
| S-EPMC6588071 | biostudies-literature
Project description:BackgroundAudit and feedback (A&F) is a widely used implementation strategy. Understanding mechanisms of action of A&F increases the likelihood that the strategy will lead to implementation of an evidence-based practice. We therefore sought to understand one hospital's experience selecting and implementing an A&F intervention, to determine the implementation strategies that were used by staff and to specify the mechanism of action of those implementation strategies using causal pathway models, with the ultimate goal of improving acute stroke treatment practices.MethodsWe selected an A&F strategy in a hospital, initially based on implementation determinants and staff consideration of their performance on acute stroke treatment measures. After 7 months of A&F, we conducted semi-structured interviews of hospital providers and administrative staff to understand how it contributed to implementing guideline-concordant acute stroke treatment (medication named tissue plasminogen activator). We coded the interviews to identify the implementation strategies that staff used following A&F and to assess their mechanisms of action.ResultsWe identified five implementation strategies that staff used following the feedback intervention. These included (1) creating folders containing the acute stroke treatment protocol for the emergency department, (2) educating providers about the protocol for acute stroke, (3) obtaining computed tomography imaging of stroke patients immediately upon emergency department arrival, (4) increasing access to acute stroke medical treatment in the emergency department, and (5) providing additional staff support for implementation of the protocol in the emergency department. We identified enablement, training, and environmental restructuring as mechanisms of action through which the implementation strategies acted to improve guideline-concordant and timely acute stroke treatment.ConclusionsA&F of a hospital's acute stroke treatment practices generated additional implementation strategies that acted through various mechanisms of action. Future studies should focus on how initial implementation strategies can be amplified through internal mechanisms.
| S-EPMC8017642 | biostudies-literature