Dataset Information

A Scalable Service to Improve Health Care Quality Through Precision Audit and Feedback: Proposal for a Randomized Controlled Trial.

ABSTRACT:

Background

Health care delivery organizations lack evidence-based strategies for using quality measurement data to improve performance. Audit and feedback (A&F), the delivery of clinical performance summaries to providers, demonstrates the potential for large effects on clinical practice but is currently implemented as a blunt one size fits most intervention. Each provider in a care setting typically receives a performance summary of identical metrics in a common format despite the growing recognition that precisionizing interventions hold significant promise in improving their impact. A precision approach to A&F prioritizes the display of information in a single metric that, for each recipient, carries the highest value for performance improvement, such as when the metric's level drops below a peer benchmark or minimum standard for the first time, thereby revealing an actionable performance gap. Furthermore, precision A&F uses an optimal message format (including framing and visual displays) based on what is known about the recipient and the intended gist meaning being communicated to improve message interpretation while reducing the cognitive processing burden. Well-established psychological principles, frameworks, and theories form a feedback intervention knowledge base to achieve precision A&F. From an informatics perspective, precision A&F requires a knowledge-based system that enables mass customization by representing knowledge configurable at the group and individual levels.

Objective

This study aims to implement and evaluate a demonstration system for precision A&F in anesthesia care and to assess the effect of precision feedback emails on care quality and outcomes in a national quality improvement consortium.

Methods

We propose to achieve our aims by conducting 3 studies: a requirements analysis and preferences elicitation study using human-centered design and conjoint analysis methods, a software service development and implementation study, and a cluster randomized controlled trial of a precision A&F service with a concurrent process evaluation. This study will be conducted with the Multicenter Perioperative Outcomes Group, a national anesthesia quality improvement consortium with >60 member hospitals in >20 US states. This study will extend the Multicenter Perioperative Outcomes Group quality improvement infrastructure by using existing data and performance measurement processes.

Results

The proposal was funded in September 2021 with a 4-year timeline. Data collection for Aim 1 began in March 2022. We plan for a 24-month trial timeline, with the intervention period of the trial beginning in March 2024.

Conclusions

The proposed aims will collectively demonstrate a precision feedback service developed using an open-source technical infrastructure for computable knowledge management. By implementing and evaluating a demonstration system for precision feedback, we create the potential to observe the conditions under which feedback interventions are effective.

International registered report identifier (irrid)

PRR1-10.2196/34990.

SUBMITTER: Landis-Lewis Z

PROVIDER: S-EPMC9131150 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe Department of Veterans Affairs (VA) has led the industry in measuring facility performance as a critical element in improving quality of care, investing substantial resources to develop and maintain valid and cost-effective measures. The External Peer Review Program (EPRP) of the VA is the official data source for monitoring facility performance, used to prioritize the quality areas needing most attention. Facility performance measurement has significantly improved preventive and chronic care, as well as overall quality; however, much variability still exists in levels of performance across measures and facilities. Audit and feedback (A&F), an important component of effective performance measurement, can help reduce this variability and improve overall performance. Previous research suggests that VA Medical Centers (VAMCs) with high EPRP performance scores tend to use EPRP data as a feedback source. However, the manner in which EPRP data are used as a feedback source by individual providers as well as service line, facility, and network leadership is not well understood. An in-depth understanding of mental models, strategies, and specific feedback process characteristics adopted by high-performing facilities is thus urgently needed.This research compares how leaders of high, low, and moderately performing VAMCs use clinical performance data from the EPRP as a feedback tool to maintain and improve quality of care.MethodsWe will conduct a qualitative, grounded theory analysis of up to 64 interviews using a novel method of sampling primary care, facility, and Veterans Integrated Service Network (VISN) leadership at high-, moderate-, and low-performing facilities. We will analyze interviews for evidence of cross-facility differences in perceptions of performance data usefulness and strategies for disseminating performance data evaluating performance, with particular attention to timeliness, individualization, and punitiveness of feedback delivery.DiscussionMost research examining feedback to improve provider and facility performance lacks a detailed understanding of the elements of effective feedback. This research will highlight the elements most commonly used at high-performing facilities and identify additional features of their successful feedback strategies not previously identified. Armed with this information, practices can implement more effective A&F interventions to improve quality of care.

Project description:BackgroundTuberculosis screening of people living with HIV (PLHIV) can contribute to early tuberculosis diagnosis and improved patient outcomes. Evidence-based guidelines for tuberculosis screening are available, but literature assessing their implementation and the quality of clinical practice is scarce.ObjectivesTo assess tuberculosis screening practices and the effectiveness of audit and performance feedback to improve quality of tuberculosis screening at HIV care clinics in Ghana.DesignHealthcare providers at 10 large HIV care clinics prospectively registered patient consultations during May and October 2014, before and after a performance feedback intervention in August 2014. The outcomes of interest were overall tuberculosis suspicion rate during consultations and provider adherence to the International Standards for Tuberculosis Care and the World Health Organizations' guidelines for symptom-based tuberculosis screening among PLHIV.ResultsTwenty-one healthcare providers registered a total of 2,666 consultations; 1,368 consultations before and 1,298 consultations after the feedback intervention. Tuberculosis suspicion rate during consultation increased from 12.6 to 20.9% after feedback (odds ratio, OR 1.83; 95% confidence interval, CI: 1.09-3.09). Before feedback, sputum smear microscopy was requested for 58.7% of patients with suspected tuberculosis, for 47.2% of patients with cough ?2 weeks, and for 27.5% of patients with a positive World Health Organization (WHO) symptom screen (any of current cough, fever, weight loss or night sweats). After feedback, patients with a positive WHO symptom screen were more likely to be suspected of tuberculosis (OR 2.21; 95% CI: 1.19-4.09) and referred for microscopy (OR 2.71; 95% CI: 1.25-5.86).ConclusionsA simple prospective audit tool identified flaws in clinical practices for tuberculosis screening of PLHIV. There was no systematic identification of people with suspected active tuberculosis. We found low initial tuberculosis suspicion rate compounded by low referral rates of relevant patients for sputum smear microscopy. Adherence to recommended standards and guidelines for tuberculosis screening improved after performance feedback.

Project description:BackgroundLaboratory tests and transfusions are sometimes ordered inappropriately, particularly in the critical care setting, which sees frequent use of both. Audit and Feedback (A&F) is a potentially useful intervention for modifying healthcare provider behaviors, but its application to the complex, team-based environment of critical care is not well understood. We conducted a systematic review of the literature on A&F interventions for improving test or transfusion ordering in the critical care setting.MethodsFive databases, two registries, and the bibliographies of relevant articles were searched. We included critical care studies that assessed the use of A&F targeting healthcare provider behaviors, alone or in combination with other interventions to improve test and transfusion ordering, as compared to historical practice, no intervention, or another healthcare behaviour change intervention. Studies were included only if they reported laboratory test or transfusion orders, or the appropriateness of orders, as outcomes. There were no restrictions based on study design, date of publication, or follow-up time. Intervention characteristics and absolute differences in outcomes were summarized. The quality of individual studies was assessed using a modified version of the Effective Practice and Organisation of Care Cochrane Review Group's criteria.ResultsWe identified 16 studies, including 13 uncontrolled before-after studies, one randomized controlled trial, one controlled before-after study, and one controlled clinical trial (quasi-experimental). These studies described 17 interventions, mostly (88%) multifaceted interventions with an A&F component. Feedback was most often provided in a written format only (41%), more than once (53%), and most often only provided data aggregated to the group-level (41%). Most studies saw a change in the hypothesized direction, but not all studies provided statistical analyses to formally test improvement. Overall study quality was low, with studies often lacking a concurrent control group.ConclusionsOur review summarizes characteristics of A&F interventions implemented in the critical care context, points to some mechanisms by which A&F might be made more effective in this setting, and provides an overview of how the appropriateness of orders was reported. Our findings suggest that A&F can be effective in the context of critical care; however, further research is required to characterize approaches that optimize the effectiveness in this setting alongside more rigorous evaluation methods.Trial registrationPROSPERO CRD42016051941.

Project description:BackgroundFeedback tools for clinical audit data that compare site-specific results to average performance over all sites can be useful for quality improvement. Proposed tools should be simple and clearly benchmark the site's performance, so that a relevant action plan can be directly implemented to improve patient care services. We aimed to develop such a tool in order to feedback data to UK HIV clinics participating in the 2015 British HIV Association (BHIVA) audit assessing compliance with the 2011 guidelines for routine investigation and monitoring of adult HIV-1- infected individuals.MethodsHIV clinic sites were asked to provide data on a random sample of 50-100 adult patients attending for HIV care during 2014 and/or 2015 by completing a self-audit spreadsheet. Outcomes audited included the proportion of patients with recorded resistance testing, viral load monitoring, adherence assessment, medications, hepatitis testing, vaccination management, risk assessments, and sexual health screening. For each outcome we benchmarked the proportion for a specific site against the average performance. We produced performance charts for each site using boxplots for the outcomes. We also used the mean and differences from the mean performance to produce a dashboard for each site. We used principal components analysis to group correlated outcomes and simplify the dashboard.ResultsThe 106 sites included in the study provided information on a total of 7768 patients. Outcomes capturing monitoring of treatment of HIV-infection showed high performance across the sites, whereas testing for hepatitis, and risk assessment for cardiovascular disease and smoking, management of flu vaccination, sexual health screening, and cervical cytology for women were very variable across sites. The principal components analysis reduced the original 12 outcomes to four factors that represented HIV care, hepatitis testing, other screening tests, and resistance testing. These provided simplified measures of adherence to guidelines which were presented as a 4 bar dashboard of performance.ConclusionOur dashboard performance charts provide easily digestible visual summaries of locally relevant audit data that are benchmarked against the overall mean and can be used to improve feedback to HIV services. Feedback from clinicians indicated that they found these charts acceptable and useful.

Project description:BackgroundMany primary care sites have implemented models to improve detection, diagnosis, and management of dementia, as per Canadian guidelines. The aim of this study is to describe the responses of clinicians, managers, and staff of sites that have implemented these models when presented with audit results, their insights on the factors that explain their results, their proposed solutions for improvement and how these align to one another.MethodsOne audit and feedback cycle was carried out in eight purposefully sampled sites in Ontario, Canada, that had previously implemented dementia care models. Audit consisted of a) chart review to assess quality of dementia care indicators, b) questionnaire to assess the physicians' knowledge, attitudes and practice toward dementia care, and c) semi-structured interviews to understand barriers and facilitators to implementing these models. Feedback was given to clinicians, managers, and staff in the form of graphic and oral presentations, followed by eight focus groups (one per site). Discussions revolved around: what audit results elicited more discussion from the participants, 2) their insights on the factors that explain their audit results, and 3) solutions they propose to improve dementia care. Deductive content and inductive thematic analyses, grounded in causal pathways models' theory was performed.FindingsThe audit and feedback process allowed the 63 participants to discuss many audit results and share their insights on a) organizational factors (lack of human resources, the importance of organized links with community services, clear roles and support from external memory clinics) and b) clinician factors (perceived competency practice and attitudes on dementia care), that could explain their audit results. Participants also provided solutions to improve dementia care in primary care (financial incentives, having clear pathways, adding tools to improve chart documentation, establish training on dementia care, and the possibility of benchmarking with other institutions). Proposed solutions were well aligned with their insights and further nuanced according to contextual details.ConclusionsThis study provides valuable information on solutions proposed by primary care clinicians, managers, and staff to improve dementia care in primary care. The solutions are grounded in clinical experience and will inform ongoing and future dementia strategies.

Project description:BackgroundThere is a significant variation among individual primary care providers in prescribing of potentially problematic, low-value medicines which cause avoidable patient harm. Audit and feedback is generally effective at improving prescribing. However, progress has been hindered by research waste, leading to unanswered questions about how to include audit and feedback for specific problems and circumstances. Trials of different ways of providing audit and feedback in implementation laboratories have been proposed as a way of improving population healthcare while generating robust evidence on feedback effects. However, there is limited experience in their design and delivery.AimTo explore priorities, feasibility, and ethical challenges of establishing a primary care prescribing audit and feedback implementation laboratory.Design and settingTwo-stage Delphi consensus process involving primary care pharmacy leads, audit and feedback researchers, and patient and public.MethodParticipants initially scored statements relating to priorities, feasibility, and ethical considerations for an implementation laboratory. These covered current feedback practice, priority topics for feedback, usefulness of feedback in improving prescribing and different types of prescribing data, acceptability and desirability of different organization levels of randomization, options for trial consent, different methods of delivering feedback, and interest in finding out how effective different ways of presenting feedback would be. After receiving collated results, participants then scored the items again. The consensus was defined using the GRADE criteria. The results were analyzed by group and overall score.ResultsFourteen participants reached consensus for 38 out of 55 statements. Addressing antibiotic and opioid prescribing emerged as the highest priorities for action. The panel supported statements around addressing high-priority prescribing issues, taking an "opt-out" approach to practice consent if waiving consent was not permitted, and randomizing at lower rather than higher organizational levels. Participants supported patient-level prescribing data and further research evaluating most of the different feedback methods we presented them with.ConclusionsThere is a good level of support for evaluating a wide range of potential enhancements to improve the effects of feedback on prescribing. The successful design and delivery of a primary care audit and feedback implementation laboratory depend on identifying shared priorities and addressing practical and ethical considerations.