Dataset Information

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation.

ABSTRACT:

Background

Dashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation.

Objective

This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement.

Methods

QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard's perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement.

Results

The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice.

Conclusions

Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement.

International registered report identifier (irrid)

RR2-10.1136/bmjopen-2019-033208.

SUBMITTER: Alvarado N

PROVIDER: S-EPMC8663683 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:Standard methods for obtaining data may delay quality improvement (QI) interventions including for bronchiolitis, a common cause of childhood hospitalization. OBJECTIVE:To describe the use of a dashboard in the context of a multifaceted QI intervention aimed at reducing the use of chest radiographs, bronchodilators, antibiotics, steroids, and viral testing in patients with bronchiolitis. METHODS:This QI initiative took place at Children's Minnesota, a large, not-for-profit children's health care organization. A multidisciplinary bronchiolitis workgroup developed a local clinical guideline and order-set. Delays in obtaining baseline data prompted a pediatric hospitalist and information technology specialist to modify a vendor's dashboard to display data related to bronchiolitis guideline metrics. Patients 2 months to 2 years old with a bronchiolitis emergency department (ED)/inpatient encounter in the period October 1, 2014 to April 30, 2018 were included. The primary outcome was a functioning dashboard; a process measure was the percentage of ED clinician logins. Outcome measures included the percent use of guideline metrics (e.g., bronchodilators) displayed on statistical process control charts (ED vs. inpatient). Balancing measures included length of stay, charge ratios, and hospital revisits. RESULTS:A workgroup (formed October 2015) implemented a bronchiolitis order-set and guideline (February 2016) followed by a bronchiolitis dashboard (August 2016) consolidating disparate data sources loaded within 2 to 4 days of discharge. In total, 35% of ED clinicians logged in. Leaders used the dashboard to target and track interventions such as a bronchodilator order alert. There were improvements in most outcome metrics; however, timing did not suggest direct dashboard impact. ED balancing measures were lower after implementation. CONCLUSION:We described use of a dashboard to support a multifaceted QI initiative for bronchiolitis. Leaders used the dashboard for targeted interventions but the dashboard did not directly impact the observed improvements. Future studies should assess reasons for low individual dashboard use.

Project description:BACKGROUND:Clinical practice improvements based on quality-register data are influenced by multiple factors. Although there is agreement that information from quality registers is valuable for quality improvement, practical ways of organising register use have been notoriously difficult to realise. The present study sought to investigate the mechanisms that lead various clinicians to use quality registers for improvement. METHODS:This research involves studying individuals' decisions in response to a Swedish programme focusing on increasing the use of quality registers. Through a case study, we focused on heart failure care and its corresponding register: the Swedish Heart Failure Register. The empirical data consisted of a purposive sample collected longitudinally by qualitative methods between 2013 and 2015. In total, 18 semi-structured interviews were carried out. We used realist evaluation to identify contexts, mechanisms, and outcomes. RESULTS:We identified four contexts - registration, use of output data, governance, and improvement projects - that provide conditions for the initiation of specific mechanisms. Given a professional theoretical perspective, we further showed that mechanisms are based on the logics of either organisational improvement or clinical practice. The two logics offer insights into the ways in which clinicians choose to embrace or reject certain registers' initiatives. CONCLUSIONS:We identified a strong path dependence, as registers have historically been tightly linked to the medical profession's competence. Few new initiatives in the studied programme reach the clinical context. We explain this through the lack of an organisational improvement logic and its corresponding mechanisms in the context of the medical profession. Implementation programmes must understand the logic of clinical practice; that is, be integrated with the ways in which work is carried out in everyday practice. Programmes need to be better at helping core health professionals to reach the highest standards of patient care.

Project description:ImportanceCancer screening deficits during the first year of the COVID-19 pandemic were found to persist into 2021. Cancer-related deaths over the next decade are projected to increase if these deficits are not addressed.ObjectiveTo assess whether participation in a nationwide quality improvement (QI) collaborative, Return-to-Screening, was associated with restoration of cancer screening.Design, setting, and participantsAccredited cancer programs electively enrolled in this QI study. Project-specific targets were established on the basis of differences in mean monthly screening test volumes (MTVs) between representative prepandemic (September 2019 and January 2020) and pandemic (September 2020 and January 2021) periods to restore prepandemic volumes and achieve a minimum of 10% increase in MTV. Local QI teams implemented evidence-based screening interventions from June to November 2021 (intervention period), iteratively adjusting interventions according to their MTVs and target. Interrupted time series analyses was used to identify the intervention effect. Data analysis was performed from January to April 2022.ExposuresCollaborative QI support included provision of a Return-to-Screening plan-do-study-act protocol, evidence-based screening interventions, QI education, programmatic coordination, and calculation of screening deficits and targets.Main outcomes and measuresThe primary outcome was the proportion of QI projects reaching target MTV and counterfactual differences in the aggregate number of screening tests across time periods.ResultsOf 859 cancer screening QI projects (452 for breast cancer, 134 for colorectal cancer, 244 for lung cancer, and 29 for cervical cancer) conducted by 786 accredited cancer programs, 676 projects (79%) reached their target MTV. There were no hospital characteristics associated with increased likelihood of reaching target MTV except for disease site (lung vs breast, odds ratio, 2.8; 95% CI, 1.7 to 4.7). During the preintervention period (April to May 2021), there was a decrease in the mean MTV (slope, -13.1 tests per month; 95% CI, -23.1 to -3.2 tests per month). Interventions were associated with a significant immediate (slope, 101.0 tests per month; 95% CI, 49.1 to 153.0 tests per month) and sustained (slope, 36.3 tests per month; 95% CI, 5.3 to 67.3 tests per month) increase in MTVs relative to the preintervention trends. Additional screening tests were performed during the intervention period compared with the prepandemic period (170 748 tests), the pandemic period (210 450 tests), and the preintervention period (722 427 tests).Conclusions and relevanceIn this QI study, participation in a national Return-to-Screening collaborative with a multifaceted QI intervention was associated with improvements in cancer screening. Future collaborative QI endeavors leveraging accreditation infrastructure may help address other gaps in cancer care.

Project description:IntroductionThis protocol describes a study of a quality improvement collaborative (QIC) to support implementation and delivery of comprehensive geriatric assessment (CGA) in UK care homes. The QIC will be formed of health and social care professionals working in and with care homes and will be supported by clinical, quality improvement and research specialists. QIC participants will receive quality improvement training using the Model for Improvement. An appreciative approach to working with care homes will be encouraged through facilitated shared learning events, quality improvement coaching and assistance with project evaluation.Methods and analysisThe QIC will be delivered across a range of partnering organisations which plan, deliver and evaluate health services for care home residents in four local areas of one geographical region. A realist evaluation framework will be used to develop a programme theory informing how QICs are thought to work, for whom and in what ways when used to implement and deliver CGA in care homes. Data collection will involve participant observations of the QIC over 18 months, and interviews/focus groups with QIC participants to iteratively define, refine, test or refute the programme theory. Two researchers will analyse field notes, and interview/focus group transcripts, coding data using inductive and deductive analysis. The key findings and linked programme theory will be summarised as context-mechanism-outcome configurations describing what needs to be in place to use QICs to implement service improvements in care homes.Ethics and disseminationThe study protocol was reviewed by the National Health Service Health Research Authority (London Bromley research ethics committee reference: 205840) and the University of Nottingham (reference: LT07092016) ethics committees. Both determined that the Proactive HEAlthcare of Older People in Care Homes study was a service and quality improvement initiative. Findings will be shared nationally and internationally through conference presentations, publication in peer-reviewed journals, a graphical illustration and a dissemination video.

Project description:Objectives:The aim of the study was to explore what components of the General Medical Council's (GMC) Quality Assurance Framework work, for whom, in what circumstances and how?Setting:UK undergraduate and postgraduate medical education and training.Participants:We conducted interviews with a stratified sample of 36 individuals. This included those who had direct experiences, as well as those with external insights, representing local, national and international organisations within and outside medicine.Intervention:The GMC quality assure education to protect patient and public safety utilising complex intervention components including meeting standards, institutional visits and monitoring performance. However, the context in which these are implemented matters. We undertook an innovative realist evaluation to test an initial programme theory. Data were analysed using framework analysis.Results:Across components of the intervention, we identified key mechanisms, including transparent reporting to promote quality improvement; dialogical feedback; partnership working facilitating interactions between regulators and providers, and role clarity in conducting proportionate interventions appropriate to risk. The GMC's framework was commended for being comprehensive and enabling a broad understanding of an organisation's performance. Unintended consequences included confusion over roles and boundaries in different contexts which often undermined effectiveness.Conclusions:This realist evaluation substantiates the literature and reveals deeper understandings about quality assuring medical education. While standardised approaches are implemented, interventions need to be contextually proportionate. Routine communication is beneficial to verify data, share concerns and check risk; however, ongoing partnership working can foster assurance. The study provides a modified programme theory to explicate how education providers and regulators can work more effectively together to uphold education quality, and ultimately protect public safety. The findings have influenced the GMC's approach to quality assurance which impacts on all medical students and doctors in training.

Project description:BackgroundPeople with type 1 diabetes and raised glucose levels are at greater risk of retinopathy, nephropathy, neuropathy, cardiovascular disease, sexual health problems and foot disease. The UK National Institute for Health and Care Excellence (NICE) recommends continuous subcutaneous 'insulin pump' therapy for people with type 1 diabetes whose HbA1c is above 69 mmol/mol. Insulin pump use can improve quality of life, cut cardiovascular risk and increase treatment satisfaction. About 90,000 people in England and Wales meet NICE criteria for insulin pumps but do not use one. Insulin pump use also varies markedly by deprivation, ethnicity, sex and location. Increasing insulin pump use is a key improvement priority. Audit and feedback is a common but variably effective intervention. Limited capabilities of healthcare providers to mount effective responses to feedback from national audits, such as the National Diabetes Audit (NDA), undermines efforts to improve care. We have co-developed a theoretically and empirically informed quality improvement collaborative (QIC) to strengthen local responses to feedback with patients and carers, national audits and healthcare providers. We will evaluate whether the QIC improves the uptake of insulin pumps following NDA feedback.MethodsWe will undertake an efficient cluster randomised trial using routine data. The QIC will be delivered alongside the NDA to specialist diabetes teams in England and Wales. Our primary outcome will be the proportion of people with type 1 diabetes and an HbA1c above 69 mmol/mol who start and continue insulin pump use during the 18-month intervention period. Secondary outcomes will assess change in glucose control and duration of pump use. Subgroup analyses will explore impacts upon inequalities by ethnicity, sex, age and deprivation. A theory-informed process evaluation will explore diabetes specialist teams' engagement, implementation, fidelity and tailoring through observations, interviews, surveys and documentary analysis. An economic evaluation will micro-cost the QIC, estimate cost-effectiveness of NDA feedback with QIC and estimate the budget impact of NHS-wide QIC roll out.DiscussionOur study responds to a need for more head-to-head trials of different ways of reinforcing feedback delivery. Our findings will have implications for other large-scale audit and feedback programmes.Trial registrationISRCTN82176651 Registered 18 October 2022.