Dataset Information

Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system.

ABSTRACT:

Background

Audit and Feedback (A&F) is a widely used quality improvement technique that measures clinicians' clinical performance and reports it back to them. Computerised A&F (e-A&F) system interfaces may consist of four key components: (1) Summaries of clinical performance; (2) Patient lists; (3) Patient-level data; (4) Recommended actions. There is a lack of evidence regarding how to best design e-A&F interfaces; establishing such evidence is key to maximising usability, and in turn improving patient safety.

Aim

To evaluate the usability of a novel theoretically-informed and research-led e-A&F system for primary care (the Performance Improvement plaN GeneratoR: PINGR).

Objectives

(1) Describe PINGR's design, rationale and theoretical basis; (2) Identify usability issues with PINGR; (3) Understand how these issues may interfere with the cognitive goals of end-users; (4) Translate the issues into recommendations for the user-centred design of e-A&F systems.

Methods

Eight experienced health system evaluators performed a usability inspection using an innovative hybrid approach consisting of five stages: (1) Development of representative user tasks, Goals, and Actions; (2) Combining Heuristic Evaluation and Cognitive Walkthrough methods into a single protocol to identify usability issues; (3) Consolidation of issues; (4) Severity rating of consolidated issues; (5) Analysis of issues according to usability heuristics, interface components, and Goal-Action structure.

Results

A final list of 47 issues were categorised into 8 heuristic themes. The most error-prone heuristics were 'Consistency and standards' (13 usability issues; 28% of the total) and 'Match between system and real world' (n=10, 21%). The recommended actions component of the PINGR interface had the most usability issues (n=21, 45%), followed by patient-level data (n=5, 11%), patient lists (n=4, 9%), and summaries of clinical performance (n=4, 9%). The most error-prone Actions across all user Goals were: (1) Patient selection from a list; (2) Data identification from a figure (both population-level and patient-level); (3) Disagreement with a system recommendation.

Conclusions

By contextualising our findings within the wider literature on health information system usability, we provide recommendations for the design of e-A&F system interfaces relating to their four key components, in addition to how they may be integrated within a system.

SUBMITTER: Brown B

PROVIDER: S-EPMC5015594 | biostudies-literature | 2016 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system.

Brown Benjamin B Balatsoukas Panos P Williams Richard R Sperrin Matthew M Buchan Iain I

International journal of medical informatics 20160716

<h4>Background</h4>Audit and Feedback (A&F) is a widely used quality improvement technique that measures clinicians' clinical performance and reports it back to them. Computerised A&F (e-A&F) system interfaces may consist of four key components: (1) Summaries of clinical performance; (2) Patient lists; (3) Patient-level data; (4) Recommended actions. There is a lack of evidence regarding how to best design e-A&F interfaces; establishing such evidence is key to maximising usability, and in turn i ...[more]

PMID: 27573327

Similar Datasets

Project description:BackgroundAudit and feedback (A&F) interventions are one of the most common approaches for implementing evidence-based practices. A key barrier to more effective A&F interventions is the lack of a theory-guided approach to the accumulation of evidence. Recent interviews with theory experts identified 313 theory-informed hypotheses, spread across 30 themes, about how to create more effective A&F interventions. In the current survey, we sought to elicit from stakeholders which hypotheses were most likely to advance the field if studied further.MethodsFrom the list of 313, three members of the research team identified 216 that were clear and distinguishable enough for prioritization. A web-based survey was then sent to 211 A&F intervention stakeholders asking them to choose up to 50 'priority' hypotheses following the header "A&F interventions will be more effective if…". Analyses included frequencies of endorsement of the individual hypotheses and themes into which they were grouped.Results68 of the 211 invited participants responded to the survey. Seven hypotheses were chosen by > 50% of respondents, including A&F interventions will be more effective… "if feedback is provided by a trusted source"; "if recipients are involved in the design/development of the feedback intervention"; "if recommendations related to the feedback are based on good quality evidence"; "if the behaviour is under the control of the recipient"; "if it addresses barriers and facilitators (drivers) to behaviour change"; "if it suggests clear action plans"; and "if target/goal/optimal rates are clear and explicit". The most endorsed theme was Recipient Priorities (four hypotheses were chosen 92 times as a 'priority' hypotheses).ConclusionsThis work determined a set of hypotheses thought by respondents to be to be most likely to advance the field through future A&F intervention research. This work can inform a coordinated research agenda that may more efficiently lead to more effective A&F interventions.

Project description:INTRODUCTION:Inappropriate use of antimicrobials in hospitals contributes to antimicrobial resistance. Antimicrobial stewardship (AMS) interventions aim to improve antimicrobial prescribing, but they are often resource and personnel intensive. Computerised decision supportsystems (CDSSs) seem a promising tool to improve antimicrobial prescribing but have been insufficiently studied in clinical trials. METHODS AND ANALYSIS:The COMPuterized Antibiotic Stewardship Study trial, is a publicly funded, open-label, cluster randomised, controlled superiority trial which aims to determine whether a multimodal CDSS intervention integrated in the electronic health record (EHR) reduces overall antibiotic exposure in adult patients hospitalised in wards of two secondary and one tertiary care centre in Switzerland compared with 'standard-of-care' AMS. Twenty-four hospital wards will be randomised 1:1 to either intervention or control, using a 'pair-matching' approach based on baseline antibiotic use, specialty and centre. The intervention will consist of (1) decision support for the choice of antimicrobial treatment and duration of treatment for selected indications (based on indication entry), (2) accountable justification for deviation from the local guidelines (with regard to the choice of molecules and duration), (3) alerts for self-guided re-evaluation of treatment on calendar day 4 of antimicrobial therapy and (4) monthly ward-level feedback of antimicrobial prescribing indicators. The primary outcome will be the difference in overall systemic antibiotic use measured in days of therapy per admission based on administration data recorded in the EHR over the whole intervention period (12 months), taking into account clustering. Secondary outcomes include qualitative and quantitative antimicrobial use indicators, economic outcomes and clinical, microbiological and patient safety indicators. ETHICS AND DISSEMINATION:Ethics approval was obtained for all participating sites (Comission Cantonale d'Éthique de la Recherche (CCER)2017-00454). The results of the trial will be submitted for publication in a peer-reviewed journal. Further dissemination activities will be presentations/posters at national and international conferences. TRIAL REGISTRATION NUMBER:NCT03120975; Pre-results.

Project description:BackgroundDespite growing evidence that deprescribing can improve clinical outcomes, quality of life and reduce the likelihood of adverse drug events, the practice is not widespread, particularly in hospital settings. Clinical risk assessment tools, like the Drug Burden Index (DBI), can help prioritise patients for medication review and prioritise medications to deprescribe, but are not integrated within routine care. The aim of this study was to conduct formative usability testing of a computerised decision support (CDS) tool, based on DBI, to identify modifications required to the tool prior to trialling in practice.MethodsOur CDS tool comprised a DBI MPage in the electronic medical record (clinical workspace) that facilitated review of a patient's DBI and medication list, access to deprescribing resources, and the ability to deprescribe. Two rounds of scenario-based formative usability testing with think-aloud protocol were used. Seventeen end-users participated in the testing, including junior and senior doctors, and pharmacists.ResultsParticipants expressed positive views about the DBI CDS tool but testing revealed a number of clear areas for improvement. These primarily related to terminology used (i.e. what is a DBI and how is it calculated?), and consistency of functionality and display. A key finding was that users wanted the CDS tool to look and function in a similar way to other decision support tools in the electronic medical record. Modifications were made to the CDS tool in response to user feedback.ConclusionUsability testing proved extremely useful for identifying components of our CDS tool that were confusing, difficult to locate or to understand. We recommend usability testing be adopted prior to implementation of any digital health intervention. We hope our revised CDS tool equips clinicians with the knowledge and confidence to consider discontinuation of inappropriate medications in routine care of hospitalised patients. In the next phase of our project, we plan to pilot test the tool in practice to evaluate its uptake and effectiveness in supporting deprescribing in routine hospital care.

Project description:BackgroundWe evaluated the impact of the pharmacist-led Safety Medication dASHboard (SMASH) intervention on medication safety in primary care.Methods and findingsSMASH comprised (1) training of clinical pharmacists to deliver the intervention; (2) a web-based dashboard providing actionable, patient-level feedback; and (3) pharmacists reviewing individual at-risk patients, and initiating remedial actions or advising general practitioners on doing so. It was implemented in 43 general practices covering a population of 235,595 people in Salford (Greater Manchester), UK. All practices started receiving the intervention between 18 April 2016 and 26 September 2017. We used an interrupted time series analysis of rates (prevalence) of potentially hazardous prescribing and inadequate blood-test monitoring, comparing observed rates post-intervention to extrapolations from a 24-month pre-intervention trend. The number of people registered to participating practices and having 1 or more risk factors for being exposed to hazardous prescribing or inadequate blood-test monitoring at the start of the intervention was 47,413 (males: 23,073 [48.7%]; mean age: 60 years [standard deviation: 21]). At baseline, 95% of practices had rates of potentially hazardous prescribing (composite of 10 indicators) between 0.88% and 6.19%. The prevalence of potentially hazardous prescribing reduced by 27.9% (95% CI 20.3% to 36.8%, p < 0.001) at 24 weeks and by 40.7% (95% CI 29.1% to 54.2%, p < 0.001) at 12 months after introduction of SMASH. The rate of inadequate blood-test monitoring (composite of 2 indicators) reduced by 22.0% (95% CI 0.2% to 50.7%, p = 0.046) at 24 weeks; the change at 12 months (23.5%) was no longer significant (95% CI -4.5% to 61.6%, p = 0.127). After 12 months, 95% of practices had rates of potentially hazardous prescribing between 0.74% and 3.02%. Study limitations include the fact that practices were not randomised, and therefore unmeasured confounding may have influenced our findings.ConclusionsThe SMASH intervention was associated with reduced rates of potentially hazardous prescribing and inadequate blood-test monitoring in general practices. This reduction was sustained over 12 months after the start of the intervention for prescribing but not for monitoring of medication. There was a marked reduction in the variation in rates of hazardous prescribing between practices.

Dataset Information

Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system.

Background

Aim

Objectives

Methods

Results

Conclusions

Publications

Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets