Dataset Information

Formative Usability Testing Reduces Severe Blood Product Ordering Errors.

ABSTRACT: BACKGROUND:Medical errors in blood product orders and administration are common, especially for pediatric patients. A failure modes and effects analysis in our health care system indicated high risk from the electronic blood ordering process. OBJECTIVES:There are two objectives of this study as follows:(1) To describe differences in the design of the original blood product orders and order sets in the system (original design), new orders and order sets designed by expert committee (DEC), and a third-version developed through user-centered design (UCD).(2) To compare the number and type of ordering errors, task completion rates, time on task, and user preferences between the original design and that developed via UCD. METHODS:A multidisciplinary expert committee proposed adjustments to existing blood product order sets resulting in the DEC order set. When that order set was tested with front-line users, persistent failure modes were detected, so orders and order sets were redesigned again via formative usability testing. Front-line users in their native clinical workspaces were observed ordering blood in realistic simulated scenarios using a think-aloud protocol. Iterative adjustments were made between participants. In summative testing, participants were randomized to use the original design or UCD for five simulated scenarios. We evaluated differences in ordering errors, time on task, and users' design preference with two-sample t-tests. RESULTS:Formative usability testing with 27 providers from seven specialties led to 18 changes made to the DEC to produce the UCD. In summative testing, error-free task completion for the original design was 36%, which increased to 66% in UCD (30%, 95% confidence interval [CI]: 3.9-57%; p?=?0.03). Time on task did not vary significantly. CONCLUSION:UCD led to substantially different blood product orders and order sets than DEC. Users made fewer errors when ordering blood products for pediatric patients in simulated scenarios when using the UCD orders and order sets compared with the original design.

SUBMITTER: Orenstein EW

PROVIDER: S-EPMC6930842 | biostudies-literature | 2019 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Formative Usability Testing Reduces Severe Blood Product Ordering Errors.

Orenstein Evan W EW Boudreaux Jeanne J Rollins Margo M Jones Jennifer J Bryant Christy C Karavite Dean D Muthu Naveen N Hike Jessica J Williams Herb H Kilgore Tania T Carter Alexis B AB Josephson Cassandra D CD

Applied clinical informatics 20191001 5

<h4>Background</h4>Medical errors in blood product orders and administration are common, especially for pediatric patients. A failure modes and effects analysis in our health care system indicated high risk from the electronic blood ordering process.<h4>Objectives</h4>There are two objectives of this study as follows:(1) To describe differences in the design of the original blood product orders and order sets in the system (original design), new orders and order sets designed by expert committee ...[more]

PMID: 31875648

Similar Datasets

Project description:BackgroundChildren are at increased risk from transfusion-related medical errors. Clinical decision support (CDS) can enhance pediatric providers' decision-making regarding transfusion practices including indications, volume, rate, and special processing instructions. Our objective was to use CDS in a pediatric health system to reduce:blood product-related safety events from ordering errors;special processing ordering errors for patients with T-cell dysfunction, sickle cell disease (SCD), or thalassemia;transfusions administered faster than 5 mL/kg/h.Materials and methodsIn this single-center before and after quality improvement study, we evaluated how user-centered design of pediatric blood product orders influenced pediatric transfusion practices and outcomes. Safety events were identified through active and passive surveillance. Other clinically relevant outcomes were identified through electronic health record queries.ResultsBlood product-related safety events from ordering errors did not change significantly from the baseline period (6 events, 0.4 per month, from 1/1/2018-3/27/2019) to the intervention period (1 event, 0.1 per month, from 3/28/2019-12/31/2019; rate ratio: 0.27 [0.01-2.25]). Packed red blood cell (PRBC) and platelet orders for patients with T-cell dysfunction that did not specify irradiation decreased significantly from 488/12,359 (3.9%) to 204/6,711 (3.0%, risk ratio: 0.77 [0.66-0.90]). PRBC orders for patients with SCD or thalassemia that did not specify phenotypically similar units fell from 386/2,876 (13.4%) to 57/1,755 (3.2%, risk ratio: 0.24 [0.18-0.32]). Transfusions administered faster than 5 mL/kg/h decreased from 4,112/14,641 (28.1%) to 2,125/9,263 (22.9%, risk ratio: 0.82 [0.78-0.85]).DiscussionUser-centered design of CDS for pediatric blood product orders significantly reduced special processing ordering errors and inappropriate transfusion rates. Larger studies are needed to evaluate the impact on safety events.

Project description:BackgroundDespite growing evidence that deprescribing can improve clinical outcomes, quality of life and reduce the likelihood of adverse drug events, the practice is not widespread, particularly in hospital settings. Clinical risk assessment tools, like the Drug Burden Index (DBI), can help prioritise patients for medication review and prioritise medications to deprescribe, but are not integrated within routine care. The aim of this study was to conduct formative usability testing of a computerised decision support (CDS) tool, based on DBI, to identify modifications required to the tool prior to trialling in practice.MethodsOur CDS tool comprised a DBI MPage in the electronic medical record (clinical workspace) that facilitated review of a patient's DBI and medication list, access to deprescribing resources, and the ability to deprescribe. Two rounds of scenario-based formative usability testing with think-aloud protocol were used. Seventeen end-users participated in the testing, including junior and senior doctors, and pharmacists.ResultsParticipants expressed positive views about the DBI CDS tool but testing revealed a number of clear areas for improvement. These primarily related to terminology used (i.e. what is a DBI and how is it calculated?), and consistency of functionality and display. A key finding was that users wanted the CDS tool to look and function in a similar way to other decision support tools in the electronic medical record. Modifications were made to the CDS tool in response to user feedback.ConclusionUsability testing proved extremely useful for identifying components of our CDS tool that were confusing, difficult to locate or to understand. We recommend usability testing be adopted prior to implementation of any digital health intervention. We hope our revised CDS tool equips clinicians with the knowledge and confidence to consider discontinuation of inappropriate medications in routine care of hospitalised patients. In the next phase of our project, we plan to pilot test the tool in practice to evaluate its uptake and effectiveness in supporting deprescribing in routine hospital care.

Project description:BACKGROUND:Supporting patient engagement and empowerment is increasingly seen as essential in providing person-centered health care to people with chronic illness. Mobile apps helping patients reflect on their concerns as preparation for consultations with their health care providers can have beneficial effects on the consultation quality. However, apps focusing on empowerment and personal strengths are still scarce. OBJECTIVE:This study aimed to (1) develop a mobile app to support patients with rheumatic diseases in reflecting on their strengths in preparation for consultations with health care providers and (2) explore patients' perceived usability of the app in a nonclinical test setting. METHODS:A prototype app was developed based on input from patients and health care providers, as reported in previous studies. The app was designed for use in self-management support settings aiming to promote awareness of strengths and to focus attention on strengths in the patient-health care provider dialogue. The features included in the prototype were as follows: (1) introduction to the topic of strengths, (2) list of examples of strengths to promote reflection and registration of own strengths, (3) summary of registered strengths, (4) value-based goal setting, (5) linking of strengths to goals, (6) summary of all registrations, and (7) options to share summary digitally or as a print version. In this study, the app was refined through a formative evaluation with patients and health care providers recruited from a specialized rheumatology hospital unit. Patients' perceptions of the app's usability were explored in a test setting with self-report measurements and semistructured interviews. The interviews were audiotaped, transcribed, and analyzed with directed content analysis. Data from questionnaires were analyzed with descriptive statistics. RESULTS:Developmental and formative evaluation included 18 patients and 7 health care providers. The evaluation resulted in minor adjustments to the prototype but no major changes in features. The usability testing included 12 patients. All participants found the usability acceptable; the median score on the System Usability Scale was 86.3 (range 70-100). All reported that it was meaningful and relevant to use the app. Out of 12 participants, 9 (75%) reported becoming more aware of their own strengths by using the app; 1 (8%) disagreed and 2 (17%) provided a neutral response. The results on the goal-related feature were mixed, with half of the patients finding it useful to link strengths to concrete goals. A statistically significant positive change from pre- to postintervention was identified on measures of self-efficacy and negative emotions. CONCLUSIONS:In this formative evaluation of a mobile app to promote patients' reflections on their strengths, patients perceived the app as meaningful and supporting awareness. The results suggest the usefulness of building in functionality to support use of strengths and goal attainment. Further studies on efficacy and usability in a clinical setting, including health care providers, are needed.

Project description:BackgroundReporting of medication errors is one of the essential mechanisms to identify risky health care systems and practices that lead to medication errors. Unreported medication errors are a real issue; one of the identified causes is a burdensome medication error reporting system. An anonymous and user-friendly mobile app for reporting medication errors could be an alternative method of reporting medication error in busy health care settings.ObjectiveThe objective of this paper is to report usability testing of the Medication Error Reporting App (MERA), a mobile app for reporting medication errors anonymously.MethodsQuantitative and qualitative methods were employed involving 45 different testers (pharmacists, doctors, and nurses) from a large tertiary hospital in Malaysia. Quantitative data was retrieved using task performance and rating of MERA and qualitative data were retrieved through focus group discussions. Three sessions, with 15 testers each session, were conducted from January to March 2018.ResultsThe majority of testers were pharmacists (23/45, 51%), female (35/45, 78%), and the mean age was 36 (SD 9) years. A total of 135 complete reports were successfully submitted by the testers (three reports per tester) and 79.2% (107/135) of the reports were correct. There was significant improvement in mean System Usability Scale scores in each session of the development process (P<.001) and mean time to report medication errors using the app was not significantly different between each session (P=.70) with an overall mean time of 6.7 (SD 2.4) minutes. Testers found the app easy to use, but doctors and nurses were unfamiliar with terms used especially medication process at which error occurred and type of error. Although, testers agreed the app can be used in the future for reporting, they were apprehensive about security, validation, and abuse of feedback featured in the app.ConclusionsMERA can be used to report medication errors easily by various health care personnel and it has the capacity to provide feedback on reporting. However, education on medication error reporting should be provided to doctors and nurses in Malaysia and the security of the app needs to be established to boost reporting by this method.

Project description:Current approaches to understanding medication ordering errors rely on relatively small manually captured error samples. These approaches are resource-intensive, do not scale for computerized provider order entry (CPOE) systems, and are likely to miss important risk factors associated with medication ordering errors. Previously, we described a dataset of CPOE-based medication voiding accompanied by univariable and multivariable regression analyses. However, these traditional techniques require expert guidance and may perform poorly compared to newer approaches. In this paper, we update that analysis using machine learning (ML) models to predict erroneous medication orders and identify its contributing factors. We retrieved patient demographics (race/ethnicity, sex, age), clinician characteristics, type of medication order (inpatient, prescription, home medication by history), and order content. We compared logistic regression, random forest, boosted decision trees, and artificial neural network models. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). The dataset included 5,804,192 medication orders, of which 28,695 (0.5%) were voided. ML correctly classified voids at reasonable accuracy; with a positive predictive value of 10%, ~20% of errors were included. Gradient boosted decision trees achieved the highest AUROC (0.7968) and AUPRC (0.0647) among all models. Logistic regression had the poorest performance. Models identified predictive factors with high face validity (e.g., student orders), and a decision tree revealed interacting contexts with high rates of errors not identified by previous regression models. Prediction models using order-entry information offers promise for error surveillance, patient safety improvements, and targeted clinical review. The improved performance of models with complex interactions points to the importance of contextual medication ordering information for understanding contributors to medication errors.

Project description:BackgroundHigh blood pressure (hypertension) disproportionately affects African American/Black (Black) women. Previous research suggests that self-managing hypertension may be challenging, yet mobile applications (apps) can help to empower patients and increase medication adherence. We developed questions to test the usability of evaluating the WHISE (Wellness, Hypertension, Information Sharing, Self-Management, Education) mobile app for Black women with hypertension.MethodsFifteen participants completed usability testing; five were potential app users (Black women with hypertension); each invited two of their peers to participate. Each testing session (n=5) included a brief overview of the app, time for participants to complete surveys and have an active discussion about the app (concurrent and retrospective think-aloud, concurrent and retrospective probing, per usability.gov), and observation of participants' body language during the session. Testing sessions were designed to familiarize participants with the app's features and examine their navigating ability.ResultsThe app received overwhelmingly positive feedback, with 80% of participants finding it to be a valuable tool in hypertension management. Participants praised the app's user-friendliness and educational value, with one stating, 'It is a good educational piece for helping people manage hypertension, at least to understand its basics.' Another participant highlighted the potential for community support, saying, 'Having a community, having some people to be accountable, to check in with and see how things are going, could encourage and motivate people to be more diligent about managing their hypertension.' Some participants also provided constructive feedback, suggesting font size adjustments (73%) and color scheme changes (60%) for certain screens.ConclusionsBased on the feedback we received, we were able to mitigate the participants' concerns about font size and color and create tutorial videos to guide future users in using the app. We completed these changes prior to deploying the app in our randomized clinical controlled trial.

Dataset Information

Formative Usability Testing Reduces Severe Blood Product Ordering Errors.

Publications

Formative Usability Testing Reduces Severe Blood Product Ordering Errors.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets