The tradeoffs between safety and alert fatigue: Data from a national evaluation of hospital medication-related clinical decision support.
Ontology highlight
ABSTRACT: Objective
The study sought to evaluate the overall performance of hospitals that used the Computerized Physician Order Entry Evaluation Tool in both 2017 and 2018, along with their performance against fatal orders and nuisance orders.Materials and methods
We evaluated 1599 hospitals that took the test in both 2017 and 2018 by using their overall percentage scores on the test, along with the percentage of fatal orders appropriately alerted on, and the percentage of nuisance orders incorrectly alerted on.Results
Hospitals showed overall improvement; the mean score in 2017 was 58.1%, and this increased to 66.2% in 2018. Fatal order performance improved slightly from 78.8% to 83.0% (P < .001), though there was almost no change in nuisance order performance (89.0% to 89.7%; P = .43). Hospitals alerting on one or more nuisance orders had a 3-percentage-point increase in their overall score.Discussion
Despite the improvement of overall scores in 2017 and 2018, there was little improvement in fatal order performance, suggesting that hospitals are not targeting the deadliest orders first. Nuisance order performance showed almost no improvement, and some hospitals may be achieving higher scores by overalerting, suggesting that the thresholds for which alerts are fired from are too low.Conclusions
Although hospitals improved overall from 2017 to 2018, there is still important room for improvement for both fatal and nuisance orders. Hospitals that incorrectly alerted on one or more nuisance orders had slightly higher overall performance, suggesting that some hospitals may be achieving higher scores at the cost of overalerting, which has the potential to cause clinician burnout and even worsen safety.
SUBMITTER: Co Z
PROVIDER: S-EPMC7647300 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundComputerized physician order entry (CPOE) systems are incorporated into clinical decision support systems (CDSSs) to reduce medication errors and improve patient safety. Automatic alerts generated from CDSSs can directly assist physicians in making useful clinical decisions and can help shape prescribing behavior. Multiple studies reported that approximately 90%-96% of alerts are overridden by physicians, which raises questions about the effectiveness of CDSSs. There is intense interest in developing sophisticated methods to combat alert fatigue, but there is no consensus on the optimal approaches so far.ObjectiveOur objective was to develop machine learning prediction models to predict physicians' responses in order to reduce alert fatigue from disease medication-related CDSSs.MethodsWe collected data from a disease medication-related CDSS from a university teaching hospital in Taiwan. We considered prescriptions that triggered alerts in the CDSS between August 2018 and May 2019. Machine learning models, such as artificial neural network (ANN), random forest (RF), naïve Bayes (NB), gradient boosting (GB), and support vector machine (SVM), were used to develop prediction models. The data were randomly split into training (80%) and testing (20%) datasets.ResultsA total of 6453 prescriptions were used in our model. The ANN machine learning prediction model demonstrated excellent discrimination (area under the receiver operating characteristic curve [AUROC] 0.94; accuracy 0.85), whereas the RF, NB, GB, and SVM models had AUROCs of 0.93, 0.91, 0.91, and 0.80, respectively. The sensitivity and specificity of the ANN model were 0.87 and 0.83, respectively.ConclusionsIn this study, ANN showed substantially better performance in predicting individual physician responses to an alert from a disease medication-related CDSS, as compared to the other models. To our knowledge, this is the first study to use machine learning models to predict physician responses to alerts; furthermore, it can help to develop sophisticated CDSSs in real-world clinical settings.
| S-EPMC7714650 | biostudies-literature
Project description:BACKGROUND:Clinical decision support systems (CDSSs) are a good strategy for preventing medication errors and reducing the incidence and severity of adverse drug events (ADEs). However, these systems are not very effective and are subject to multiple limitations that prevent their implementation in clinical practice. OBJECTIVES:The objective of this study was to evaluate the effectiveness of an advanced CDSS, HIGEA, which generates alerts based on predefined clinical rules to identify patients at risk of an ADE. METHODS:A multidisciplinary team defined the system and the clinical rules focusing on medication errors commonly encountered in clinical practice. Four intervention programs were defined: (1) dose adjustment in renal impairment; (2) adjustment of anticoagulation/antiplatelet therapy; (3) detection of biochemical/hematologic toxicities; and (4) therapeutic drug monitoring. We performed a 6-month observational prospective study to analyze the effectiveness of these clinical rules by calculating the positive predictive value (PPV). RESULTS:The team defined 211 clinical rules. During the study period, HIGEA generated 1,086 alerts (8.9 alerts per working day), which were reviewed by pharmacists. Fifty-one percent (554/1,086) of alerts generated an intervention to prevent a possible ADE; of these, 66% (368/554) required a documented modification to therapy owing to a real prescription error intercepted. The intervention program that induced the highest number of modifications to therapy was the dose adjustment in renal impairment program (PPV?=?0.51), followed by the adjustment of anticoagulation/antiplatelet therapy program (PPV?=?0.24). The percentage of accepted interventions was similar in surgical units (68%), medical units (67%), and critical care units (63%). CONCLUSION:Our study offers evidence that HIGEA is highly effective in preventing potential ADEs at the prescription stage.
| S-EPMC6637024 | biostudies-literature
Project description:OBJECTIVE:Alert fatigue limits the effectiveness of medication safety alerts, a type of computerized clinical decision support (CDS). Researchers have suggested alternative interactive designs, as well as tailoring alerts to clinical roles. As examples, alerts may be tiered to convey risk, and certain alerts may be sent to pharmacists. We aimed to evaluate which variants elicit less alert fatigue. MATERIALS AND METHODS:We searched for articles published between 2007 and 2017 using the PubMed, Embase, CINAHL, and Cochrane databases. We included articles documenting peer-reviewed empirical research that described the interactive design of a CDS system, to which clinical role it was presented, and how often prescribers accepted the resultant advice. Next, we compared the acceptance rates of conventional CDS-presenting prescribers with interruptive modal dialogs (ie, "pop-ups")-with alternative designs, such as role-tailored alerts. RESULTS:Of 1011 articles returned by the search, we included 39. We found different methods for measuring acceptance rates; these produced incomparable results. The most common type of CDS-in which modals interrupted prescribers-was accepted the least often. Tiering by risk, providing shortcuts for common corrections, requiring a reason to override, and tailoring CDS to match the roles of pharmacists and prescribers were the most common alternatives. Only 1 alternative appeared to increase prescriber acceptance: role tailoring. Possible reasons include the importance of etiquette in delivering advice, the cognitive benefits of delegation, and the difficulties of computing "relevance." CONCLUSIONS:Alert fatigue may be mitigated by redesigning the interactive behavior of CDS and tailoring CDS to clinical roles. Further research is needed to develop alternative designs, and to standardize measurement methods to enable meta-analyses.
| S-EPMC6748819 | biostudies-literature
Project description:BackgroundSepsis remains the top cause of morbidity and mortality of hospitalised patients despite concerted efforts. Clinical decision support for sepsis has shown mixed results reflecting heterogeneous populations, methodologies and interventions.ObjectivesTo determine whether the addition of a real-time electronic health record (EHR)-based clinical decision support alert improves adherence to treatment guidelines and clinical outcomes in hospitalised patients with suspected severe sepsis.DesignPatient-level randomisation, single blinded.SettingMedical and surgical inpatient units of an academic, tertiary care medical centre.Patients1123 adults over the age of 18 admitted to inpatient wards (intensive care units (ICU) excluded) at an academic teaching hospital between November 2014 and March 2015.InterventionsPatients were randomised to either usual care or the addition of an EHR-generated alert in response to a set of modified severe sepsis criteria that included vital signs, laboratory values and physician orders.Measurements and main resultsThere was no significant difference between the intervention and control groups in primary outcome of the percentage of patients with new antibiotic orders at 3 hours after the alert (35% vs 37%, p=0.53). There was no difference in secondary outcomes of in-hospital mortality at 30 days, length of stay greater than 72 hours, rate of transfer to ICU within 48 hours of alert, or proportion of patients receiving at least 30 mL/kg of intravenous fluids.ConclusionsAn EHR-based severe sepsis alert did not result in a statistically significant improvement in several sepsis treatment performance measures.
| S-EPMC6860967 | biostudies-literature
Project description:BackgroundThe clinical decision support system(CDSS) has potential to improving medication safety. However, the effects of the intervention were conflicting and uncertain. Meanwhile, the reporting and methodological quality of this field were unknown.ObjectiveThe aim of this overview is to evaluate the effects of CDSS on medication safety and to examine the methodological and reporting quality.MethodsPubMed, Embase and Cochrane Library were searched to August 2015. Systematic reviews (SRs) investigating the effects of CDSS on medication safety were included. Outcomes were determined in advance and assessed separately for process of care and patient outcomes. The methodological quality was assessed by Assessment of Multiple Systematic Reviews (AMSTAR) and the reporting quality was examined by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).ResultsTwenty systematic reviews, consisting of 237 unique randomized controlled trials(RCTs) and 176 non-RCTs were included. Evidence that CDSS significantly impacted process of care was found in 108 out of 143 unique studies of the 16 SRs examining this effect (75%). Only 18 out of 90 unique studies of the 13 SRs reported significantly evidence that CDSS positively impacted patient outcomes (20%). Ratings for the overall scores of AMSTAR resulted in a mean score of 8.3 with a range of scores from 7.5 to 10.5. The reporting quality was varied. Some contents were particularly strong. However, some contents were poor.ConclusionsCDSS reduces medication error by obviously improving process of care and inconsistently improving patient outcomes. Larger samples and longer-term studies are required to ensure more reliable evidence base on the effects of CDSS on patient outcomes. The methodological and reporting quality were varied and some realms need to be improved.
| S-EPMC5157990 | biostudies-literature
Project description:Objective:To develop an empirically derived taxonomy of clinical decision support (CDS) alert malfunctions. Materials and Methods:We identified CDS alert malfunctions using a mix of qualitative and quantitative methods: (1) site visits with interviews of chief medical informatics officers, CDS developers, clinical leaders, and CDS end users; (2) surveys of chief medical informatics officers; (3) analysis of CDS firing rates; and (4) analysis of CDS overrides. We used a multi-round, manual, iterative card sort to develop a multi-axial, empirically derived taxonomy of CDS malfunctions. Results:We analyzed 68 CDS alert malfunction cases from 14 sites across the United States with diverse electronic health record systems. Four primary axes emerged: the cause of the malfunction, its mode of discovery, when it began, and how it affected rule firing. Build errors, conceptualization errors, and the introduction of new concepts or terms were the most frequent causes. User reports were the predominant mode of discovery. Many malfunctions within our database caused rules to fire for patients for whom they should not have (false positives), but the reverse (false negatives) was also common. Discussion:Across organizations and electronic health record systems, similar malfunction patterns recurred. Challenges included updates to code sets and values, software issues at the time of system upgrades, difficulties with migration of CDS content between computing environments, and the challenge of correctly conceptualizing and building CDS. Conclusion:CDS alert malfunctions are frequent. The empirically derived taxonomy formalizes the common recurring issues that cause these malfunctions, helping CDS developers anticipate and prevent CDS malfunctions before they occur or detect and resolve them expediently.
| S-EPMC6019061 | biostudies-literature
Project description:Background: Children have a higher risk of medication errors (MEs) than adults. The Institute for Safe Medication Practice (ISMP) defined high-alert medications (HAMs) as a group of medications that could cause significant patient harm or even death when they are used in error. Nurses are actively involved in and responsible for patient care, especially in medication administration. This study aimed to estimate the knowledge, decision-making basis and confidence and decision support needs related to HAMs among pediatric nurses in China. Methods: A web-based, cross-sectional survey was conducted among pediatric nurses who were recruited from 14 member hospitals of the Pediatric Nursing Alliance of National Children's Medical Center in China using a convenient sampling technique. Data were collected using a self-administered instrument composed of four parts: the demographic characteristics of participants, participants' knowledge about HAMs, participants' self-evaluation of the basis of and confidence in decision-making, and decision support needs regarding HAMs. Among the participants, the maximum score for HAM knowledge was 100. All data were entered and analyzed using SPSS 20.0. Results: A total of 966 nurses participated in this study. Nurses were found to have insufficient knowledge about HAMs, with a median (IQR) of 75.0 (70.0, 80.0), out of a maximum score of 100. Knowledge about HAM administration was significantly higher than that about HAM regulation, with a p value < 0.001. The three lowest-scoring items concerned HAM regulation, and the "Treat fentanyl skin patches as a regulated narcotic" item obtained the lowest score, with only 1/5 of respondents answering it correctly. Most participants reported that their basis for decision-making about HAMs was drug instructions (90.0%) or drug handbooks (81.9%) and evaluated their confidence in decision-making about HAMs as high or relatively high (84.6%). The decision-making difficulties when encountering HAMs focused on most stages of HAM administration, especially the appropriateness of prescriptions, checks, preparation and administration. The vast majority of participants assessed decision support as necessary or very necessary (92.0%), and the most popular options for decision support were computerized clinical decision support systems (46.4%) and real-time online communication with pharmacists (23.9%). Conclusion: Our study demonstrated the inadequacies in HAM knowledge, the basis and difficulty of decision-making, and decision support needs regarding HAMs in Chinese pediatric nurses. Nurses need greater support in HAM administration, including not only training but also adequate technology, mutually beneficial interprofessional collaboration, and a positive institutional culture.
| S-EPMC9343802 | biostudies-literature
Project description:BackgroundAlthough usage and acceptance are important factors for a successful implementation of clinical decision support systems for medication, most studies only concentrate on their design and outcome. Our objective was to comparatively investigate a set of traditional medication safety measures such as medication safety training for physicians, paper-based posters and checklists concerning potential medication problems versus the additional benefit of a computer-assisted medication check. We concentrated on usage, acceptance and suitability of such interventions in a busy emergency department (ED) of a 749 bed acute tertiary care hospital.MethodsA retrospective, qualitative evaluation study was conducted using a field observation and a questionnaire-based survey. Six physicians were observed while treating 20 patient cases; the questionnaire, based on the Technology Acceptance Model 2 (TAM2), has been answered by nine ED physicians.ResultsDuring field observations, we did not observe direct use of any of the implemented interventions for medication safety (paper-based and electronic). Questionnaire results indicated that the electronic medication safety check was the most frequently used intervention, followed by checklist and posters. However, despite their positive attitude, physicians most often stated that they use the interventions in only up to ten percent for subjectively "critical" orders. Main reasons behind the low usage were deficits in ease-of-use and fit to the workflow. The intention to use the interventions was rather high after overcoming these barriers.ConclusionsMethodologically, the study contributes to Technology Acceptance Model (TAM) research in an ED setting and confirms TAM2 as a helpful diagnostic tool in identifying barriers for a successful implementation of medication safety interventions. In our case, identified barriers explaining the low utilization of the implemented medication safety interventions - despite their positive reception - include deficits in accessibility, briefing for the physicians about the interventions, ease-of-use and compatibility to the working environment.
| S-EPMC3733614 | biostudies-literature
Project description:INTRODUCTION:Drug-drug interaction (DDI) alerts in hospital electronic medication management (EMM) systems are generated at the point of prescribing to warn doctors about potential interactions in their patients' medication orders. This project aims to determine the impact of DDI alerts on DDI rates and on patient harm in the inpatient setting. It also aims to identify barriers and facilitators to optimal use of alerts, quantify the alert burden posed to prescribers with implementation of DDI alerts and to develop algorithms to improve the specificity of DDI alerting systems. METHODS AND ANALYSIS:A controlled pre-post design will be used. Study sites include six major referral hospitals in two Australian states, New South Wales and Queensland. Three hospitals will act as control sites and will implement an EMM system without DDI alerts, and three as intervention sites with DDI alerts. The medical records of 280 patients admitted in the 6?months prior to and 6?months following implementation of the EMM system at each site (total 3360 patients) will be retrospectively reviewed by study pharmacists to identify potential DDIs, clinically relevant DDIs and associated patient harm. To identify barriers and facilitators to optimal use of alerts, 10-15 doctors working at each intervention hospital will take part in observations and interviews. Non-identifiable DDI alert data will be extracted from EMM systems 6-12 months after system implementation in order to quantify alert burden on prescribers. Finally, data collected from chart review and EMM systems will be linked with clinically relevant DDIs to inform the development of algorithms to trigger only clinically relevant DDI alerts in EMM systems. ETHICS AND DISSEMINATION:This research was approved by the Hunter New England Human Research Ethics Committee (18/02/21/4.07). Study results will be published in peer-reviewed journals and presented at local and international conferences and workshops.
| S-EPMC6701635 | biostudies-literature
Project description:AKI carries a significant mortality and morbidity risk. Use of a clinical decision support system (CDSS) might improve outcomes. We conducted a multicenter, sequential period analysis of 528,108 patients without ESRD before admission, from October of 2012 to September of 2015, to determine whether use of a CDSS reduces hospital length of stay and in-hospital mortality for patients with AKI. We compared patients treated 12 months before (181,696) and 24 months after (346,412) implementation of the CDSS. Coprimary outcomes were hospital mortality and length of stay adjusted by demographics and comorbidities. AKI was diagnosed in 64,512 patients (12.2%). Crude mortality rate fell from 10.2% before to 9.4% after CDSS implementation (odds ratio, 0.91; 95% confidence interval [95% CI], 0.86 to 0.96; P=0.001) for patients with AKI but did not change in patients without AKI (from 1.5% to 1.4%). Mean hospital duration decreased from 9.3 to 9.0 days (P<0.001) for patients with AKI, with no change for patients without AKI. In multivariate mixed-effects models, the adjusted odds ratio (95% CI) was 0.76 (0.70 to 0.83) for mortality and 0.66 (0.61 to 0.72) for dialysis (P<0.001). Change in adjusted hospital length of stay was also significant (incidence rate ratio, 0.91; 95% CI, 0.89 to 0.92), decreasing from 7.2 to 6.0 days for patients with AKI. Results were robust to sensitivity analyses and were sustained for the duration of follow-up. Hence, implementation of a CDSS for AKI resulted in a small but sustained decrease in hospital mortality, dialysis use, and length of stay.
| S-EPMC5791078 | biostudies-literature