A six-year repeated evaluation of computerized clinical decision support system user acceptability.
Ontology highlight
ABSTRACT:
SUBMITTER: Grout RW
PROVIDER: S-EPMC5836810 | biostudies-literature | 2018 Apr
REPOSITORIES: biostudies-literature
Publications
A six-year repeated evaluation of computerized clinical decision support system user acceptability.
Grout Randall W RW Cheng Erika R ER Carroll Aaron E AE Bauer Nerissa S NS Downs Stephen M SM
International journal of medical informatics 20180202
<h4>Objective</h4>Long-term acceptability among computerized clinical decision support system (CDSS) users in pediatrics is unknown. We examine user acceptance patterns over six years of our continuous computerized CDSS integration and updates.<h4>Materials and methods</h4>Users of Child Health Improvement through Computer Automation (CHICA), a CDSS integrated into clinical workflows and used in several urban pediatric community clinics, completed annual surveys including 11 questions covering u ...[more]
Similar Datasets
Project description:BACKGROUND:Emergency departments (EDs) frequently care for individuals with opioid use disorder (OUD). Buprenorphine (BUP) is an effective treatment option for patients with OUD that can safely be initiated in the ED. At present, BUP is rarely initiated as a part of routine ED care. Clinical decision support (CDS) could accelerate adoption of ED-initiated BUP into routine emergency care. OBJECTIVE:This study aimed to design and formatively evaluate a user-centered decision support tool for ED initiation of BUP for patients with OUD. METHODS:User-centered design with iterative prototype development was used. Initial observations and interviews identified workflows and information needs. The design team and key stakeholders reviewed prototype designs to ensure accuracy. A total of 5 prototypes were evaluated and iteratively refined based on input from 26 attending and resident physicians. RESULTS:Early feedback identified concerns with the initial CDS design: an alert with several screens. The timing of the alert led to quick dismissal without using the tool. User feedback on subsequent iterations informed the development of a flexible tool to support clinicians with varied levels of experience with the intervention by providing both one-click options for direct activation of care pathways and user-activated support for critical decision points. The final design resolved challenging navigation issues through targeted placement, color, and design of the decision support modules and care pathways. In final testing, users expressed that the tool could be easily learned without training and was reasonable for use during routine emergency care. CONCLUSIONS:A user-centered design process helped designers to better understand users' needs for a Web-based clinical decision tool to support ED initiation of BUP for OUD. The process identified varying needs across user experience and familiarity with the protocol, leading to a flexible design supporting both direct care pathways and user-initiated decision support.
| S-EPMC6414819 | biostudies-literature
Project description:ObjectiveIn 2004, results from The Texas Medication Algorithm Project (TMAP) showed better clinical outcomes for patients whose physicians adhered to a paper-and-pencil algorithm compared to patients who received standard clinical treatment for major depressive disorder (MDD). However, implementation of and fidelity to the treatment algorithm among various providers was observed to be inadequate. A computerized decision support system (CDSS) for the implementation of the TMAP algorithm for depression has since been developed to improve fidelity and adherence to the algorithm.MethodThis was a 2-group, parallel design, clinical trial (one patient group receiving MDD treatment from physicians using the CDSS and the other patient group receiving usual care) conducted at 2 separate primary care clinics in Texas from March 2005 through June 2006. Fifty-five patients with MDD (DSM-IV criteria) with no significant difference in disease characteristics were enrolled, 32 of whom were treated by physicians using CDSS and 23 were treated by physicians using usual care. The study's objective was to evaluate the feasibility and efficacy of implementing a CDSS to assist physicians acutely treating patients with MDD compared to usual care in primary care. Primary efficacy outcomes for depression symptom severity were based on the 17-item Hamilton Depression Rating Scale (HDRS(17)) evaluated by an independent rater.ResultsPatients treated by physicians employing CDSS had significantly greater symptom reduction, based on the HDRS(17), than patients treated with usual care (P < .001).ConclusionsThe CDSS algorithm, utilizing measurement-based care, was superior to usual care for patients with MDD in primary care settings. Larger randomized controlled trials are needed to confirm these findings.Trial registrationclinicaltrials.gov Identifier: NCT00551083.
| S-EPMC2736031 | biostudies-literature
Project description:BackgroundClinical decision support systems are implemented in many hospitals to prevent medication errors and associated harm. They are however associated with a high burden of false positive alerts and alert fatigue. The aim of this study was to evaluate a drug-drug interaction (DDI) clinical decision support system in terms of its performance, uptake and user satisfaction and to identify barriers and opportunities for improvement.MethodsA quantitative evaluation and end-user survey were performed in a large teaching hospital. First, very severe DDI alerts generated between 2019 and 2021 were evaluated retrospectively. Data collection comprised alert burden, override rates, the number of alert overrides reviewed by pharmacists and the resulting pharmacist recommendations as well as their acceptance rate. Second, an e-survey was carried out among prescribers to assess satisfaction, usefulness and relevance of DDI alerts as well as reasons for overriding.ResultsA total of 38,409 very severe DDI alerts were generated, of which 88.2% were overridden by the prescriber. In 3.2% of reviewed overrides, a recommendation by the pharmacist was provided, of which 79.2% was accepted. False positive alerts were caused by a too broad screening interval and lack of incorporation of patient-specific characteristics, such as QTc values. Co-prescribing of a non-vitamin K oral anticoagulant and a low molecular weight heparin accounted for 49.8% of alerts, of which 92.2% were overridden. In 88 (1.1%) of these overridden alerts, concurrent therapy was still present. Despite the high override rate, the e-survey revealed that the DDI clinical decision support system was found useful by prescribers.ConclusionsIdentified barriers were the lack of DDI-specific screening intervals and inclusion of patient-specific characteristics, both leading to a high number of false positive alerts and risk for alert fatigue. Despite these barriers, the added value of the DDI clinical decision support system was recognized by prescribers. Hence, integration of DDI-specific screening intervals and patient-specific characteristics is warranted to improve the performance of the DDI software.
| S-EPMC8864797 | biostudies-literature
Project description:ObjectiveComputerized clinical decision support systems (CCDSSs) promise improvements in care quality; however, uptake is often suboptimal. We sought to characterize system use, its predictors, and user feedback for the Electronic Asthma Management System (eAMS)-an electronic medical record system-integrated, point-of-care CCDSS for asthma-and applied the GUIDES checklist as a framework to identify areas for improvement.Materials and methodsThe eAMS was tested in a 1-year prospective cohort study across 3 Ontario primary care sites. We recorded system usage by clinicians and patient characteristics through system logs and chart reviews. We created multivariable models to identify predictors of (1) CCDSS opening and (2) creation of a self-management asthma action plan (AAP) (final CCDSS step). Electronic questionnaires captured user feedback.ResultsOver 1 year, 490 asthma patients saw 121 clinicians. The CCDSS was opened in 205 of 1033 (19.8%) visits and an AAP created in 121 of 1033 (11.7%) visits. Multivariable predictors of opening the CCDSS and producing an AAP included clinic site, having physician-diagnosed asthma, and presenting with an asthma- or respiratory-related complaint. The system usability scale score was 66.3 ± 16.5 (maximum 100). Reported usage barriers included time and system accessibility.DiscussionThe eAMS was used in a minority of asthma patient visits. Varying workflows and cultures across clinics, physician beliefs regarding asthma diagnosis, and relevance of the clinical complaint influenced uptake.ConclusionsConsidering our findings in the context of the GUIDES checklist helped to identify improvements to drive uptake and provides lessons relevant to CCDSS design across diseases.
| S-EPMC7309244 | biostudies-literature
Project description:ObjectiveWith the increasing prevalence of type 2 diabetes (T2D) in youth, primary care providers must identify patients at high risk and implement evidence-based screening promptly. Clinical decision support systems (CDSSs) provide clinicians with personalized reminders according to best evidence. One example is the Child Health Improvement through Computer Automation (CHICA) system, which, as we have previously shown, significantly improves screening for T2D. Given that the long-term success of any CDSS depends on its acceptability and its users' perceptions, we examined what clinicians think of the CHICA diabetes module.MethodsCHICA users completed an annual quality improvement and satisfaction questionnaire. Between May and August of 2015 and 2016, the survey included two statements related to the T2D-module: (1) "CHICA improves my ability to identify patients who might benefit from screening for T2D" and (2) "CHICA makes it easier to get the lab tests necessary to identify patients who have diabetes or prediabetes." Answers were scored using a 5-point Likert scale and were later converted to a 2-point scale: agree and disagree. The Pearson chi-square test was used to assess the relationship between responses and the respondents. Answers per cohort were compared using the Mann-Whitney U-test.ResultsThe majority of respondents (N = 60) agreed that CHICA improved their ability to identify patients who might benefit from screening but disagreed as to whether it helped them get the necessary laboratories. Scores were comparable across both years.ConclusionCHICA was endorsed as being effective for T2D screening. Research is needed to improve satisfaction for getting laboratories with CHICA.
| S-EPMC7220796 | biostudies-literature
Project description:Glucocorticoids are widely used for a variety of diseases, but the prevention of glucocorticoid-induced osteoporosis is sometimes neglected. Therefore, the effectiveness of a computerized clinical decision support system (CDSS) to improve the performance rate of preventive care for glucocorticoid-induced osteoporosis was evaluated. We conducted a prospective cohort study of outpatients who used glucocorticoids for three months or longer and who met the indication for preventive care based on a guideline. The CDSS recommended bisphosphonate (BP) prescription and bone mineral density (BMD) testing based on the risk of osteoporosis. The observation period was one year (phase 1: October 2017-September 2018) before implementation and the following one year (phase 2: October 2018-September 2019) after implementation of the CDSS. Potential alerts were collected without displaying them during phase 1, and the alerts were displayed during phase 2. We measured BP prescriptions and BMD testing for long-term prescription of glucocorticoids. A total of 938 patients (phase 1, 457 patients; phase 2, 481 patients) were included, and the baseline characteristics were similar between the phases. The median age was 71 years, and men accounted for 51%. The primary disease for prescription of glucocorticoids was rheumatic disease (28%), followed by hematologic diseases (18%). The prevalence of patients who needed an alert for BP prescription (67% vs. 63%, P = 0.24) and the acceptance rate of BP prescription (16% vs. 19%, P = 0.33) were similar between the phases. The number of patients who had orders for BMD testing was significantly increased (4% vs. 24%, P < 0.001) after CDSS implementation. The number of patients who needed an alert for BMD testing was significantly decreased from 93% in phase 1 to 87% in phase 2 (P = 0.004). In conclusion, the CDSS significantly increased BMD testing in patients with a higher risk of glucocorticoid-induced osteoporosis, but did not increase BP prescription.
| S-EPMC9440130 | biostudies-literature
Project description:BackgroundAutomated clinical decision support systems (CDSS) are associated with improvements in health care delivery to those with long-term conditions, including diabetes. A CDSS was introduced to two Scottish regions (combined diabetes population ~30 000) via a national diabetes electronic health record. This study aims to describe users' reactions to the CDSS and to quantify impact on clinical processes and outcomes over two improvement cycles: December 2013 to February 2014 and August 2014 to November 2014.MethodsFeedback was sought via patient questionnaires, health care professional (HCP) focus groups, and questionnaires. Multivariable regression was used to analyze HCP SCI-Diabetes usage (with respect to CDSS message presence/absence) and case-control comparison of clinical processes/outcomes. Cases were patients whose HCP received a CDSS messages during the study period. Closely matched controls were selected from regions outside the study, following similar clinical practice (without CDSS). Clinical process measures were screening rates for diabetes-related complications. Clinical outcomes included HbA1c at 1 year.ResultsThe CDSS had no adverse impact on consultations. HCPs were generally positive toward CDSS and used it within normal clinical workflow. CDSS messages were generated for 5692 cases, matched to 10 667 controls. Following clinic, the probability of patients being appropriately screened for complications more than doubled for most measures. Mean HbA1c improved in cases and controls but more so in cases (-2.3 mmol/mol [-0.2%] versus -1.1 [-0.1%], P = .003).Discussion and conclusionsThe CDSS was well received; associated with improved efficiencies in working practices; and large improvements in guideline adherence. These evidence-based, early interventions can significantly reduce costly and devastating complications.
| S-EPMC5851216 | biostudies-literature
Project description:BackgroundDespite the unprecedented performance of deep learning algorithms in clinical domains, full reviews of algorithmic predictions by human experts remain mandatory. Under these circumstances, artificial intelligence (AI) models are primarily designed as clinical decision support systems (CDSSs). However, from the perspective of clinical practitioners, the lack of clinical interpretability and user-centered interfaces hinders the adoption of these AI systems in practice.ObjectiveThis study aims to develop an AI-based CDSS for assisting polysomnographic technicians in reviewing AI-predicted sleep staging results. This study proposed and evaluated a CDSS that provides clinically sound explanations for AI predictions in a user-centered manner.MethodsOur study is based on a user-centered design framework for developing explanations in a CDSS that identifies why explanations are needed, what information should be contained in explanations, and how explanations can be provided in the CDSS. We conducted user interviews, user observation sessions, and an iterative design process to identify three key aspects for designing explanations in the CDSS. After constructing the CDSS, the tool was evaluated to investigate how the CDSS explanations helped technicians. We measured the accuracy of sleep staging and interrater reliability with macro-F1 and Cohen κ scores to assess quantitative improvements after our tool was adopted. We assessed qualitative improvements through participant interviews that established how participants perceived and used the tool.ResultsThe user study revealed that technicians desire explanations that are relevant to key electroencephalogram (EEG) patterns for sleep staging when assessing the correctness of AI predictions. Here, technicians wanted explanations that could be used to evaluate whether the AI models properly locate and use these patterns during prediction. On the basis of this, information that is closely related to sleep EEG patterns was formulated for the AI models. In the iterative design phase, we developed a different visualization strategy for each pattern based on how technicians interpreted the EEG recordings with these patterns during their workflows. Our evaluation study on 9 polysomnographic technicians quantitatively and qualitatively investigated the helpfulness of the tool. For technicians with <5 years of work experience, their quantitative sleep staging performance improved significantly from 56.75 to 60.59 with a P value of .05. Qualitatively, participants reported that the information provided effectively supported them, and they could develop notable adoption strategies for the tool.ConclusionsOur findings indicate that formulating clinical explanations for automated predictions using the information in the AI with a user-centered design process is an effective strategy for developing a CDSS for sleep staging.
| S-EPMC8811694 | biostudies-literature
Project description:ObjectiveWe created a system using a triad of change management, electronic surveillance, and algorithms to detect sepsis and deliver highly sensitive and specific decision support to the point of care using a mobile application. The investigators hypothesized that this system would result in a reduction in sepsis mortality.MethodsA before-and-after model was used to study the impact of the interventions on sepsis-related mortality. All patients admitted to the study units were screened per the Institute for Healthcare Improvement Surviving Sepsis Guidelines using real-time electronic surveillance. Sepsis surveillance algorithms that adjusted clinical parameters based on comorbid medical conditions were deployed for improved sensitivity and specificity. Nurses received mobile alerts for all positive sepsis screenings as well as severe sepsis and shock alerts over a period of 10 months. Advice was given for early goal-directed therapy. Sepsis mortality during a control period from January 1, 2011 to September 30, 2013 was used as baseline for comparison.ResultsThe primary outcome, sepsis mortality, decreased by 53% (P = 0.03; 95% CI, 1.06-5.25). The 30-day readmission rate reduced from 19.08% during the control period to 13.21% during the study period (P = 0.05; 95% CI, 0.97-2.52). No significant change in length of hospital stay was noted. The system had observed sensitivity of 95% and specificity of 82% for detecting sepsis compared to gold-standard physician chart review.ConclusionA program consisting of change management and electronic surveillance with highly sensitive and specific decision support delivered to the point of care resulted in significant reduction in deaths from sepsis.
| S-EPMC7654083 | biostudies-literature
Project description:Laboratory test overutilization increases health care costs, leads to unwarranted investigations, and may have a negative impact on health outcomes. The American Society of Clinical Pathology, in its Choosing Wisely Campaign, advocates that inflammation be investigated with C-reactive protein (CRP) instead of Erythrocyte Sedimentation Rate (ESR). London Health Sciences Centre (LHSC), a tertiary care hospital organization in Ontario, Canada, set a goal to reduce inappropriate ESR orders by 50%. After developing appropriateness criteria for ESR, we used a series of PDSA cycles to reduce inappropriate ESR ordering and analyzed our results with an interrupted time series design. Our intervention began with an educational bulletin and moved to city-wide implementation of computerized Clinical Decision Support (CDS). After implementation, ESR orders decreased by 40% from 386 orders per week to 241 orders per week. Our results are supported by previous literature on the effectiveness of CDS in reducing overutilization and suggest that provider habit is a significant contributor to inappropriate ordering.
| S-EPMC4822023 | biostudies-other