Evaluating the Efficiency and Safety of Speech Recognition within a Commercial Electronic Health Record System: A Replication Study.
Ontology highlight
ABSTRACT:
SUBMITTER: Hodgson T
PROVIDER: S-EPMC5955718 | biostudies-literature | 2018 Apr
REPOSITORIES: biostudies-literature
Publications
Evaluating the Efficiency and Safety of Speech Recognition within a Commercial Electronic Health Record System: A Replication Study.
Hodgson Tobias T Magrabi Farah F Coiera Enrico E
Applied clinical informatics 20180401 2
<h4>Objective</h4>To conduct a replication study to validate previously identified significant risks and inefficiencies associated with the use of speech recognition (SR) for documentation within an electronic health record (EHR) system.<h4>Methods</h4>Thirty-five emergency department clinicians undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR using a commercial EHR system. The experiment design, setting, and tasks (E2) replicated an earlier study (E ...[more]
Similar Datasets
Project description:ObjectiveTo compare the efficiency and safety of using speech recognition (SR) assisted clinical documentation within an electronic health record (EHR) system with use of keyboard and mouse (KBM).MethodsThirty-five emergency department clinicians undertook randomly allocated clinical documentation tasks using KBM or SR on a commercial EHR system. Tasks were simple or complex, and with or without interruption. Outcome measures included task completion times and observed errors. Errors were classed by their potential for patient harm. Error causes were classified as due to IT system/system integration, user interaction, comprehension, or as typographical. User-related errors could be by either omission or commission.ResultsMean task completion times were 18.11% slower overall when using SR compared to KBM (P = .001), 16.95% slower for simple tasks (P = .050), and 18.40% slower for complex tasks (P = .009). Increased errors were observed with use of SR (KBM 32, SR 138) for both simple (KBM 9, SR 75; P < 0.001) and complex (KBM 23, SR 63; P < 0.001) tasks. Interruptions did not significantly affect task completion times or error rates for either modality.DiscussionFor clinical documentation, SR was slower and increased the risk of documentation errors, including errors with the potential to cause clinical harm compared to KBM. Some of the observed increase in errors may be due to suboptimal SR to EHR integration and workflow.ConclusionUse of SR to drive interactive clinical documentation in the EHR requires careful evaluation. Current generation implementations may require significant development before they are safe and effective. Improving system integration and workflow, as well as SR accuracy and user-focused error correction strategies, may improve SR performance.
| S-EPMC7651984 | biostudies-literature
Project description:To develop and implement a clinical decision support (CDS) tool to improve antibiotic prescribing in neonatal intensive care units (NICUs) and to evaluate user acceptance of the CDS tool.Following sociotechnical analysis of NICU prescribing processes, a CDS tool for empiric and targeted antimicrobial therapy for healthcare-associated infections (HAIs) was developed and incorporated into a commercial electronic health record (EHR) in two NICUs. User logs were reviewed and NICU prescribers were surveyed for their perceptions of the CDS tool.The CDS tool aggregated selected laboratory results, including culture results, to make treatment recommendations for common clinical scenarios. From July 2010 to May 2012, 1,303 CDS activations for 452 patients occurred representing 22% of patients prescribed antibiotics during this period. While NICU clinicians viewed two culture results per tool activation, prescribing recommendations were viewed during only 15% of activations. Most (63%) survey respondents were aware of the CDS tool, but fewer (37%) used it during their most recent NICU rotation. Respondents considered the most useful features to be summarized culture results (43%) and antibiotic recommendations (48%).During the study period, the CDS tool functionality was hindered by EHR upgrades, implementation of a new laboratory information system, and changes to antimicrobial testing methodologies. Loss of functionality may have reduced viewing antibiotic recommendations. In contrast, viewing culture results was frequently performed, likely because this feature was perceived as useful and functionality was preserved.To improve CDS tool visibility and usefulness, we recommend early user and information technology team involvement which would facilitate use and mitigate implementation challenges.
| S-EPMC4081742 | biostudies-literature
Project description:BackgroundThe rapid expansion in the use of electronic health records (EHR) has increased the number of medical errors originating in health information systems (HIS). The sociotechnical approach helps in understanding risks in the development, implementation, and use of EHR and health information technology (HIT) while accounting for complex interactions of technology within the health care system.ObjectiveThis study addresses two important questions: (1) "which of the common EHR error types are associated with perceived high- and extreme-risk severity ratings among EHR users?", and (2) "which variables are associated with high- and extreme-risk severity ratings?"MethodsThis study was a quantitative, non-experimental, descriptive study of EHR users. We conducted a cross-sectional web-based questionnaire study at the largest hospital district in Finland. Statistical tests included the reliability of the summative scales tested with Cronbach's alpha. Logistic regression served to assess the association of the independent variables to each of the eight risk factors examined.ResultsA total of 2864 eligible respondents provided the final data. Almost half of the respondents reported a high level of risk related to the error type "extended EHR unavailability". The lowest overall risk level was associated with "selecting incorrectly from a list of items". In multivariate analyses, profession and clinical unit proved to be the strongest predictors for high perceived risk. Physicians perceived risk levels to be the highest ( P<.001 in six of eight error types), while emergency departments, operating rooms, and procedure units were associated with higher perceived risk levels ( P<.001 in four of eight error types). Previous participation in eLearning courses on EHR-use was associated with lower risk for some of the risk factors.ConclusionsBased on a large number of Finnish EHR users in hospitals, this study indicates that HIT safety hazards should be taken very seriously, particularly in operating rooms, procedure units, emergency departments, and intensive care units/critical care units. Health care organizations should use proactive and systematic assessments of EHR risks before harmful events occur. An EHR training program should be compulsory for all EHR users in order to address EHR safety concerns resulting from the failure to use HIT appropriately.
| S-EPMC4890731 | biostudies-literature
Project description:ObjectiveMethods to identify and study safety risks of electronic health records (EHRs) are underdeveloped and largely depend on limited end-user reports. "Safety huddles" have been found useful in creating a sense of collective situational awareness that increases an organization's capacity to respond to safety concerns. We explored the use of safety huddles for identifying and learning about EHR-related safety concerns.DesignData were obtained from daily safety huddle briefing notes recorded at a single midsized tertiary-care hospital in the United States over 1 year. Huddles were attended by key administrative, clinical, and information technology staff. We conducted a content analysis of huddle notes to identify what EHR-related safety concerns were discussed. We expanded a previously developed EHR-related error taxonomy to categorize types of EHR-related safety concerns recorded in the notes.ResultsOn review of daily huddle notes spanning 249 days, we identified 245 EHR-related safety concerns. For our analysis, we defined EHR technology to include a specific EHR functionality, an entire clinical software application, or the hardware system. Most concerns (41.6%) involved " EHR technology working incorrectly, " followed by 25.7% involving " EHR technology not working at all. " Concerns related to "EHR technology missing or absent" accounted for 16.7%, whereas 15.9% were linked to " user errors ."ConclusionsSafety huddles promoted discussion of several technology-related issues at the organization level and can serve as a promising technique to identify and address EHR-related safety concerns. Based on our findings, we recommend that health care organizations consider huddles as a strategy to promote understanding and improvement of EHR safety.
| S-EPMC5391729 | biostudies-literature
Project description:ImportanceThe use of electronic health records (EHRs) is directly associated with physician burnout. An underlying factor associated with burnout may be EHR-related fatigue owing to insufficient user-centered interface design and suboptimal usability.ObjectiveTo examine the association between EHR use and fatigue, as measured by pupillometry, and efficiency, as measured by mouse clicks, time, and number of EHR screens, among intensive care unit (ICU) physicians completing a simulation activity in a prominent EHR.Design, setting, and participantsA cross-sectional, simulation-based EHR usability assessment of a leading EHR system was conducted from March 20 to April 5, 2018, among 25 ICU physicians and physician trainees at a southeastern US academic medical center. Participants completed 4 simulation patient cases in the EHR that involved information retrieval and task execution while wearing eye-tracking glasses. Fatigue was quantified through continuous eye pupil data; EHR efficiency was characterized through task completion time, mouse clicks, and EHR screen visits. Data were analyzed from June 1, 2018, to August 31, 2019.Main outcomes and measuresPrimary outcomes were physician fatigue, measured by pupillometry (with lower scores indicating greater fatigue), and EHR efficiency, measured by task completion times, number of mouse clicks, and number of screens visited during EHR simulation.ResultsThe 25 ICU physicians (13 women; mean [SD] age, 32.1 [6.1] years) who completed a simulation exercise involving 4 patient cases (mean [SD] completion time, 34:43 [11:41] minutes) recorded a total of 14 hours and 27 minutes of EHR activity. All physician participants experienced physiological fatigue at least once during the exercise, and 20 of 25 participants (80%) experienced physiological fatigue within the first 22 minutes of EHR use. Physicians who experienced EHR-related fatigue in 1 patient case were less efficient in the subsequent patient case, as demonstrated by longer task completion times (r = -0.521; P = .007), higher numbers of mouse clicks (r = -0.562; P = .003), and more EHR screen visits (r = -0.486; P = .01).Conclusions and relevanceThis study reports high rates of fatigue among ICU physicians during short periods of EHR simulation, which were negatively associated with EHR efficiency and included a carryover association across patient cases. More research is needed to investigate the underlying causes of EHR-associated fatigue, to support user-centered EHR design, and to inform safe EHR use policies and guidelines.
| S-EPMC7284310 | biostudies-literature
Project description:BackgroundTaiwan has insufficient nursing resources due to the high turnover rate of health care providers. Therefore, reducing the heavy workload of these employees is essential. Herein, speech transcription, which has various potential clinical applications, was employed for the documentation of nursing records. The requirement of including only one speaker per transcription facilitated data collection and system development. Moreover, authorization from patients was unnecessary.ObjectiveThe aim of this study was to construct a speech recognition system for nursing records such that health care providers can complete nursing records without typing or with only a few edits.MethodsNursing records in Taiwan are mainly written in Mandarin, with technical terms and abbreviations presented in both Mandarin and English. Therefore, the training set consisted of English code-switching information. Next, transfer learning (TL) and meta-TL (MTL) methods, which perform favorably in code-switching scenarios, were applied.ResultsAs of September 2021, the China Medical University Hospital Artificial Intelligence Speech (CMaiSpeech) data set was established by manually annotating approximately 100 hours of recordings from 525 speakers. The word error rate (WER) of the benchmark model of syllable-based TL was 29.54% in code-switching. The WER of the proposed model of syllable-based MTL was 22.20% in code-switching. The test set comprised 17,247 words. Moreover, in a clinical case, the proposed model of syllable-based MTL yielded a WER of 31.06% in code-switching. The clinical test set contained 1159 words.ConclusionsThis paper has two main contributions. First, the CMaiSpeech data set-a Mandarin-English corpus-has been established. Health care providers in Taiwan are often compelled to use a mixture of Mandarin and English in nursing records. Second, an automatic speech recognition system for nursing record document conversion was proposed. The proposed system can shorten the work handover time and further reduce the workload of health care providers.
| S-EPMC9773023 | biostudies-literature
Project description:ObjectiveA recent Institute of Medicine report called for attention to safety issues related to electronic health records (EHRs). We analyzed EHR-related safety concerns reported within a large, integrated healthcare system.MethodsThe Informatics Patient Safety Office of the Veterans Health Administration (VA) maintains a non-punitive, voluntary reporting system to collect and investigate EHR-related safety concerns (ie, adverse events, potential events, and near misses). We analyzed completed investigations using an eight-dimension sociotechnical conceptual model that accounted for both technical and non-technical dimensions of safety. Using the framework analysis approach to qualitative data, we identified emergent and recurring safety concerns common to multiple reports.ResultsWe extracted 100 consecutive, unique, closed investigations between August 2009 and May 2013 from 344 reported incidents. Seventy-four involved unsafe technology and 25 involved unsafe use of technology. A majority (70%) involved two or more model dimensions. Most often, non-technical dimensions such as workflow, policies, and personnel interacted in a complex fashion with technical dimensions such as software/hardware, content, and user interface to produce safety concerns. Most (94%) safety concerns related to either unmet data-display needs in the EHR (ie, displayed information available to the end user failed to reduce uncertainty or led to increased potential for patient harm), software upgrades or modifications, data transmission between components of the EHR, or 'hidden dependencies' within the EHR.DiscussionEHR-related safety concerns involving both unsafe technology and unsafe use of technology persist long after 'go-live' and despite the sophisticated EHR infrastructure represented in our data source. Currently, few healthcare institutions have reporting and analysis capabilities similar to the VA.ConclusionsBecause EHR-related safety concerns have complex sociotechnical origins, institutions with long-standing as well as recent EHR implementations should build a robust infrastructure to monitor and learn from them.
| S-EPMC4215044 | biostudies-literature
Project description:PurposeThe aim of this study was to develop and validate a large Korean sentence set with varying degrees of semantic predictability that can be used for testing speech recognition and lexical processing.MethodSentences differing in the degree of final-word predictability (predictable, neutral, and anomalous) were created with words selected to be suitable for both native and nonnative speakers of Korean. Semantic predictability was evaluated through a series of cloze tests in which native (n = 56) and nonnative (n = 19) speakers of Korean participated. This study also used a computer language model to evaluate final-word predictabilities; this is a novel approach that the current study adopted to reduce human effort in validating a large number of sentences, which produced results comparable to those of the cloze tests. In a speech recognition task, the sentences were presented to native (n = 23) and nonnative (n = 21) speakers of Korean in speech-shaped noise at two levels of noise.ResultsThe results of the speech-in-noise experiment demonstrated that the intelligibility of the sentences was similar to that of related English corpora. That is, intelligibility was significantly different depending on the semantic condition, and the sentences had the right degree of difficulty for assessing intelligibility differences depending on noise levels and language experience.ConclusionsThis corpus (1,021 sentences in total) adds to the target languages available in speech research and will allow researchers to investigate a range of issues in speech perception in Korean.Supplemental materialhttps://doi.org/10.23641/asha.24045582.
| S-EPMC10558151 | biostudies-literature
Project description:BackgroundAlthough automated dispensing robots have been implemented for medication dispensing in Japan, their effect is yet to be fully investigated. In this study, we evaluated the effect of automated dispensing robots and collaborative work with pharmacy support staff on medication dispensing.MethodsA robotic dispensing system integrating the following three components was established: (1) automated dispensing robot (Drug Station®), which is operated by pharmacy support staff, (2) automated dispensing robot for powdered medicine (Mini DimeRo®), and (3) bar-coded medication dispensing support system with personal digital assistance (Hp-PORIMS®). Subsequently, we evaluated the incidences of dispensing errors and dispensing times before and after introducing the robotic dispensing system. Dispensing errors were classified into two categories, namely prevented dispensing errors and unprevented dispensing errors. The incidence of dispensing errors was calculated as follows: incidence of dispensing errors = total number of dispensing errors/total number of medication orders in each prescription.ResultsAfter introducing the robotic dispensing system, the total incidence of prevented dispensing errors was significantly reduced (0.204% [324/158,548] to 0.044% [50/114,111], p < 0.001). The total incidence of unprevented dispensing errors was significantly reduced (0.015% [24/158,548] to 0.002% [2/114,111], p < 0.001). The number of cases of wrong strength and wrong drug, which can seriously impact a patient's health, reduced to almost zero. The median dispensing time of pharmacists per prescription was significantly reduced (from 60 to 23 s, p < 0.001).ConclusionsThe robotic dispensing system enabled the process of medication dispensing by pharmacist to be partially and safely shared with automated dispensing robots and pharmacy support staff. Therefore, clinical care for patients by pharmacists could be enhanced by ensuring quality and safety of medication.
| S-EPMC9526262 | biostudies-literature
Project description:There is controversy over the impact of electronic health record (EHR) systems on cost of care and safety. The authors studied the effects of an inpatient EHR system with computerized provider order entry on selected measures of cost of care and safety. Laboratory tests per week per hospitalization decreased from 13.9 to 11.4 (18%; p < 0.001). Radiology examinations per hospitalization decreased from 2.06 to 1.93 (6.3%; p < 0.009). Monthly transcription costs declined from $74,596 to $18,938 (74.6%; p < 0.001). Reams of copy paper ordered per month decreased from 1668 to 1224 (26.6%; p < 0.001). Medication errors per 1000 hospital days decreased from 17.9 to 15.4 (14.0%; p < 0.030), while near misses per 1000 hospital days increased from 9.0 to 12.5 (38.9%; p < 0.037), and the percentage of medication events that were medication errors decreased from 66.5% to 55.2% (p < 0.007). In this manuscript, we demonstrate that the implementation of an inpatient EHR with computerized provider order entry can result in rapid improvement in measures of cost of care and safety.
| S-EPMC3116263 | biostudies-literature