Dataset Information

Evaluating the Effectiveness of NoteAid in a Community Hospital Setting: Randomized Trial of Electronic Health Record Note Comprehension Interventions With Patients.

ABSTRACT:

Background

Interventions to define medical jargon have been shown to improve electronic health record (EHR) note comprehension among crowdsourced participants on Amazon Mechanical Turk (AMT). However, AMT participants may not be representative of the general population or patients who are most at-risk for low health literacy.

Objective

In this work, we assessed the efficacy of an intervention (NoteAid) for EHR note comprehension among participants in a community hospital setting.

Methods

Participants were recruited from Lowell General Hospital (LGH), a community hospital in Massachusetts, to take the ComprehENotes test, a web-based test of EHR note comprehension. Participants were randomly assigned to control (n=85) or intervention (n=89) groups to take the test without or with NoteAid, respectively. For comparison, we used a sample of 200 participants recruited from AMT to take the ComprehENotes test (100 in the control group and 100 in the intervention group).

Results

A total of 174 participants were recruited from LGH, and 200 participants were recruited from AMT. Participants in both intervention groups (community hospital and AMT) scored significantly higher than participants in the control groups (P<.001). The average score for the community hospital participants was significantly lower than the average score for the AMT participants (P<.001), consistent with the lower education levels in the community hospital sample. Education level had a significant effect on scores for the community hospital participants (P<.001).

Conclusions

Use of NoteAid was associated with significantly improved EHR note comprehension in both community hospital and AMT samples. Our results demonstrate the generalizability of ComprehENotes as a test of EHR note comprehension and the effectiveness of NoteAid for improving EHR note comprehension.

SUBMITTER: Lalor JP

PROVIDER: S-EPMC8160802 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundOptimization of the electronic medical record (EMR) is essential to support the clinician and to improve the quality and efficiency of patient care. The present report describes the development and implementation of a standardized template that is embedded in the EMR and is focused on a comprehensive physical examination during the evaluation of pediatric supracondylar humeral fractures. We compared the completeness of physical examinations as well as the timing of detection and documentation of neurovascular injuries before and after implementation of the template. We hypothesized that the use of a template would increase the completeness of examinations and would lead to earlier documentation of neurovascular injuries.MethodsA multidisciplinary quality-improvement task force was created to address neurovascular documentation practices for patients who underwent operative treatment of supracondylar humeral fractures. Following a series of formative and process evaluations, a standardized EMR template was implemented. Neurovascular examination documentation practices that were in use before (pre-template group, n = 224) and after (template group, n = 300) the implementation of the template were compared. Logistic regression analyses of the 2 groups were used to compare the likelihood of a complete neurovascular examination and the timing of neurovascular injury identification.ResultsThere was significant improvement in the documentation of the vascular (odds ratio [OR], 70.7; 95% confidence interval [CI], 39.5 to 126.6; p < 0.0001), motor (OR, 17.6; 95% CI, 9.5 to 32.7; p < 0.0001), and sensory (OR, 23.9; 95% CI, 12.9 to 44.4; p < 0.0001) examinations in the template group. Neurological injuries were more likely to be identified preoperatively in the template group compared with the pre-template group (OR, 6.8; 95% CI, 1.7 to 27.1; p = 0.0067).ConclusionsThe incorporation of a standardized template in the EMR improved the completeness and timing of documentation of neurological injury. Standardized EMR templates developed by a clinically driven multidisciplinary task force have the potential to improve the quality of clinical documentation and to ease communication among providers.Level of evidenceLevel III. See Instructions for Authors for a complete description of levels of evidence.

Project description:BACKGROUND:Patient portals are widely adopted in the United States and allow millions of patients access to their electronic health records (EHRs), including their EHR clinical notes. A patient's ability to understand the information in the EHR is dependent on their overall health literacy. Although many tests of health literacy exist, none specifically focuses on EHR note comprehension. OBJECTIVE:The aim of this paper was to develop an instrument to assess patients' EHR note comprehension. METHODS:We identified 6 common diseases or conditions (heart failure, diabetes, cancer, hypertension, chronic obstructive pulmonary disease, and liver failure) and selected 5 representative EHR notes for each disease or condition. One note that did not contain natural language text was removed. Questions were generated from these notes using Sentence Verification Technique and were analyzed using item response theory (IRT) to identify a set of questions that represent a good test of ability for EHR note comprehension. RESULTS:Using Sentence Verification Technique, 154 questions were generated from the 29 EHR notes initially obtained. Of these, 83 were manually selected for inclusion in the Amazon Mechanical Turk crowdsourcing tasks and 55 were ultimately retained following IRT analysis. A follow-up validation with a second Amazon Mechanical Turk task and IRT analysis confirmed that the 55 questions test a latent ability dimension for EHR note comprehension. A short test of 14 items was created along with the 55-item test. CONCLUSIONS:We developed ComprehENotes, an instrument for assessing EHR note comprehension from existing EHR notes, gathered responses using crowdsourcing, and used IRT to analyze those responses, thus resulting in a set of questions to measure EHR note comprehension. Crowdsourced responses from Amazon Mechanical Turk can be used to estimate item parameters and select a subset of items for inclusion in the test set using IRT. The final set of questions is the first test of EHR note comprehension.

Project description:Electronic health records (EHRs) have been successfully used in data science and machine learning projects. However, most of these data are collected for clinical use rather than for retrospective analysis. This means that researchers typically face many different issues when attempting to access and prepare the data for secondary use. We aimed to investigate how raw EHRs can be accessed and prepared in retrospective data science projects in a disciplined, effective, and efficient way. We report our experience and findings from a large-scale data science project analyzing routinely acquired retrospective data from the Kepler University Hospital in Linz, Austria. The project involved data collection from more than 150,000 patients over a period of 10 years. It included diverse data modalities, such as static demographic data, irregularly acquired laboratory test results, regularly sampled vital signs, and high-frequency physiological waveform signals. Raw medical data can be corrupted in many unexpected ways that demand thorough manual inspection and highly individualized data cleaning solutions. We present a general data preparation workflow, which was shaped in the course of our project and consists of the following 7 steps: obtain a rough overview of the available EHR data, define clinically meaningful labels for supervised learning, extract relevant data from the hospital's data warehouses, match data extracted from different sources, deidentify them, detect errors and inconsistencies therein through a careful exploratory analysis, and implement a suitable data processing pipeline in actual code. Only few of the data preparation issues encountered in our project were addressed by generic medical data preprocessing tools that have been proposed recently. Instead, highly individualized solutions for the specific data used in one's own research seem inevitable. We believe that the proposed workflow can serve as a guidance for practitioners, helping them to identify and address potential problems early and avoid some common pitfalls.