Dataset Information

ComprehENotes, an Instrument to Assess Patient Reading Comprehension of Electronic Health Record Notes: Development and Validation.

ABSTRACT: BACKGROUND:Patient portals are widely adopted in the United States and allow millions of patients access to their electronic health records (EHRs), including their EHR clinical notes. A patient's ability to understand the information in the EHR is dependent on their overall health literacy. Although many tests of health literacy exist, none specifically focuses on EHR note comprehension. OBJECTIVE:The aim of this paper was to develop an instrument to assess patients' EHR note comprehension. METHODS:We identified 6 common diseases or conditions (heart failure, diabetes, cancer, hypertension, chronic obstructive pulmonary disease, and liver failure) and selected 5 representative EHR notes for each disease or condition. One note that did not contain natural language text was removed. Questions were generated from these notes using Sentence Verification Technique and were analyzed using item response theory (IRT) to identify a set of questions that represent a good test of ability for EHR note comprehension. RESULTS:Using Sentence Verification Technique, 154 questions were generated from the 29 EHR notes initially obtained. Of these, 83 were manually selected for inclusion in the Amazon Mechanical Turk crowdsourcing tasks and 55 were ultimately retained following IRT analysis. A follow-up validation with a second Amazon Mechanical Turk task and IRT analysis confirmed that the 55 questions test a latent ability dimension for EHR note comprehension. A short test of 14 items was created along with the 55-item test. CONCLUSIONS:We developed ComprehENotes, an instrument for assessing EHR note comprehension from existing EHR notes, gathered responses using crowdsourcing, and used IRT to analyze those responses, thus resulting in a set of questions to measure EHR note comprehension. Crowdsourced responses from Amazon Mechanical Turk can be used to estimate item parameters and select a subset of items for inclusion in the test set using IRT. The final set of questions is the first test of EHR note comprehension.

SUBMITTER: Lalor JP

PROVIDER: S-EPMC5943623 | biostudies-literature | 2018 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

ComprehENotes, an Instrument to Assess Patient Reading Comprehension of Electronic Health Record Notes: Development and Validation.

Lalor John P JP Wu Hao H Chen Li L Mazor Kathleen M KM Yu Hong H

Journal of medical Internet research 20180425 4

<h4>Background</h4>Patient portals are widely adopted in the United States and allow millions of patients access to their electronic health records (EHRs), including their EHR clinical notes. A patient's ability to understand the information in the EHR is dependent on their overall health literacy. Although many tests of health literacy exist, none specifically focuses on EHR note comprehension.<h4>Objective</h4>The aim of this paper was to develop an instrument to assess patients' EHR note comp ...[more]

PMID: 29695372

Similar Datasets

Project description:Medical terms are a major obstacle for patients to comprehend their electronic health record (EHR) notes. Clinical natural language processing (NLP) systems that link EHR terms to lay terms or definitions allow patients to easily access helpful information when reading through their EHR notes, and have shown to improve patient EHR comprehension. However, high-quality lay language resources for EHR terms are very limited in the public domain. Because expanding and curating such a resource is a costly process, it is beneficial and even necessary to identify terms important for patient EHR comprehension first.We aimed to develop an NLP system, called adapted distant supervision (ADS), to rank candidate terms mined from EHR corpora. We will give EHR terms ranked as high by ADS a higher priority for lay language annotation-that is, creating lay definitions for these terms.Adapted distant supervision uses distant supervision from consumer health vocabulary and transfer learning to adapt itself to solve the problem of ranking EHR terms in the target domain. We investigated 2 state-of-the-art transfer learning algorithms (ie, feature space augmentation and supervised distant supervision) and designed 5 types of learning features, including distributed word representations learned from large EHR data for ADS. For evaluating ADS, we asked domain experts to annotate 6038 candidate terms as important or nonimportant for EHR comprehension. We then randomly divided these data into the target-domain training data (1000 examples) and the evaluation data (5038 examples). We compared ADS with 2 strong baselines, including standard supervised learning, on the evaluation data.The ADS system using feature space augmentation achieved the best average precision, 0.850, on the evaluation set when using 1000 target-domain training examples. The ADS system using supervised distant supervision achieved the best average precision, 0.819, on the evaluation set when using only 100 target-domain training examples. The 2 ADS systems both performed significantly better than the baseline systems (P<.001 for all measures and all conditions). Using a rich set of learning features contributed to ADS's performance substantially.ADS can effectively rank terms mined from EHRs. Transfer learning improved ADS's performance even with a small number of target-domain training examples. EHR terms prioritized by ADS were used to expand a lay language resource that supports patient EHR comprehension. The top 10,000 EHR terms ranked by ADS are available upon request.

Project description:ImportanceAs health information transparency increases, patients more often seek their health data. More than 44 million patients in the US can now readily access their ambulatory visit notes online, and the practice is increasing abroad. Few studies have assessed documentation errors that patients identify in their notes and how these may inform patient engagement and safety strategies.ObjectiveTo assess the frequency and types of errors identified by patients who read open ambulatory visit notes.Design, setting, and participantsIn this survey study, a total of 136?815 patients at 3 US health care organizations with open notes, including 79 academic and community ambulatory care practices, received invitations to an online survey from June 5 to October 20, 2017. Patients who had at least 1 ambulatory note and had logged onto the portal at least once in the past 12 months were included. Data analysis was performed from July 3, 2018, to April 27, 2020.ExposuresAccess to ambulatory care open notes through patient portals for up to 7 years (2010-2017).Main outcomes and measuresProportion of patients reporting a mistake and how serious they perceived the mistake to be, factors associated with finding errors characterized by patients as serious, and categories of patient-reported errors.ResultsOf 136?815 patients who received survey invitations, 29?656 (21.7%) responded and 22?889 patients (mean [SD] age, 55.16 [15.96] years; 14 447 [63.1%] female; 18 301 [80.0%] white) read 1 or more notes in the past 12 months and completed error questions. Of these patients, 4830 (21.1%) reported a perceived mistake and 2043 (42.3%) reported that the mistake was serious (somewhat serious: 1563 [32.4%]; very serious: 480 [9.9%]). In multivariable analysis, female patients (relative risk [RR], 1.79; 95% CI, 1.72-1.85), more educated patients (RR, 1.38; 95% CI, 1.29-1.48), sicker patients (RR, 1.89; 95% CI, 1.84-1.94), those aged 45 to 64 years (RR, 2.23; 95% CI, 2.06-2.42), those 65 years or older (RR, 2.00; 95% CI, 1.73-2.32), and those who read more than 1 note (2-3 notes: RR, 1.82; 95% CI, 1.34-2.47; ?4 notes: RR, 3.09; 95% CI, 2.02-4.73) were more likely to report a mistake that they found to be serious compared with their reference groups. After categorization of patient-reported very serious mistakes, those specifically mentioning the word diagnosis or describing a specific error in current or past diagnoses were most common (98 of 356 [27.5%]), followed by inaccurate medical history (85 of 356 [23.9%]), medications or allergies (50 of 356 [14.0%]), and tests, procedures, or results (30 of 356 [8.4%]). A total of 23 (6.5%) reflected notes reportedly written on the wrong patient. Of 433 very serious errors, 255 (58.9%) included at least 1 perceived error potentially associated with the diagnostic process (eg, history, physical examination, tests, referrals, and communication).Conclusions and relevanceIn this study, patients who read ambulatory notes online perceived mistakes, a substantial proportion of which they found to be serious. Older and sicker patients were twice as likely to report a serious error compared with younger and healthier patients, indicating important safety and quality implications. Sharing notes with patients may help engage them to improve record accuracy and health care safety together with practitioners.

Project description:BackgroundIt is assumed that the implementation of health information technology introduces new vulnerabilities within a complex sociotechnical health care system, but no international consensus exists on a standardized format for enhancing the collection, analysis, and interpretation of technology-induced errors.ObjectiveThis study aims to develop a classification for patient safety incident reporting associated with the use of mature electronic health records (EHRs). It also aims to validate the classification by using a data set of incidents during a 6-month period immediately after the implementation of a new EHR system.MethodsThe starting point of the classification development was the Finnish Technology-Induced Error Risk Assessment Scale tool, based on research on commonly recognized error types. A multiprofessional research team used iterative tests on consensus building to develop a classification system. The final classification, with preliminary descriptions of classes, was validated by applying it to analyze EHR-related error incidents (n=428) during the implementation phase of a new EHR system and also to evaluate this classification's characteristics and applicability for reporting incidents. Interrater agreement was applied.ResultsThe number of EHR-related patient safety incidents during the implementation period (n=501) was five-fold when compared with the preimplementation period (n=82). The literature identified new error types that were added to the emerging classification. Error types were adapted iteratively after several test rounds to develop a classification for reporting patient safety incidents in the clinical use of a high-maturity EHR system. Of the 427 classified patient safety incidents, interface problems accounted for 96 (22.5%) incident reports, usability problems for 73 (17.1%), documentation problems for 60 (14.1%), and clinical workflow problems for 33 (7.7%). Altogether, 20.8% (89/427) of reports were related to medication section problems, and downtime problems were rare (n=8). During the classification work, 14.8% (74/501) of reports of the original sample were rejected because of insufficient information, even though the reports were deemed to be related to EHRs. The interrater agreement during the blinded review was 97.7%.ConclusionsThis study presents a new classification for EHR-related patient safety incidents applicable to mature EHRs. The number of EHR-related patient safety incidents during the implementation period may reflect patient safety challenges during the implementation of a new type of high-maturity EHR system. The results indicate that the types of errors previously identified in the literature change with the EHR development cycle.

Project description:BackgroundAdvance care planning allows patients to articulate their future care preferences should they no longer be able to make decisions on their own. Early advance care planning in outpatient settings provides benefits such as less aggressive care and fewer hospitalizations, yet it is underutilized due to barriers such as provider time constraints and communication complexity. Novel methods, such as patient portals, provide a unique opportunity to conduct advance care planning previsit planning for outpatient care. This follow-up to our pilot study aimed to conduct pragmatic testing of a novel electronic health record-tethered framework and its effects on advance care planning delivery in a real-world primary care setting.ObjectiveOur intervention tested a previsit advance care planning workflow centered around a framework sent via secure electronic health record-linked patient portal in a real-world clinical setting. The primary objective of this study was to determine its impact on frequency and quality of advance care planning documentation.MethodsWe conducted a pragmatic trial including 2 sister clinical sites, one site implementing the intervention and the other continuing standard care. A total of 419 patients aged between 50 and 93 years with active portal accounts received intervention (n=200) or standard care (n=219). Chart review analyzed the presence of advance care planning and its quality and was graded with previously established scoring criteria based on advance care planning best practice guidelines from multiple nations.ResultsA total of 19.5% (39/200) of patients who received previsit planning responded to the framework. We found that the intervention site had statistically significant improvement in new advance care planning documentation rates (P<.01) and quality (P<.01) among all eligible patients. Advance care planning documentation rates increased by 105% (19/39 to 39/39) and quality improved among all patients who engaged in the previsit planning framework (n=39). Among eligible patients aged between 50 and 60 years at the intervention site, advance care planning documentation rates increased by 37% (27/96 to 37/96). Advance care planning documentation rates increased 34% among high users (27/67 to 36/67).ConclusionsAdvance care planning previsit planning using a secure electronic health record-supported patient portal framework yielded improvement in the presence of advance care planning documentation, with highest improvement in active patient portal users and patients aged between 50 and 60 years. Targeted previsit patient portal advance care planning delivery in these populations can potentially improve the quality of care in these populations.

Dataset Information

ComprehENotes, an Instrument to Assess Patient Reading Comprehension of Electronic Health Record Notes: Development and Validation.

Publications

ComprehENotes, an Instrument to Assess Patient Reading Comprehension of Electronic Health Record Notes: Development and Validation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets