Ambiguity in medical concept normalization: An analysis of types and coverage in electronic health record datasets.
Ontology highlight
ABSTRACT: Objectives
Normalizing mentions of medical concepts to standardized vocabularies is a fundamental component of clinical text analysis. Ambiguity-words or phrases that may refer to different concepts-has been extensively researched as part of information extraction from biomedical literature, but less is known about the types and frequency of ambiguity in clinical text. This study characterizes the distribution and distinct types of ambiguity exhibited by benchmark clinical concept normalization datasets, in order to identify directions for advancing medical concept normalization research.Materials and methods
We identified ambiguous strings in datasets derived from the 2 available clinical corpora for concept normalization and categorized the distinct types of ambiguity they exhibited. We then compared observed string ambiguity in the datasets with potential ambiguity in the Unified Medical Language System (UMLS) to assess how representative available datasets are of ambiguity in clinical language.Results
We found that <15% of strings were ambiguous within the datasets, while over 50% were ambiguous in the UMLS, indicating only partial coverage of clinical ambiguity. The percentage of strings in common between any pair of datasets ranged from 2% to only 36%; of these, 40% were annotated with different sets of concepts, severely limiting generalization. Finally, we observed 12 distinct types of ambiguity, distributed unequally across the available datasets, reflecting diverse linguistic and medical phenomena.Discussion
Existing datasets are not sufficient to cover the diversity of clinical concept ambiguity, limiting both training and evaluation of normalization methods for clinical text. Additionally, the UMLS offers important semantic information for building and evaluating normalization methods.Conclusions
Our findings identify 3 opportunities for concept normalization research, including a need for ambiguity-specific clinical datasets and leveraging the rich semantics of the UMLS in new methods and evaluation measures for normalization.
SUBMITTER: Newman-Griffis D
PROVIDER: S-EPMC7936394 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Introduction:The adoption of electronic health records (EHRs) presents new challenges for information gathering, documentation, and patient care. Medical students spend a significant amount of time using the EHR during their clerkship experiences and will continue to do so as they progress to residency. However, formal training varies between institutions and leaves gaps in data-gathering skills, documentation skills, and order entry-these three skills formed the basis for our learning objectives. We designed a workshop using a simulated EHR to teach these skills. Methods:We offered the workshop during a class-wide transition-to-internship course for senior medical students. After a brief didactic, students worked in small groups using a simulated EHR to complete cases addressing each of the three learning objectives. Faculty facilitators assisted groups and then guided a large-group discussion of the challenges encountered during the cases. Results:Twenty-five senior medical students attended the first optional workshop. Of these students, 66.7% recommended that the workshop continue to be included in the curriculum. Comments from those who recommended otherwise suggested that many of them would recommend the workshop if it used our local EHR (Epic). Correct answers to the factual questions increased for most questions between the pretest and the posttest. Confidence to perform all skills targeted in the learning objectives increased between the pretest and the posttest. Discussion:This EHR workshop was well received by senior medical students and increased confidence in EHR skills, including data gathering, documentation, and handling unsolicited information with a plan including order entry.
| S-EPMC6946580 | biostudies-literature
Project description:We consider the task of Medical Concept Normalization (MCN) which aims to map informal medical phrases such as "loosing weight" to formal medical concepts, such as "Weight loss". Deep learning models have shown high performance across various MCN datasets containing small number of target concepts along with adequate number of training examples per concept. However, scaling these models to millions of medical concepts entails the creation of much larger datasets which is cost and effort intensive. Recent works have shown that training MCN models using automatically labeled examples extracted from medical knowledge bases partially alleviates this problem. We extend this idea by computationally creating a distant dataset from patient discussion forums. We extract informal medical phrases and medical concepts from these forums using a synthetically trained classifier and an off-the-shelf medical entity linker respectively. We use pretrained sentence encoding models to find the k-nearest phrases corresponding to each medical concept. These mappings are used in combination with the examples obtained from medical knowledge bases to train an MCN model. Our approach outperforms the previous state-of-the-art by 15.9% and 17.1% classification accuracy across two datasets while avoiding manual labeling.
| S-EPMC7415240 | biostudies-literature
Project description:IntroductionElectronic health records (EHR) are linked together to examine disease history and to undertake research into the causes and outcomes of disease. However, the process of constructing algorithms for phenotyping (e.g., identifying disease characteristics) or health characteristics (e.g., smoker) is very time consuming and resource costly. In addition, results can vary greatly between researchers. Reusing or building on algorithms that others have created is a compelling solution to these problems. However, sharing algorithms is not a common practice and many published studies do not detail the clinical code lists used by the researchers in the disease/characteristic definition. To address these challenges, a number of centres across the world have developed health data portals which contain concept libraries (e.g., algorithms for defining concepts such as disease and characteristics) in order to facilitate disease phenotyping and health studies.ObjectivesThis study aims to review the literature of existing concept libraries, examine their utilities, identify the current gaps, and suggest future developments.MethodsThe five-stage framework of Arksey and O'Malley was used for the literature search. This approach included defining the research questions, identifying relevant studies through literature review, selecting eligible studies, charting and extracting data, and summarising and reporting the findings.ResultsThis review identified seven publicly accessible Electronic Health data concept libraries which were developed in different countries including UK, USA, and Canada. The concept libraries (n = 7) investigated were either general libraries that hold phenotypes of multiple specialties (n = 4) or specialized libraries that manage only certain specialities such as rare diseases (n = 3). There were some clear differences between the general libraries such as archiving data from different electronic sources, and using a range of different types of coding systems. However, they share some clear similarities such as enabling users to upload their own code lists, and allowing users to use/download the publicly accessible code. In addition, there were some differences between the specialized libraries such as difference in ability to search, and if it was possible to use different searching queries such as simple or complex searches. Conversely, there were some similarities between the specialized libraries such as enabling users to upload their own concepts into the libraries and to show where they were published, which facilitates assessing the validity of the concepts. All the specialized libraries aimed to encourage the reuse of research methods such as lists of clinical code and/or metadata.ConclusionThe seven libraries identified have been developed independently and appear to replicate similar concepts but in different ways. Collaboration between similar libraries would greatly facilitate the use of these libraries for the user. The process of building code lists takes time and effort. Access to existing code lists increases consistency and accuracy of definitions across studies. Concept library developers should collaborate with each other to raise awareness of their existence and of their various functions, which could increase users' contributions to those libraries and promote their wide-ranging adoption.
| S-EPMC8210840 | biostudies-literature
Project description:BackgroundElectronic Health Records (EHRs) are now widely used to create a single, shared, and reliable source of patient data throughout healthcare organizations. However, health professionals continue to experience mismatches between their working practices and what the EHR allows or directs them to do. Health professionals adopt working practices other than those imposed by the EHR to overcome such mismatches, known as workarounds. Our study aims to inductively develop a typology of enduring EHR workarounds and explore their consequences by answering the question: What types of EHR workarounds persist, and what are the user-perceived consequences?MethodsThis single case study was conducted within the Internal Medicine department of a Dutch hospital that had implemented an organization-wide, commercial EHR system over two years ago. Data were collected through observations of six EHR users (see Additional file 1, observation scheme) and 17 semi-structured interviews with physicians, nurses, administrators, and EHR support staff members. Documents were analysed to contextualize these data (see Additional file 2, interview protocol).ResultsThrough a qualitative analysis, 11 workarounds were identified, predominantly performed by physicians. These workarounds are categorized into three types either performed while working with the system (in-system workflow sequence workarounds and in-system data entry workarounds) or bypassing the system (out-system workarounds). While these workarounds seem to offer short-term benefits for the performer, they often create threats for the user, the patient, the overall healthcare organization, and the system.ConclusionThis study increases our understanding of the enduring phenomenon of working around Electronic Health Records by presenting a typology of those workarounds that persist after adoption and by reflecting on the user-perceived risks and benefits. The typology helps EHR users and their managers to identify enduring types of workarounds and differentiate between the harmful and less harmful ones. This distinction can inform their decisions to discourage or obviate the need for certain workarounds, while legitimating others.
| S-EPMC8186102 | biostudies-literature
Project description:Background and study aims
The Federal Ministry of Education and Research (BMBF) has funded the development of a patient controlled “Personal Electronic Health Record” (PEPA) as part of their current INFOPAT project. PEPA allows for the exchange of medical data between all interested groups, including the patients thenselves. A PEPA contains medical reports and findings including, for example, x-rays, CT scans and MRIs. The patients can safeguard their information and access all content. The aim of this study is to test the PEPA-prototype for cancer patients being treated at the National Center for Tumor Diseases (NCT) in Heidelberg.
Who can participate?
Adult colorectal cancer patients being treated at the National Center for Tumor Diseases (NCT) in Heidelberg.
What does the study involve?
There are two part to this study. The first part involves focus groups, in which a group of people are asked their opinion and attitudes towards, for example, an idea, product or service. Here, focus groups are set up that involve the patients, doctors and other health professionals to discuss current problems with the coordination of care offered to cancer patients, as well as discuss what people want to get from using PEPA (i.e. user requirements). The PEPA-prototype is then developed taking into account the conclusions from the focus groups. In the second part of the study, patients are encouraged to use the PEPA-prototype to prepare and follow up meetings with their physicians.
| 2424890 | ecrin-mdr-crc
Project description:Usability testing is increasingly being recognized as a way to increase the usability and safety of health information technology (HIT). Medical simulation centers can serve as testing environments for HIT usability studies. We integrated the quality assurance version of our emergency department (ED) electronic health record (EHR) into our medical simulation center and piloted a clinical care scenario in which emergency medicine resident physicians evaluated a simulated ED patient and documented electronically using the ED EHR. Meticulous planning and close collaboration with expert simulation staff was important for designing test scenarios, pilot testing, and running the sessions. Similarly, working with information systems teams was important for integration of the EHR. Electronic tools are needed to facilitate entry of fictitious clinical results while the simulation scenario is unfolding. EHRs can be successfully integrated into existing simulation centers, which may provide realistic environments for usability testing, training, and evaluation of human-computer interactions.
| S-EPMC3994869 | biostudies-other
Project description:OBJECTIVES:The goal of this study was to use machine learning to more accurately predict survival after echocardiography. BACKGROUND:Predicting patient outcomes (e.g., survival) following echocardiography is primarily based on ejection fraction (EF) and comorbidities. However, there may be significant predictive information within additional echocardiography-derived measurements combined with clinical electronic health record data. METHODS:Mortality was studied in 171,510 unselected patients who underwent 331,317 echocardiograms in a large regional health system. The authors investigated the predictive performance of nonlinear machine learning models compared with that of linear logistic regression models using 3 different inputs: 1) clinical variables, including 90 cardiovascular-relevant International Classification of Diseases, Tenth Revision, codes, and age, sex, height, weight, heart rate, blood pressures, low-density lipoprotein, high-density lipoprotein, and smoking; 2) clinical variables plus physician-reported EF; and 3) clinical variables and EF, plus 57 additional echocardiographic measurements. Missing data were imputed with a multivariate imputation by using a chained equations algorithm (MICE). The authors compared models versus each other and baseline clinical scoring systems by using a mean area under the curve (AUC) over 10 cross-validation folds and across 10 survival durations (6 to 60 months). RESULTS:Machine learning models achieved significantly higher prediction accuracy (all AUC >0.82) over common clinical risk scores (AUC = 0.61 to 0.79), with the nonlinear random forest models outperforming logistic regression (p < 0.01). The random forest model including all echocardiographic measurements yielded the highest prediction accuracy (p < 0.01 across all models and survival durations). Only 10 variables were needed to achieve 96% of the maximum prediction accuracy, with 6 of these variables being derived from echocardiography. Tricuspid regurgitation velocity was more predictive of survival than LVEF. In a subset of studies with complete data for the top 10 variables, multivariate imputation by chained equations yielded slightly reduced predictive accuracies (difference in AUC of 0.003) compared with the original data. CONCLUSIONS:Machine learning can fully utilize large combinations of disparate input variables to predict survival after echocardiography with superior accuracy.
| S-EPMC6286869 | biostudies-literature
Project description:BACKGROUND:Children with health insurance have increased access to healthcare and receive higher quality care. However, despite recent initiatives expanding children's coverage, many remain uninsured. New technologies present opportunities for helping clinics provide enrollment support for patients. We developed and tested electronic health record (EHR)-based tools to help clinics provide children's insurance assistance. METHODS:We used mixed methods to understand tool adoption, and to assess impact of tool use on insurance coverage, healthcare utilization, and receipt of recommended care. We conducted intent-to-treat (ITT) analyses comparing pediatric patients in 4 intervention clinics (n?=?15,024) to those at 4 matched control clinics (n?=?12,227). We conducted effect-of-treatment-on-the-treated (ETOT) analyses comparing intervention clinic patients with tool use (n?=?2240) to intervention clinic patients without tool use (n?=?12,784). RESULTS:Tools were used for only 15% of eligible patients. Qualitative data indicated that tool adoption was limited by: (1) concurrent initiatives that duplicated the work associated with the tools, and (2) inability to obtain accurate insurance coverage data and end dates. The ITT analyses showed that intervention clinic patients had higher odds of gaining insurance coverage (adjusted odds ratio [aOR]?=?1.32, 95% confidence interval [95%CI] 1.14-1.51) and lower odds of losing coverage (aOR?=?0.77, 95%CI 0.68-0.88), compared to control clinic patients. Similarly, ETOT findings showed that intervention clinic patients with tool use had higher odds of gaining insurance (aOR?=?1.83, 95%CI 1.64-2.04) and lower odds of losing coverage (aOR?=?0.70, 95%CI 0.53-0.91), compared to patients without tool use. The ETOT analyses also showed higher rates of receipt of return visits, well-child visits, and several immunizations among patients for whom the tools were used. CONCLUSIONS:This pragmatic trial, the first to evaluate EHR-based insurance assistance tools, suggests that it is feasible to create and implement tools that help clinics provide insurance enrollment support to pediatric patients. While ITT findings were limited by low rates of tool use, ITT and ETOT findings suggest tool use was associated with better odds of gaining and keeping coverage. Further, ETOT findings suggest that use of such tools may positively impact healthcare utilization and quality of pediatric care. TRIAL REGISTRATION:ClinicalTrials.gov, NCT02298361 ; retrospectively registered on November 5, 2014.
| S-EPMC5946500 | biostudies-literature
Project description:BACKGROUND:The increased use of electronic medical records (EMRs) in Canadian primary health care practice has resulted in an expansion of the availability of EMR data. Potential users of these data need to understand their quality in relation to the uses to which they are applied. Herein, we propose a basic model for assessing primary health care EMR data quality, comprising a set of data quality measures within four domains. We describe the process of developing and testing this set of measures, share the results of applying these measures in three EMR-derived datasets, and discuss what this reveals about the measures and EMR data quality. The model is offered as a starting point from which data users can refine their own approach, based on their own needs. METHODS:Using an iterative process, measures of EMR data quality were created within four domains: comparability; completeness; correctness; and currency. We used a series of process steps to develop the measures. The measures were then operationalized, and tested within three datasets created from different EMR software products. RESULTS:A set of eleven final measures were created. We were not able to calculate results for several measures in one dataset because of the way the data were collected in that specific EMR. Overall, we found variability in the results of testing the measures (e.g. sensitivity values were highest for diabetes, and lowest for obesity), among datasets (e.g. recording of height), and by patient age and sex (e.g. recording of blood pressure, height and weight). CONCLUSIONS:This paper proposes a basic model for assessing primary health care EMR data quality. We developed and tested multiple measures of data quality, within four domains, in three different EMR-derived primary health care datasets. The results of testing these measures indicated that not all measures could be utilized in all datasets, and illustrated variability in data quality. This is one step forward in creating a standard set of measures of data quality. Nonetheless, each project has unique challenges, and therefore requires its own data quality assessment before proceeding.
| S-EPMC6373085 | biostudies-other
Project description:Given the ubiquity of electronic health records (EHR), health administrators should be formally trained on the use and evaluation of EHR data for common operational tasks and managerial decision-making. A teaching electronic medical record (tEMR) is a fully operational electronic medical record that uses de-identified electronic patient data and provides a framework for students to familiarize themselves with the data, features, and functionality of an EHR. Although purported to be of value in health administration programs, specific benefits of using a tEMR in health administration education is unknown. We sought to examine Master of Health Administration (MHA) students' perceptions of the use, challenges, and benefits of a tEMR. We analyzed qualitative data collected from a focus group session with students who were exposed to the tEMR during a semester MHA course. We also administered pre- and post-survey questions on students' self-efficacy and perceptions of the ease of use, usefulness, and intention to use health care data analysis in their future jobs. We found several MHA students valued their exposure to the tEMR, as this provided them a realistic environment to explore de-identified patient data. Scores for students' perceived ease of using healthcare data analysis in their future job significantly increased following use of the tEMR (pre-test score M=3.31, SD=0.21; post-test score M=3.71, SD=0.18; p=0.01). Student exposure and use of a tEMR may positively affect perceptions of using EHR data for strategic and managerial tasks typical of health administrators.
| S-EPMC9721109 | biostudies-literature