Dataset Information

Replication of Real-World Evidence in Oncology Using Electronic Health Record Data Extracted by Machine Learning.

ABSTRACT: Meaningful real-world evidence (RWE) generation requires unstructured data found in electronic health records (EHRs) which are often missing from administrative claims; however, obtaining relevant data from unstructured EHR sources is resource-intensive. In response, researchers are using natural language processing (NLP) with machine learning (ML) techniques (i.e., ML extraction) to extract real-world data (RWD) at scale. This study assessed the quality and fitness-for-use of EHR-derived oncology data curated using NLP with ML as compared to the reference standard of expert abstraction. Using a sample of 186,313 patients with lung cancer from a nationwide EHR-derived de-identified database, we performed a series of replication analyses demonstrating some common analyses conducted in retrospective observational research with complex EHR-derived data to generate evidence. Eligible patients were selected into biomarker- and treatment-defined cohorts, first with expert-abstracted then with ML-extracted data. We utilized the biomarker- and treatment-defined cohorts to perform analyses related to biomarker-associated survival and treatment comparative effectiveness, respectively. Across all analyses, the results differed by less than 8% between the data curation methods, and similar conclusions were reached. These results highlight that high-performance ML-extracted variables trained on expert-abstracted data can achieve similar results as when using abstracted data, unlocking the ability to perform oncology research at scale.

SUBMITTER: Benedum CM

PROVIDER: S-EPMC10046618 | biostudies-literature | 2023 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Replication of Real-World Evidence in Oncology Using Electronic Health Record Data Extracted by Machine Learning.

Benedum Corey M CM Sondhi Arjun A Fidyk Erin E Cohen Aaron B AB Nemeth Sheila S Adamson Blythe B Estévez Melissa M Bozkurt Selen S

Cancers 20230320 6

Meaningful real-world evidence (RWE) generation requires unstructured data found in electronic health records (EHRs) which are often missing from administrative claims; however, obtaining relevant data from unstructured EHR sources is resource-intensive. In response, researchers are using natural language processing (NLP) with machine learning (ML) techniques (i.e., <i>ML extraction</i>) to extract real-world data (RWD) at scale. This study assessed the quality and fitness-for-use of EHR-derived ...[more]

PMID: 36980739

Similar Datasets

Project description:BackgroundPre-travel consultation and chemoprophylaxis measures for malaria are a key component in the prevention of imported malaria in travelers. In this study we report a predictive tool for assessing personalized malaria risk in travelers based on the analysis of electronic medical records from travel consultations. The tool aims to guide physicians in the recommendation of appropriate prophylaxis prior to their trip. We also provide best-practice recommendations for pre-processing noisy and highly sparse real world evidence data.MethodsWe leveraged a large EMR dataset, containing demographic information about travelers and their destination. The data has been previously preprocessed using various strategies to handle missing and unbalanced data. We compared multiple machine learning approaches to assess the risk of malaria acquisition in travelers during their travels. Additionally, a feature importance analysis was performed using SHAP (SHapley Additive Explanations) values to identify patterns associated with malaria risk.ResultsOur study revealed that our XGB models achieved high predictive capacity (AUC >0.80). The most significant features predicting malaria infection during travel included travel destinations with low malaria risk, vaccination history, number of countries visited, age, and trip duration. Remarkably, we were able to obtain a reduced model with only five features. When comparing this model with a population of travelers recommended for malaria chemoprophylaxis, we observed that it was deemed necessary in only 40% of these travelers. This suggests that 60% received chemoprophylaxis despite having a low personalized risk of malaria.ConclusionWe have developed an algorithmic tool that utilizes a concise survey to generate a personalized travel risk assessment, effectively minimizing the prescription of unnecessary malaria chemoprophylaxis. Through the identification of patterns linked to predictions, our model significantly enhances the efficacy of pre-travel consultations.

Project description:ObjectiveElectronic health records (EHR) offer medical and pharmacogenomics research unprecedented opportunities to identify and classify patients at risk. EHRs are collections of highly inter-dependent records that include biological, anatomical, physiological, and behavioral observations. They comprise a patient's clinical phenome, where each patient has thousands of date-stamped records distributed across many relational tables. Development of EHR computer-based phenotyping algorithms require time and medical insight from clinical experts, who most often can only review a small patient subset representative of the total EHR records, to identify phenotype features. In this research we evaluate whether relational machine learning (ML) using inductive logic programming (ILP) can contribute to addressing these issues as a viable approach for EHR-based phenotyping.MethodsTwo relational learning ILP approaches and three well-known WEKA (Waikato Environment for Knowledge Analysis) implementations of non-relational approaches (PART, J48, and JRIP) were used to develop models for nine phenotypes. International Classification of Diseases, Ninth Revision (ICD-9) coded EHR data were used to select training cohorts for the development of each phenotypic model. Accuracy, precision, recall, F-Measure, and Area Under the Receiver Operating Characteristic (AUROC) curve statistics were measured for each phenotypic model based on independent manually verified test cohorts. A two-sided binomial distribution test (sign test) compared the five ML approaches across phenotypes for statistical significance.ResultsWe developed an approach to automatically label training examples using ICD-9 diagnosis codes for the ML approaches being evaluated. Nine phenotypic models for each ML approach were evaluated, resulting in better overall model performance in AUROC using ILP when compared to PART (p=0.039), J48 (p=0.003) and JRIP (p=0.003).DiscussionILP has the potential to improve phenotyping by independently delivering clinically expert interpretable rules for phenotype definitions, or intuitive phenotypes to assist experts.ConclusionRelational learning using ILP offers a viable approach to EHR-driven phenotyping.

Project description:BackgroundPoor functional status is a key marker of morbidity, yet is not routinely captured in clinical encounters. We developed and evaluated the accuracy of a machine learning algorithm that leveraged electronic health record (EHR) data to provide a scalable process for identification of functional impairment.MethodsWe identified a cohort of patients with an electronically captured screening measure of functional status (Older Americans Resources and Services ADL/IADL) between 2018 and 2020 (N = 6484). Patients were classified using unsupervised learning K means and t-distributed Stochastic Neighbor Embedding into normal function (NF), mild to moderate functional impairment (MFI), and severe functional impairment (SFI) states. Using 11 EHR clinical variable domains (832 variable input features), we trained an Extreme Gradient Boosting supervised machine learning algorithm to distinguish functional status states, and measured prediction accuracies. Data were randomly split into training (80%) and test (20%) sets. The SHapley Additive Explanations (SHAP) feature importance analysis was used to list the EHR features in rank order of their contribution to the outcome.ResultsMedian age was 75.3 years, 62% female, 60% White. Patients were classified as 53% NF (n = 3453), 30% MFI (n = 1947), and 17% SFI (n = 1084). Summary of model performance for identifying functional status state (NF, MFI, SFI) was AUROC (area under the receiving operating characteristic curve) 0.92, 0.89, and 0.87, respectively. Age, falls, hospitalization, home health use, labs (e.g., albumin), comorbidities (e.g., dementia, heart failure, chronic kidney disease, chronic pain), and social determinants of health (e.g., alcohol use) were highly ranked features in predicting functional status states.ConclusionA machine learning algorithm run on EHR clinical data has potential utility for differentiating functional status in the clinical setting. Through further validation and refinement, such algorithms can complement traditional screening methods and result in a population-based strategy for identifying patients with poor functional status who need additional health resources.

Dataset Information

Replication of Real-World Evidence in Oncology Using Electronic Health Record Data Extracted by Machine Learning.

Publications

Replication of Real-World Evidence in Oncology Using Electronic Health Record Data Extracted by Machine Learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets