Dataset Information

Comparing automated vs. manual data collection for COVID-specific medications from electronic health records.

ABSTRACT:

Introduction

Data extraction from electronic health record (EHR) systems occurs through manual abstraction, automated extraction, or a combination of both. While each method has its strengths and weaknesses, both are necessary for retrospective observational research as well as sudden clinical events, like the COVID-19 pandemic. Assessing the strengths, weaknesses, and potentials of these methods is important to continue to understand optimal approaches to extracting clinical data. We set out to assess automated and manual techniques for collecting medication use data in patients with COVID-19 to inform future observational studies that extract data from the electronic health record (EHR).

Materials and methods

For 4,123 COVID-positive patients hospitalized and/or seen in the emergency department at an academic medical center between 03/03/2020 and 05/15/2020, we compared medication use data of 25 medications or drug classes collected through manual abstraction and automated extraction from the EHR. Quantitatively, we assessed concordance using Cohen's kappa to measure interrater reliability, and qualitatively, we audited observed discrepancies to determine causes of inconsistencies.

Results

For the 16 inpatient medications, 11 (69%) demonstrated moderate or better agreement; 7 of those demonstrated strong or almost perfect agreement. For 9 outpatient medications, 3 (33%) demonstrated moderate agreement, but none achieved strong or almost perfect agreement. We audited 12% of all discrepancies (716/5,790) and, in those audited, observed three principal categories of error: human error in manual abstraction (26%), errors in the extract-transform-load (ETL) or mapping of the automated extraction (41%), and abstraction-query mismatch (33%).

Conclusion

Our findings suggest many inpatient medications can be collected reliably through automated extraction, especially when abstraction instructions are designed with data architecture in mind. We discuss quality issues, concerns, and improvements for institutions to consider when crafting an approach. During crises, institutions must decide how to allocate limited resources. We show that automated extraction of medications is feasible and make recommendations on how to improve future iterations.

SUBMITTER: Yin AL

PROVIDER: S-EPMC8529289 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:The objective was to compare case ascertainment, agreement, validity, and missing values for clinical research data obtained, processed, and linked electronically from electronic health records (EHR) compared to "manual" data processing and record abstraction in a cohort of out-of-hospital trauma patients.? This was a secondary analysis of two sets of data collected for a prospective, population-based, out-of-hospital trauma cohort evaluated by 10 emergency medical services (EMS) agencies transporting to 16 hospitals, from January 1, 2006, through October 2, 2007. Eighteen clinical, operational, procedural, and outcome variables were collected and processed separately and independently using two parallel data processing strategies by personnel blinded to patients in the other group. The electronic approach included EHR data exports from EMS agencies, reformatting, and probabilistic linkage to outcomes from local trauma registries and state discharge databases. The manual data processing approach included chart matching, data abstraction, and data entry by a trained abstractor. Descriptive statistics, measures of agreement, and validity were used to compare the two approaches to data processing.? During the 21-month period, 418 patients underwent both data processing methods and formed the primary cohort. Agreement was good to excellent (kappa = 0.76 to 0.97; intraclass correlation coefficient [ICC] = 0.49 to 0.97), with exact agreement in 67% to 99% of cases and a median difference of zero for all continuous and ordinal variables. The proportions of missing out-of-hospital values were similar between the two approaches, although electronic processing generated more missing outcomes (87 of 418, 21%, 95% confidence interval [CI] = 17% to 25%) than the manual approach (11 of 418, 3%, 95% CI = 1% to 5%). Case ascertainment of eligible injured patients was greater using electronic methods (n = 3,008) compared to manual methods (n = 629).? In this sample of out-of-hospital trauma patients, an all-electronic data processing strategy identified more patients and generated values with good agreement and validity compared to traditional data collection and processing methods.

Project description:BackgroundHealth care and public health professionals rely on accurate, real-time monitoring of infectious diseases for outbreak preparedness and response. Early detection of outbreaks is improved by systems that are comprehensive and specific with respect to the pathogen but are rapid in reporting the data. It has proven difficult to implement these requirements on a large scale while maintaining patient privacy.ObjectiveThe aim of this study was to demonstrate the automated export, aggregation, and analysis of infectious disease diagnostic test results from clinical laboratories across the United States in a manner that protects patient confidentiality. We hypothesized that such a system could aid in monitoring the seasonal occurrence of respiratory pathogens and may have advantages with regard to scope and ease of reporting compared with existing surveillance systems.MethodsWe describe a system, BioFire Syndromic Trends, for rapid disease reporting that is syndrome-based but pathogen-specific. Deidentified patient test results from the BioFire FilmArray multiplex molecular diagnostic system are sent directly to a cloud database. Summaries of these data are displayed in near real time on the Syndromic Trends public website. We studied this dataset for the prevalence, seasonality, and coinfections of the 20 respiratory pathogens detected in over 362,000 patient samples acquired as a standard-of-care testing over the last 4 years from 20 clinical laboratories in the United States.ResultsThe majority of pathogens show influenza-like seasonality, rhinovirus has fall and spring peaks, and adenovirus and the bacterial pathogens show constant detection over the year. The dataset can also be considered in an ecological framework; the viruses and bacteria detected by this test are parasites of a host (the human patient). Interestingly, the rate of pathogen codetections, on average 7.94% (28,741/362,101), matches predictions based on the relative abundance of organisms present.ConclusionsSyndromic Trends preserves patient privacy by removing or obfuscating patient identifiers while still collecting much useful information about the bacterial and viral pathogens that they harbor. Test results are uploaded to the database within a few hours of completion compared with delays of up to 10 days for other diagnostic-based reporting systems. This work shows that the barriers to establishing epidemiology systems are no longer scientific and technical but rather administrative, involving questions of patient privacy and data ownership. We have demonstrated here that these barriers can be overcome. This first look at the resulting data stream suggests that Syndromic Trends will be able to provide high-resolution analysis of circulating respiratory pathogens and may aid in the detection of new outbreaks.