Dataset Information

Automating Electronic Clinical Data Capture for Quality Improvement and Research: The CERTAIN Validation Project of Real World Evidence.

ABSTRACT:

Background

The availability of high fidelity electronic health record (EHR) data is a hallmark of the learning health care system. Washington State's Surgical Care Outcomes and Assessment Program (SCOAP) is a network of hospitals participating in quality improvement (QI) registries wherein data are manually abstracted from EHRs. To create the Comparative Effectiveness Research and Translation Network (CERTAIN), we semi-automated SCOAP data abstraction using a centralized federated data model, created a central data repository (CDR), and assessed whether these data could be used as real world evidence for QI and research.

Objectives

Describe the validation processes and complexities involved and lessons learned.

Methods

Investigators installed a commercial CDR to retrieve and store data from disparate EHRs. Manual and automated abstraction systems were conducted in parallel (10/2012-7/2013) and validated in three phases using the EHR as the gold standard: 1) ingestion, 2) standardization, and 3) concordance of automated versus manually abstracted cases. Information retrieval statistics were calculated.

Results

Four unaffiliated health systems provided data. Between 6 and 15 percent of data elements were abstracted: 51 to 86 percent from structured data; the remainder using natural language processing (NLP). In phase 1, data ingestion from 12 out of 20 feeds reached 95 percent accuracy. In phase 2, 55 percent of structured data elements performed with 96 to 100 percent accuracy; NLP with 89 to 91 percent accuracy. In phase 3, concordance ranged from 69 to 89 percent. Information retrieval statistics were consistently above 90 percent.

Conclusions

Semi-automated data abstraction may be useful, although raw data collected as a byproduct of health care delivery is not immediately available for use as real world evidence. New approaches to gathering and analyzing extant data are required.

SUBMITTER: Devine EB

PROVIDER: S-EPMC5983060 | biostudies-literature | 2018 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Automating Electronic Clinical Data Capture for Quality Improvement and Research: The CERTAIN Validation Project of Real World Evidence.

Devine Emily Beth EB Van Eaton Erik E Zadworny Megan E ME Symons Rebecca R Devlin Allison A Yanez David D Yetisgen Meliha M Keyloun Katelyn R KR Capurro Daniel D Alfonso-Cristancho Rafael R Flum David R DR Tarczy-Hornoch Peter P

EGEMS (Washington, DC) 20180522 1

<h4>Background</h4>The availability of high fidelity electronic health record (EHR) data is a hallmark of the learning health care system. Washington State's Surgical Care Outcomes and Assessment Program (SCOAP) is a network of hospitals participating in quality improvement (QI) registries wherein data are manually abstracted from EHRs. To create the Comparative Effectiveness Research and Translation Network (CERTAIN), we semi-automated SCOAP data abstraction using a centralized federated data m ...[more]

PMID: 29881766

Similar Datasets

Project description:Background: Adaptive interventions are often used in individualized health care to meet the unique needs of clients. Recently, more researchers have adopted the Sequential Multiple Assignment Randomized Trial (SMART), a type of research design, to build optimal adaptive interventions. SMART requires research participants to be randomized multiple times over time, depending upon their response to earlier interventions. Despite the increasing popularity of SMART designs, conducting a successful SMART study poses unique technological and logistical challenges (e.g., effectively concealing and masking allocation sequence to investigators, involved health care providers, and subjects) in addition to other challenges common to all study designs (e.g., study invitations, eligibility screening, consenting procedures, and data confidentiality protocols). Research Electronic Data Capture (REDCap) is a secure, browser-based web application widely used by researchers for data collection. REDCap offers unique features that support researchersâ€™ ability to conduct rigorous SMARTs. This manuscript provides an effective strategy for performing automatic double randomization for SMARTs using REDCap. Methods: Between January and March 2022, we conducted a SMART using a sample of adult (age 18 and older) New Jersey residents to optimize an adaptive intervention to increase COVID-19 testing uptake. In the current report, we discuss how we used REDCap for our SMART, which required double randomization. Further, we share our REDCap project XML file for future investigators to use when designing and conducting SMARTs. Results: We report on the randomization feature that REDCap offers and describe how the study team automated an additional randomization that was required for our SMART. An application programming interface was used to automate the double randomizations in conjunction with the randomization feature provided by REDCap. Conclusions: REDCap offers powerful tools to facilitate the implementation of longitudinal data collection and SMARTs. Investigators can make use of this electronic data capturing system to reduce errors and bias in the implementation of their SMARTs by automating double randomization. Trial registration: The SMART study was prospectively registered at Clinicaltrials.gov; registration number: NCT04757298, date of registration: 17/02/2021. Keywords: Research Electronic Data Capture (REDCap), randomized controlled trials (RCT), adaptive interventions, Sequential Multiple Assignment Randomized Trial (SMART), randomization, experimental design, reducing human errors, automation.

Project description:BackgroundStatistical analysis, which has become an integral part of evidence-based medicine, relies heavily on data quality that is of critical importance in modern clinical research. Input data are not only at risk of being falsified or fabricated, but also at risk of being mishandled by investigators.ObjectiveThe urgent need to assure the highest data quality possible has led to the implementation of various auditing strategies designed to monitor clinical trials and detect errors of different origin that frequently occur in the field. The objective of this study was to describe a machine learning-based algorithm to detect anomalous patterns in data created as a consequence of carelessness, systematic error, or intentionally by entering fabricated values.MethodsA particular electronic data capture (EDC) system, which is used for data management in clinical registries, is presented including its architecture and data structure. This EDC system features an algorithm based on machine learning designed to detect anomalous patterns in quantitative data. The detection algorithm combines clustering with a series of 7 distance metrics that serve to determine the strength of an anomaly. For the detection process, the thresholds and combinations of the metrics were used and the detection performance was evaluated and validated in the experiments involving simulated anomalous data and real-world data.ResultsFive different clinical registries related to neuroscience were presented-all of them running in the given EDC system. Two of the registries were selected for the evaluation experiments and served also to validate the detection performance on an independent data set. The best performing combination of the distance metrics was that of Canberra, Manhattan, and Mahalanobis, whereas Cosine and Chebyshev metrics had been excluded from further analysis due to the lowest performance when used as single distance metric-based classifiers.ConclusionsThe experimental results demonstrate that the algorithm is universal in nature, and as such may be implemented in other EDC systems, and is capable of anomalous data detection with a sensitivity exceeding 85%.

Project description:BackgroundRemote management is partially replacing routine follow-up in patients implanted with cardiac implantable electronic devices (CIEDs). Although it reduces clinical staff time compared with standard in-office follow-up, a new definition of roles and responsibilities may be needed to review remote transmissions in an effective, efficient, and timely manner. Whether remote triage may be outsourced to an external remote monitoring center (ERMC) is still unclear.ObjectiveThe aim of this health care quality improvement project was to evaluate the feasibility of outsourcing remote triage to an ERMC to improve patient care and health care resource utilization.MethodsPatients (N=153) with implanted CIEDs were followed up for 8 months. An ERMC composed of nurses and physicians reviewed remote transmissions daily following a specific remote monitoring (RM) protocol. A 6-month benchmarking phase where patients' transmissions were managed directly by hospital staff was evaluated as a term of comparison.ResultsA total of 654 transmissions were recorded in the RM system and managed by the ERMC team within 2 working days, showing a significant time reduction compared with standard RM management (100% vs 11%, respectively, within 2 days; P<.001). A total of 84.3% (551/654) of the transmissions did not include a prioritized event and did not require escalation to the hospital clinician. High priority was assigned to 2.3% (15/654) of transmissions, which were communicated to the hospital team by email within 1 working day. Nonurgent device status events occurred in 88 cases and were communicated to the hospital within 2 working days. Of these, 11% (10/88) were followed by a hospitalization.ConclusionsThe outsourcing of RM management to an ERMC safely provides efficacy and efficiency gains in patients' care compared with a standard in-hospital management. Moreover, the externalization of RM management could be a key tool for saving dedicated staff and facility time with possible positive economic impact.Trial registrationClinicalTrials.gov NCT01007474; http://clinicaltrials.gov/ct2/show/NCT01007474.

Dataset Information

Automating Electronic Clinical Data Capture for Quality Improvement and Research: The CERTAIN Validation Project of Real World Evidence.

Background

Objectives

Methods

Results

Conclusions

Publications

Automating Electronic Clinical Data Capture for Quality Improvement and Research: The CERTAIN Validation Project of Real World Evidence.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets