Dataset Information

Electronic versus manual data processing: evaluating the use of electronic health records in out-of-hospital clinical research.

ABSTRACT: The objective was to compare case ascertainment, agreement, validity, and missing values for clinical research data obtained, processed, and linked electronically from electronic health records (EHR) compared to "manual" data processing and record abstraction in a cohort of out-of-hospital trauma patients.? This was a secondary analysis of two sets of data collected for a prospective, population-based, out-of-hospital trauma cohort evaluated by 10 emergency medical services (EMS) agencies transporting to 16 hospitals, from January 1, 2006, through October 2, 2007. Eighteen clinical, operational, procedural, and outcome variables were collected and processed separately and independently using two parallel data processing strategies by personnel blinded to patients in the other group. The electronic approach included EHR data exports from EMS agencies, reformatting, and probabilistic linkage to outcomes from local trauma registries and state discharge databases. The manual data processing approach included chart matching, data abstraction, and data entry by a trained abstractor. Descriptive statistics, measures of agreement, and validity were used to compare the two approaches to data processing.? During the 21-month period, 418 patients underwent both data processing methods and formed the primary cohort. Agreement was good to excellent (kappa = 0.76 to 0.97; intraclass correlation coefficient [ICC] = 0.49 to 0.97), with exact agreement in 67% to 99% of cases and a median difference of zero for all continuous and ordinal variables. The proportions of missing out-of-hospital values were similar between the two approaches, although electronic processing generated more missing outcomes (87 of 418, 21%, 95% confidence interval [CI] = 17% to 25%) than the manual approach (11 of 418, 3%, 95% CI = 1% to 5%). Case ascertainment of eligible injured patients was greater using electronic methods (n = 3,008) compared to manual methods (n = 629).? In this sample of out-of-hospital trauma patients, an all-electronic data processing strategy identified more patients and generated values with good agreement and validity compared to traditional data collection and processing methods.

SUBMITTER: Newgard CD

PROVIDER: S-EPMC3278860 | biostudies-literature | 2012 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Electronic versus manual data processing: evaluating the use of electronic health records in out-of-hospital clinical research.

Newgard Craig D CD Zive Dana D Jui Jonathan J Weathers Cody C Daya Mohamud M

Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 20120201 2

<h4>Objectives</h4> The objective was to compare case ascertainment, agreement, validity, and missing values for clinical research data obtained, processed, and linked electronically from electronic health records (EHR) compared to "manual" data processing and record abstraction in a cohort of out-of-hospital trauma patients.<h4>Methods</h4> This was a secondary analysis of two sets of data collected for a prospective, population-based, out-of-hospital trauma cohort evaluated by 10 emergency m ...[more]

PMID: 22320373

Similar Datasets

Project description:ImportanceInternational Classification of Diseases-coded hospital discharge data do not accurately reflect whether firearm injuries were caused by assault, unintentional injury, self-harm, legal intervention, or were of undetermined intent. Applying natural language processing (NLP) and machine learning (ML) techniques to electronic health record (EHR) narrative text could be associated with improved accuracy of firearm injury intent data.ObjectiveTo assess the accuracy with which an ML model identified firearm injury intent.Design, setting, and participantsA cross-sectional retrospective EHR review was conducted at 3 level I trauma centers, 2 from health care institutions in Boston, Massachusetts, and 1 from Seattle, Washington, between January 1, 2000, and December 31, 2019; data analysis was performed from January 18, 2021, to August 22, 2022. A total of 1915 incident cases of firearm injury in patients presenting to emergency departments at the model development institution and 769 from the external validation institution with a firearm injury code assigned according to International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) or International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Clinical Modification (ICD-10-CM), in discharge data were included.ExposuresClassification of firearm injury intent.Main outcomes and measuresIntent classification accuracy by the NLP model was compared with ICD codes assigned by medical record coders in discharge data. The NLP model extracted intent-relevant features from narrative text that were then used by a gradient-boosting classifier to determine the intent of each firearm injury. Classification accuracy was evaluated against intent assigned by the research team. The model was further validated using an external data set.ResultsThe NLP model was evaluated in 381 patients presenting with firearm injury at the model development site (mean [SD] age, 39.2 [13.0] years; 348 [91.3%] men) and 304 patients at the external development site (mean [SD] age, 31.8 [14.8] years; 263 [86.5%] men). The model proved more accurate than medical record coders in assigning intent to firearm injuries at the model development site (accident F-score, 0.78 vs 0.40; assault F-score, 0.90 vs 0.78). The model maintained this improvement on an external validation set from a second institution (accident F-score, 0.64 vs 0.58; assault F-score, 0.88 vs 0.81). While the model showed some degradation between institutions, retraining the model using data from the second institution further improved performance on that site's records (accident F-score, 0.75; assault F-score, 0.92).Conclusions and relevanceThe findings of this study suggest that NLP ML can be used to improve the accuracy of firearm injury intent classification compared with ICD-coded discharge data, particularly for cases of accident and assault intents (the most prevalent and commonly misclassified intent types). Future research could refine this model using larger and more diverse data sets.

Project description:BackgroundEHR phenotyping offers the ability to rapidly assemble a precisely defined cohort of patients prescreened for eligibility to participate in health-related research. Even so, stakeholders in the process must still contend with the practical and ethical challenges associated with research recruitment. Patient perspectives on these matters are particularly important given that the success of research recruitment depends on patients' willingness to participate.MethodsWe conducted 15 focus groups (n =?110 participants) in four counties in diverse regions of the southeastern US: Appalachia, the Mississippi Delta, and the Piedmont area of North Carolina. Based on a hypothetical study of a behavioral intervention for type 2 diabetes, we asked about the acceptability and appropriateness of direct investigator versus physician-mediated contact with patients for research recruitment, and whether patients should be asked to opt in or opt out of further contact in response to recruitment letters.ResultsFor initial contact, nearly all participants said it would be acceptable for researchers to contact patients directly and three-fourths said that it would be acceptable for researchers to contact patients through their physicians. When we asked which would be most appropriate, a substantial majority chose direct contact. Themes that arose in the discussion included trust and transparency, decision-making power, the effect on research, and the effect on patient care. For response expectations, the vast majority of participants said both opt-in and opt-out would be acceptable-typically finding neither especially problematic and noting that both afford patients the opportunity to make their own decisions.ConclusionsExternal validity relies heavily on researchers' success enrolling eligible patients and failure to reach accrual targets is a costly and common barrier to advancing scientific knowledge. Our results suggest that patients recognize multiple advantages and disadvantages of different research recruitment strategies and place value on the implications not just for themselves, but also for researchers and healthcare providers. Our findings, including rich qualitative detail, contribute to the body of empirical and ethical literature on improving research recruitment and suggest specific ways forward as well as important areas for future research.

Project description:There is an urgent need to monitor the mental health of large populations, especially during crises such as the COVID-19 pandemic, to timely identify the most at-risk subgroups and to design targeted prevention campaigns. We therefore developed and validated surveillance indicators related to suicidality: the monthly number of hospitalisations caused by suicide attempts and the prevalence among them of five known risks factors. They were automatically computed analysing the electronic health records of fifteen university hospitals of the Paris area, France, using natural language processing algorithms based on artificial intelligence. We evaluated the relevance of these indicators conducting a retrospective cohort study. Considering 2,911,920 records contained in a common data warehouse, we tested for changes after the pandemic outbreak in the slope of the monthly number of suicide attempts by conducting an interrupted time-series analysis. We segmented the assessment time in two sub-periods: before (August 1, 2017, to February 29, 2020) and during (March 1, 2020, to June 31, 2022) the COVID-19 pandemic. We detected 14,023 hospitalisations caused by suicide attempts. Their monthly number accelerated after the COVID-19 outbreak with an estimated trend variation reaching 3.7 (95%CI 2.1-5.3), mainly driven by an increase among girls aged 8-17 (trend variation 1.8, 95%CI 1.2-2.5). After the pandemic outbreak, acts of domestic, physical and sexual violence were more often reported (prevalence ratios: 1.3, 95%CI 1.16-1.48; 1.3, 95%CI 1.10-1.64 and 1.7, 95%CI 1.48-1.98), fewer patients died (p = 0.007) and stays were shorter (p < 0.001). Our study demonstrates that textual clinical data collected in multiple hospitals can be jointly analysed to compute timely indicators describing mental health conditions of populations. Our findings also highlight the need to better take into account the violence imposed on women, especially at early ages and in the aftermath of the COVID-19 pandemic.

Project description:BACKGROUND:Electronic medical records (EMRs) contain a wealth of information that can support data-driven decision making in health care policy design and service planning. Although research using EMRs has become increasingly prevalent, challenges such as coding inconsistency, data validity, and lack of suitable measures in important domains still hinder the progress. OBJECTIVE:The objective of this study was to design a structured way to process records in administrative EMR systems for health services research and assess validity in selected areas. METHODS:On the basis of a local hospital EMR system in Singapore, we developed a structured framework for EMR data processing, including standardization and phenotyping of diagnosis codes, construction of cohort with multilevel views, and generation of variables and proxy measures to supplement primary data. Disease complexity was estimated by Charlson Comorbidity Index (CCI) and Polypharmacy Score (PPS), whereas socioeconomic status (SES) was estimated by housing type. Validity of modified diagnosis codes and derived measures were investigated. RESULTS:Visit-level (N=7,778,761) and patient-level records (n=549,109) were generated. The International Classification of Diseases, Tenth Revision, Australian Modification (ICD-10-AM) codes were standardized to the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) with a mapping rate of 87.1%. In all, 97.4% of the ICD-9-CM codes were phenotyped successfully using Clinical Classification Software by Agency for Healthcare Research and Quality. Diagnosis codes that underwent modification (truncation or zero addition) in standardization and phenotyping procedures had the modification validated by physicians, with validity rates of more than 90%. Disease complexity measures (CCI and PPS) and SES were found to be valid and robust after a correlation analysis and a multivariate regression analysis. CCI and PPS were correlated with each other and positively correlated with health care utilization measures. Larger housing type was associated with lower government subsidies received, suggesting association with higher SES. Profile of constructed cohorts showed differences in disease prevalence, disease complexity, and health care utilization in those aged above 65 years and those aged 65 years or younger. CONCLUSIONS:The framework proposed in this study would be useful for other researchers working with EMR data for health services research. Further analyses would be needed to better understand differences observed in the cohorts.

Dataset Information

Electronic versus manual data processing: evaluating the use of electronic health records in out-of-hospital clinical research.

Publications

Electronic versus manual data processing: evaluating the use of electronic health records in out-of-hospital clinical research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets