Dataset Information

Determining the Electronic Signature of Infection in Electronic Health Record Data.

ABSTRACT:

Objectives

Recent sepsis studies have defined patients as "infected" using a combination of culture and antibiotic orders rather than billing data. However, the accuracy of these definitions is unclear. We aimed to compare the accuracy of different established criteria for identifying infected patients using detailed chart review.

Design

Retrospective observational study.

Setting

Six hospitals from three health systems in Illinois.

Patients

Adult admissions with blood culture or antibiotic orders, or Angus International Classification of Diseases infection codes and death were eligible for study inclusion as potentially infected patients. Nine-hundred to 1,000 of these admissions were randomly selected from each health system for chart review, and a proportional number of patients who did not meet chart review eligibility criteria were also included and deemed not infected.

Interventions

None.

Measurements and main results

The accuracy of published billing code criteria by Angus et al and electronic health record criteria by Rhee et al and Seymour et al (Sepsis-3) was determined using the manual chart review results as the gold standard. A total of 5,215 patients were included, with 2,874 encounters analyzed via chart review and a proportional 2,341 added who did not meet chart review eligibility criteria. In the study cohort, 27.5% of admissions had at least one infection. This was most similar to the percentage of admissions with blood culture orders (26.8%), Angus infection criteria (28.7%), and the Sepsis-3 criteria (30.4%). Sepsis-3 criteria was the most sensitive (81%), followed by Angus (77%) and Rhee (52%), while Rhee (97%) and Angus (90%) were more specific than the Sepsis-3 criteria (89%). Results were similar for patients with organ dysfunction during their admission.

Conclusions

Published criteria have a wide range of accuracy for identifying infected patients, with the Sepsis-3 criteria being the most sensitive and Rhee criteria being the most specific. These findings have important implications for studies investigating the burden of sepsis on a local and national level.

SUBMITTER: Churpek MM

PROVIDER: S-EPMC8217098 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundDisease surveillance of diabetes among youth has relied mainly upon manual chart review. However, increasingly available structured electronic health record (EHR) data have been shown to yield accurate determinations of diabetes status and type. Validated algorithms to determine date of diabetes diagnosis are lacking. The objective of this work is to validate two EHR-based algorithms to determine date of diagnosis of diabetes.MethodsA rule-based ICD-10 algorithm identified youth with diabetes from structured EHR data over the period of 2009 through 2017 within three children's hospitals that participate in the SEARCH for Diabetes in Youth Study: Cincinnati Children's Hospital, Cincinnati, OH, Seattle Children's Hospital, Seattle, WA, and Children's Hospital Colorado, Denver, CO. Previous research and a multidisciplinary team informed the creation of two algorithms based upon structured EHR data to determine date of diagnosis among diabetes cases. An ICD-code algorithm was defined by the year of occurrence of a second ICD-9 or ICD-10 diabetes code. A multiple-criteria algorithm consisted of the year of first occurrence of any of the following: diabetes-related ICD code, elevated glucose, elevated HbA1c, or diabetes medication. We assessed algorithm performance by percent agreement with a gold standard date of diagnosis determined by chart review.ResultsAmong 3777 cases, both algorithms demonstrated high agreement with true diagnosis year and differed in classification (p = 0.006): 86.5% agreement for the ICD code algorithm and 85.9% agreement for the multiple-criteria algorithm. Agreement was high for both type 1 and type 2 cases for the ICD code algorithm. Performance improved over time.ConclusionsYear of occurrence of the second ICD diabetes-related code in the EHR yields an accurate diagnosis date within these pediatric hospital systems. This may lead to increased efficiency and sustainability of surveillance methods for incidence of diabetes among youth.

Project description:ObjectivesWe tested the hypothesis that routine monitoring data could describe a detailed and distinct pathophysiologic phenotype of impending hypoglycemia in adult ICU patients.DesignRetrospective analysis leading to model development and validation.SettingAll ICU admissions wherein patients received insulin therapy during a 4-year period at the University of Virginia Medical Center. Each ICU was equipped with continuous physiologic monitoring systems whose signals were archived in an electronic data warehouse along with the entire medical record.PatientsEleven thousand eight hundred forty-seven ICU patient admissions.InterventionsThe primary outcome was hypoglycemia, defined as any episode of blood glucose less than 70 mg/dL where 50% dextrose injection was administered within 1 hour. We used 61 physiologic markers (including vital signs, laboratory values, demographics, and continuous cardiorespiratory monitoring variables) to inform the model.Measurements and main resultsOur dataset consisted of 11,847 ICU patient admissions, 721 (6.1%) of which had one or more hypoglycemic episodes. Multivariable logistic regression analysis revealed a pathophysiologic signature of 41 independent variables that best characterized ICU hypoglycemia. The final model had a cross-validated area under the receiver operating characteristic curve of 0.83 (95% CI, 0.78-0.87) for prediction of impending ICU hypoglycemia. We externally validated the model in the Medical Information Mart for Intensive Care III critical care dataset, where it also demonstrated good performance with an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.77-0.81).ConclusionsWe used data from a large number of critically ill inpatients to develop and externally validate a predictive model of impending ICU hypoglycemia. Future steps include incorporating this model into a clinical decision support system and testing its effects in a multicenter randomized controlled clinical trial.

Project description:BACKGROUND:Readmissions after hospitalization for pneumonia are common, but the few risk-prediction models have poor to modest predictive ability. Data routinely collected in the electronic health record (EHR) may improve prediction. OBJECTIVE:To develop pneumonia-specific readmission risk-prediction models using EHR data from the first day and from the entire hospital stay ("full stay"). DESIGN:Observational cohort study using stepwise-backward selection and cross-validation. SUBJECTS:Consecutive pneumonia hospitalizations from 6 diverse hospitals in north Texas from 2009-2010. MEASURES:All-cause nonelective 30-day readmissions, ascertained from 75 regional hospitals. RESULTS:Of 1463 patients, 13.6% were readmitted. The first-day pneumonia-specific model included sociodemographic factors, prior hospitalizations, thrombocytosis, and a modified pneumonia severity index; the full-stay model included disposition status, vital sign instabilities on discharge, and an updated pneumonia severity index calculated using values from the day of discharge as additional predictors. The full-stay pneumonia-specific model outperformed the first-day model (C statistic 0.731 vs 0.695; P = 0.02; net reclassification index = 0.08). Compared to a validated multi-condition readmission model, the Centers for Medicare and Medicaid Services pneumonia model, and 2 commonly used pneumonia severity of illness scores, the full-stay pneumonia-specific model had better discrimination (C statistic range 0.604-0.681; P < 0.01 for all comparisons), predicted a broader range of risk, and better reclassified individuals by their true risk (net reclassification index range, 0.09-0.18). CONCLUSIONS:EHR data collected from the entire hospitalization can accurately predict readmission risk among patients hospitalized for pneumonia. This approach outperforms a first-day pneumonia-specific model, the Centers for Medicare and Medicaid Services pneumonia model, and 2 commonly used pneumonia severity of illness scores. Journal of Hospital Medicine 2017;12:209-216.

Project description:ImportanceSecondary prevention interventions to reduce post-stroke cognitive impairment (PSCI) can be aided by the early identification of high-risk individuals who would benefit from risk factor modification.ObjectiveTo develop and evaluate a predictive model to identify patients at increased risk of PSCI over 5 years using data easily accessible from electronic health records.DesignCohort study with patients enrolled between 2003-2016 with follow-up through 2022.SettingPrimary care practices affiliated with two academic medical centers.ParticipantsIndividuals 45 years or older, without prior stroke or prevalent cognitive impairment, with primary care visits and an incident ischemic stroke between 2003-2016 (development/internal validation cohort) or 2010-2022 (external validation cohort).ExposuresPredictors of PSCI were ascertained from the electronic health record.Main outcomeThe outcome was incident dementia/cognitive impairment within 5 years and beginning 3 months following stroke, ascertained using ICD-9/10 codes. For model variable selection, we considered potential predictors of PSCI and constructed 400 bootstrap samples with two-thirds of the model derivation sample. We ran 10-fold cross-validated Cox proportional hazards models using a least absolute shrinkage and selection operator (LASSO) penalty. Variables selected in >25% of samples were included.ResultsThe analysis included 332 incident diagnoses of PSCI in the development cohort (n=3,741), and 161 and 128 incident diagnoses in the internal (n=1,925) and external (n=2,237) validation cohorts. The c-statistic for predicting PSCI was 0.731 (95% CI: 0.694-0.768) in the internal validation cohort, and 0.724 (95% CI: 0.681-0.766) in the external validation cohort. A risk score based on the beta coefficients of predictors from the development cohort stratified patients into low (0-7 points), intermediate (8-11 points), and high (12-35 points) risk groups. The hazard ratios for incident PSCI were significantly different by risk categories in internal (High, HR: 6.2, 95% CI 4.1-9.3; Intermediate, HR 2.7, 95% CI: 1.8-4.1) and external (High, HR: 6.1, 95% CI: 3.9-9.6; Intermediate, HR 2.8, 95% CI: 1.9-4.3) validation cohorts.Conclusions and relevanceFive-year risk of PSCI can be accurately predicted using routinely collected data. Model output can be used to risk stratify and identify individuals at increased risk for PSCI for preventive efforts.

Project description:Electronic health records (EHRs) have been successfully used in data science and machine learning projects. However, most of these data are collected for clinical use rather than for retrospective analysis. This means that researchers typically face many different issues when attempting to access and prepare the data for secondary use. We aimed to investigate how raw EHRs can be accessed and prepared in retrospective data science projects in a disciplined, effective, and efficient way. We report our experience and findings from a large-scale data science project analyzing routinely acquired retrospective data from the Kepler University Hospital in Linz, Austria. The project involved data collection from more than 150,000 patients over a period of 10 years. It included diverse data modalities, such as static demographic data, irregularly acquired laboratory test results, regularly sampled vital signs, and high-frequency physiological waveform signals. Raw medical data can be corrupted in many unexpected ways that demand thorough manual inspection and highly individualized data cleaning solutions. We present a general data preparation workflow, which was shaped in the course of our project and consists of the following 7 steps: obtain a rough overview of the available EHR data, define clinically meaningful labels for supervised learning, extract relevant data from the hospital's data warehouses, match data extracted from different sources, deidentify them, detect errors and inconsistencies therein through a careful exploratory analysis, and implement a suitable data processing pipeline in actual code. Only few of the data preparation issues encountered in our project were addressed by generic medical data preprocessing tools that have been proposed recently. Instead, highly individualized solutions for the specific data used in one's own research seem inevitable. We believe that the proposed workflow can serve as a guidance for practitioners, helping them to identify and address potential problems early and avoid some common pitfalls.

Project description:ObjectiveTo demonstrate that electronic health record (EHR) data can be used in an automated approach to evaluate cataract surgery outcomes.DesignRetrospective analysis.SubjectsResident and faculty surgeons.MethodsElectronic health record data were collected from cataract surgeries performed at the Johns Hopkins Wilmer Eye Institute, and cases were categorized into resident or attending as primary surgeon. Preoperative and postoperative visual acuity (VA) and unplanned return to operating room were extracted from the EHR.Main outcome measuresPostoperative VA and reoperation rate within 90 days.ResultsThis study analyzed 14 537 cataract surgery cases over 32 months. Data were extracted from the EHR using an automated approach to assess surgical outcomes for resident and attending surgeons. Of 337 resident surgeries with both preoperative and postoperative VA data, 248 cases (74%) had better postoperative VA, and 170 cases (51%) had more than 2 lines improvement. There was no statistical difference in the proportion of cases with better postoperative VA or more than 2 lines improvement between resident and attending cases. Attending surgeons had a statistically greater proportion of cases with postoperative VA better than 20/40, but this finding has to be considered in the context that, on average, resident cases started out with poorer baseline VA.A multivariable regression model of VA outcomes vs. resident/attending status that controlled for preoperative VA, patient age, American Society of Anesthesiologists (ASA) score, and estimated income found that resident status, preoperative VA, patient age, ASA score, and estimated income were all significant predictors of VA. The rate of unplanned return to the operating room within 90 days of cataract surgery was not statistically different between resident (1.8%) and attending (1.2%) surgeons.ConclusionsThis study demonstrates that EHR data can be used to evaluate and monitor surgical outcomes in an ongoing way. Analysis of EHR-extracted cataract outcome data showed that preoperative VA, ASA classification, and attending/resident status were important in predicting postoperative VA outcomes. These findings suggest that the utilization of EHR data could enable continuous assessment of surgical outcomes and inform interventions to improve resident training. Financial Disclosure(s): Proprietary or commercial disclosure may be found after the references.

Project description:BackgroundPoor functional status is a key marker of morbidity, yet is not routinely captured in clinical encounters. We developed and evaluated the accuracy of a machine learning algorithm that leveraged electronic health record (EHR) data to provide a scalable process for identification of functional impairment.MethodsWe identified a cohort of patients with an electronically captured screening measure of functional status (Older Americans Resources and Services ADL/IADL) between 2018 and 2020 (N = 6484). Patients were classified using unsupervised learning K means and t-distributed Stochastic Neighbor Embedding into normal function (NF), mild to moderate functional impairment (MFI), and severe functional impairment (SFI) states. Using 11 EHR clinical variable domains (832 variable input features), we trained an Extreme Gradient Boosting supervised machine learning algorithm to distinguish functional status states, and measured prediction accuracies. Data were randomly split into training (80%) and test (20%) sets. The SHapley Additive Explanations (SHAP) feature importance analysis was used to list the EHR features in rank order of their contribution to the outcome.ResultsMedian age was 75.3 years, 62% female, 60% White. Patients were classified as 53% NF (n = 3453), 30% MFI (n = 1947), and 17% SFI (n = 1084). Summary of model performance for identifying functional status state (NF, MFI, SFI) was AUROC (area under the receiving operating characteristic curve) 0.92, 0.89, and 0.87, respectively. Age, falls, hospitalization, home health use, labs (e.g., albumin), comorbidities (e.g., dementia, heart failure, chronic kidney disease, chronic pain), and social determinants of health (e.g., alcohol use) were highly ranked features in predicting functional status states.ConclusionA machine learning algorithm run on EHR clinical data has potential utility for differentiating functional status in the clinical setting. Through further validation and refinement, such algorithms can complement traditional screening methods and result in a population-based strategy for identifying patients with poor functional status who need additional health resources.