Dataset Information

Development and External Validation of a Machine Learning Tool to Rule Out COVID-19 Among Adults in the Emergency Department Using Routine Blood Tests: A Large, Multicenter, Real-World Study.

ABSTRACT: Background: Conventional diagnosis of COVID-19 with reverse transcription polymerase chain reaction (RT-PCR) testing (hereafter, PCR) is associated with prolonged time to diagnosis and significant costs to run the test. The SARS-CoV-2 virus might lead to characteristic patterns in the results of widely available, routine blood tests that could be identified with machine learning methodologies. Machine learning modalities integrating findings from these common laboratory test results might accelerate ruling out COVID-19 in emergency department patients.

Objective: We sought to develop (ie, train and internally validate with cross-validation techniques) and externally validate a machine learning model to rule out COVID 19 using only routine blood tests among adults in emergency departments.

Methods: Using clinical data from emergency departments (EDs) from 66 US hospitals before the pandemic (before the end of December 2019) or during the pandemic (March-July 2020), we included patients aged ?20 years in the study time frame. We excluded those with missing laboratory results. Model training used 2183 PCR-confirmed cases from 43 hospitals during the pandemic; negative controls were 10,000 prepandemic patients from the same hospitals. External validation used 23 hospitals with 1020 PCR-confirmed cases and 171,734 prepandemic negative controls. The main outcome was COVID 19 status predicted using same-day routine laboratory results. Model performance was assessed with area under the receiver operating characteristic (AUROC) curve as well as sensitivity, specificity, and negative predictive value (NPV).

Results: Of 192,779 patients included in the training, external validation, and sensitivity data sets (median age decile 50 [IQR 30-60] years, 40.5% male [78,249/192,779]), AUROC for training and external validation was 0.91 (95% CI 0.90-0.92). Using a risk score cutoff of 1.0 (out of 100) in the external validation data set, the model achieved sensitivity of 95.9% and specificity of 41.7%; with a cutoff of 2.0, sensitivity was 92.6% and specificity was 59.9%. At the cutoff of 2.0, the NPVs at a prevalence of 1%, 10%, and 20% were 99.9%, 98.6%, and 97%, respectively.

Conclusions: A machine learning model developed with multicenter clinical data integrating commonly collected ED laboratory data demonstrated high rule-out accuracy for COVID-19 status, and might inform selective use of PCR-based testing.

SUBMITTER: Plante TB

PROVIDER: S-EPMC7713695 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and External Validation of a Machine Learning Tool to Rule Out COVID-19 Among Adults in the Emergency Department Using Routine Blood Tests: A Large, Multicenter, Real-World Study.

Plante Timothy B TB Blau Aaron M AM Berg Adrian N AN Weinberg Aaron S AS Jun Ik C IC Tapson Victor F VF Kanigan Tanya S TS Adib Artur B AB

Journal of medical Internet research 20201202 12

<h4>Background</h4>Conventional diagnosis of COVID-19 with reverse transcription polymerase chain reaction (RT-PCR) testing (hereafter, PCR) is associated with prolonged time to diagnosis and significant costs to run the test. The SARS-CoV-2 virus might lead to characteristic patterns in the results of widely available, routine blood tests that could be identified with machine learning methodologies. Machine learning modalities integrating findings from these common laboratory test results might ...[more]

PMID: 33226957

Similar Datasets

Project description:ObjectivesTraditional approaches to safety and quality screening in the emergency department (ED) are porous and low yield for identifying adverse events (AEs). A better approach may be in the use of trigger tool methodology. We recently developed a novel ED trigger tool using a multidisciplinary, multicenter approach. We conducted a multicenter test of this tool and assess its performance.MethodsIn design and participants, we studied the ED trigger tool for a 13-month period at four EDs. All patients 18 years and older with Emergency Severity Index acuity levels of 1 to 3 seen by a provider were eligible. Reviewers completed standardized training modules. Each site reviewed 50 randomly selected visits per month. A first-level reviewer screened for presence of predefined triggers (findings that increase the probability of an AE). If no trigger is present, the review is deemed complete. When present, a trigger prompts an in-depth review for an AE. Any event identified is assigned a level of harm using the Medication Event Reporting and Prevention (MERP) Index, ranging from a near miss (A) to patient death (I). Events are noted as present on arrival or in the ED, an act of commission or omission, and are assigned one of four event categories. A second-level physician performs a confirmatory review of all AEs and independently reviews 10% of cases to estimate the false-negative rate. All AEs or potential AEs were reviewed in monthly group calls for consensus on findings. The primary outcome is the proportion of visits in which an AE is identified, overall and by site. Secondary outcomes include categories of events, distribution of harm ratings, and association of AEs with sociodemographic and clinical factors and triggers. We present sociodemographic data and details about AEs and results of logistic regression for associations of AEs with of triggers, sociodemographics, and clinical variables.ResultsWe captured 2594 visits that are representative, within site, of their patient population. Overall, the sample is 64% white, 54% female, and with a mean age of 51. Variability is observed between sites for age, race, and insurance, but not sex. A total of 240 events were identified in 228 visits (8.8%) of which 53.3% were present on arrival, 19.7% were acts of omission, and 44.6% were medication-related, with some variability across sites. A MERP F score (contributing to need for admission, higher level of care, or prolonged hospitalization) was the most common severity level (35.4% of events). Overall, 185 (77.1%) of 240 events involved patient harm (MERP level ≥ E), affecting 175 visits (6.7%). Triggers were present in 951 visits (36.6%). Presence of any trigger was strongly associated with an AE (adjusted odds ratio = 4.6, 95% confidence interval = 3.2-6.6). Ten triggers were individually associated with AEs (adjusted odds ratio = 2.1-7.7). Variability was observed across sites in individual trigger associations, event rates, and categories, but not in severity ratings of events. The overall false-negative rate was 6.1%.ConclusionsThe trigger tool approach was successful in identifying meaningful events. The ED trigger tool seems to be a promising approach for identifying all-cause harm in the ED.

Dataset Information

Development and External Validation of a Machine Learning Tool to Rule Out COVID-19 Among Adults in the Emergency Department Using Routine Blood Tests: A Large, Multicenter, Real-World Study.

Publications

Development and External Validation of a Machine Learning Tool to Rule Out COVID-19 Among Adults in the Emergency Department Using Routine Blood Tests: A Large, Multicenter, Real-World Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets