Dataset Information

Prognostication of patients with COVID-19 using artificial intelligence based on chest x-rays and clinical data: a retrospective study.

ABSTRACT:

Background

Chest x-ray is a relatively accessible, inexpensive, fast imaging modality that might be valuable in the prognostication of patients with COVID-19. We aimed to develop and evaluate an artificial intelligence system using chest x-rays and clinical data to predict disease severity and progression in patients with COVID-19.

Methods

We did a retrospective study in multiple hospitals in the University of Pennsylvania Health System in Philadelphia, PA, USA, and Brown University affiliated hospitals in Providence, RI, USA. Patients who presented to a hospital in the University of Pennsylvania Health System via the emergency department, with a diagnosis of COVID-19 confirmed by RT-PCR and with an available chest x-ray from their initial presentation or admission, were retrospectively identified and randomly divided into training, validation, and test sets (7:1:2). Using the chest x-rays as input to an EfficientNet deep neural network and clinical data, models were trained to predict the binary outcome of disease severity (ie, critical or non-critical). The deep-learning features extracted from the model and clinical data were used to build time-to-event models to predict the risk of disease progression. The models were externally tested on patients who presented to an independent multicentre institution, Brown University affiliated hospitals, and compared with severity scores provided by radiologists.

Findings

1834 patients who presented via the University of Pennsylvania Health System between March 9 and July 20, 2020, were identified and assigned to the model training (n=1285), validation (n=183), or testing (n=366) sets. 475 patients who presented via the Brown University affiliated hospitals between March 1 and July 18, 2020, were identified for external testing of the models. When chest x-rays were added to clinical data for severity prediction, area under the receiver operating characteristic curve (ROC-AUC) increased from 0·821 (95% CI 0·796-0·828) to 0·846 (0·815-0·852; p<0·0001) on internal testing and 0·731 (0·712-0·738) to 0·792 (0·780-0 ·803; p<0·0001) on external testing. When deep-learning features were added to clinical data for progression prediction, the concordance index (C-index) increased from 0·769 (0·755-0·786) to 0·805 (0·800-0·820; p<0·0001) on internal testing and 0·707 (0·695-0·729) to 0·752 (0·739-0·764; p<0·0001) on external testing. The image and clinical data combined model had significantly better prognostic performance than combined severity scores and clinical data on internal testing (C-index 0·805 vs 0·781; p=0·0002) and external testing (C-index 0·752 vs 0·715; p<0·0001).

Interpretation

In patients with COVID-19, artificial intelligence based on chest x-rays had better prognostic performance than clinical data or radiologist-derived severity scores. Using artificial intelligence, chest x-rays can augment clinical data in predicting the risk of progression to critical illness in patients with COVID-19.

Funding

Brown University, Amazon Web Services Diagnostic Development Initiative, Radiological Society of North America, National Cancer Institute and National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health.

SUBMITTER: Jiao Z

PROVIDER: S-EPMC7990487 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundComputed tomography (CT) imaging and artificial intelligence (AI)-based analyses have aided in the diagnosis and prediction of the severity of COVID-19. However, the potential of AI-based CT quantification of pneumonia in assessing patients with COVID-19 has not yet been fully explored. This study aimed to investigate the potential of AI-based CT quantification of COVID-19 pneumonia to predict the critical outcomes and clinical characteristics of patients with residual lung lesions.MethodsThis retrospective cohort study included 1,200 hospitalized patients with COVID-19 from four hospitals. The incidence of critical outcomes (requiring the support of high-flow oxygen or invasive mechanical ventilation or death) and complications during hospitalization (bacterial infection, renal failure, heart failure, thromboembolism, and liver dysfunction) was compared between the groups of pneumonia with high/low-percentage lung lesions, based on AI-based CT quantification. Additionally, 198 patients underwent CT scans 3 months after admission to analyze prognostic factors for residual lung lesions.ResultsThe pneumonia group with a high percentage of lung lesions (N = 400) had a higher incidence of critical outcomes and complications during hospitalization than the low percentage group (N = 800). Multivariable analysis demonstrated that AI-based CT quantification of pneumonia was independently associated with critical outcomes (adjusted odds ratio [aOR] 10.5, 95% confidence interval [CI] 5.59-19.7), as well as with oxygen requirement (aOR 6.35, 95% CI 4.60-8.76), IMV requirement (aOR 7.73, 95% CI 2.52-23.7), and mortality rate (aOR 6.46, 95% CI 1.87-22.3). Among patients with follow-up CT scans (N = 198), the multivariable analysis revealed that the pneumonia group with a high percentage of lung lesions on admission (aOR 4.74, 95% CI 2.36-9.52), older age (aOR 2.53, 95% CI 1.16-5.51), female sex (aOR 2.41, 95% CI 1.13-5.11), and medical history of hypertension (aOR 2.22, 95% CI 1.09-4.50) independently predicted persistent residual lung lesions.ConclusionsAI-based CT quantification of pneumonia provides valuable information beyond qualitative evaluation by physicians, enabling the prediction of critical outcomes and residual lung lesions in patients with COVID-19.

Project description:ObjectivesTo study whether a trained convolutional neural network (CNN) can be of assistance to radiologists in differentiating Coronavirus disease (COVID)-positive from COVID-negative patients using chest X-ray (CXR) through an ambispective clinical study. To identify subgroups of patients where artificial intelligence (AI) can be of particular value and analyse what imaging features may have contributed to the performance of AI by means of visualisation techniques.MethodsCXR of 487 patients were classified into [4] categories-normal, classical COVID, indeterminate, and non-COVID by consensus opinion of 2 radiologists. CXR which were classified as "normal" and "indeterminate" were then subjected to analysis by AI, and final categorisation provided as guided by prediction of the network. Precision and recall of the radiologist alone and radiologist assisted by AI were calculated in comparison to reverse transcriptase-polymerase chain reaction (RT-PCR) as the gold standard. Attention maps of the CNN were analysed to understand regions in the CXR important to the AI algorithm in making a prediction.ResultsThe precision of radiologists improved from 65.9 to 81.9% and recall improved from 17.5 to 71.75 when assistance with AI was provided. AI showed 92% accuracy in classifying "normal" CXR into COVID or non-COVID. Analysis of attention maps revealed attention on the cardiac shadow in these "normal" radiographs.ConclusionThis study shows how deployment of an AI algorithm can complement a human expert in the determination of COVID status. Analysis of the detected features suggests possible subtle cardiac changes, laying ground for further investigative studies into possible cardiac changes.Key points• Through an ambispective clinical study, we show how assistance with an AI algorithm can improve recall (sensitivity) and precision (positive predictive value) of radiologists in assessing CXR for possible COVID in comparison to RT-PCR. • We show that AI achieves the best results in images classified as "normal" by radiologists. We conjecture that possible subtle cardiac in the CXR, imperceptible to the human eye, may have contributed to this prediction. • The reported results may pave the way for a human computer collaboration whereby the expert with some help from the AI algorithm achieves higher accuracy in predicting COVID status on CXR than previously thought possible when considering either alone.

Project description:Background There are characteristic findings of coronavirus disease 2019 (COVID-19) on chest images. An artificial intelligence (AI) algorithm to detect COVID-19 on chest radiographs might be useful for triage or infection control within a hospital setting, but prior reports have been limited by small data sets, poor data quality, or both. Purpose To present DeepCOVID-XR, a deep learning AI algorithm to detect COVID-19 on chest radiographs, that was trained and tested on a large clinical data set. Materials and Methods DeepCOVID-XR is an ensemble of convolutional neural networks developed to detect COVID-19 on frontal chest radiographs, with reverse-transcription polymerase chain reaction test results as the reference standard. The algorithm was trained and validated on 14 788 images (4253 positive for COVID-19) from sites across the Northwestern Memorial Health Care System from February 2020 to April 2020 and was then tested on 2214 images (1192 positive for COVID-19) from a single hold-out institution. Performance of the algorithm was compared with interpretations from five experienced thoracic radiologists on 300 random test images using the McNemar test for sensitivity and specificity and the DeLong test for the area under the receiver operating characteristic curve (AUC). Results A total of 5853 patients (mean age, 58 years ± 19 [standard deviation]; 3101 women) were evaluated across data sets. For the entire test set, accuracy of DeepCOVID-XR was 83%, with an AUC of 0.90. For 300 random test images (134 positive for COVID-19), accuracy of DeepCOVID-XR was 82%, compared with that of individual radiologists (range, 76%-81%) and the consensus of all five radiologists (81%). DeepCOVID-XR had a significantly higher sensitivity (71%) than one radiologist (60%, P < .001) and significantly higher specificity (92%) than two radiologists (75%, P < .001; 84%, P = .009). AUC of DeepCOVID-XR was 0.88 compared with the consensus AUC of 0.85 (P = .13 for comparison). With consensus interpretation as the reference standard, the AUC of DeepCOVID-XR was 0.95 (95% CI: 0.92, 0.98). Conclusion DeepCOVID-XR, an artificial intelligence algorithm, detected coronavirus disease 2019 on chest radiographs with a performance similar to that of experienced thoracic radiologists in consensus. © RSNA, 2020 Supplemental material is available for this article. See also the editorial by van Ginneken in this issue.

Project description:ObjectiveTo evaluate whether the initial chest X-ray (CXR) severity assessed by an AI system may have prognostic utility in patients with COVID-19.MethodsThis retrospective single-center study included adult patients presenting to the emergency department (ED) between February 25 and April 9, 2020, with SARS-CoV-2 infection confirmed on real-time reverse transcriptase polymerase chain reaction (RT-PCR). Initial CXRs obtained on ED presentation were evaluated by a deep learning artificial intelligence (AI) system and compared with the Radiographic Assessment of Lung Edema (RALE) score, calculated by two experienced radiologists. Death and critical COVID-19 (admission to intensive care unit (ICU) or deaths occurring before ICU admission) were identified as clinical outcomes. Independent predictors of adverse outcomes were evaluated by multivariate analyses.ResultsSix hundred ninety-seven 697 patients were included in the study: 465 males (66.7%), median age of 62 years (IQR 52-75). Multivariate analyses adjusting for demographics and comorbidities showed that an AI system-based score ≥ 30 on the initial CXR was an independent predictor both for mortality (HR 2.60 (95% CI 1.69 - 3.99; p < 0.001)) and critical COVID-19 (HR 3.40 (95% CI 2.35-4.94; p < 0.001)). Other independent predictors were RALE score, older age, male sex, coronary artery disease, COPD, and neurodegenerative disease.ConclusionAI- and radiologist-assessed disease severity scores on CXRs obtained on ED presentation were independent and comparable predictors of adverse outcomes in patients with COVID-19.Trial registrationClinicalTrials.gov NCT04318366 ( https://clinicaltrials.gov/ct2/show/NCT04318366 ).Key points• AI system-based score ≥ 30 and a RALE score ≥ 12 at CXRs performed at ED presentation are independent and comparable predictors of death and/or ICU admission in COVID-19 patients. • Other independent predictors are older age, male sex, coronary artery disease, COPD, and neurodegenerative disease. • The comparable performance of the AI system in relation to a radiologist-assessed score in predicting adverse outcomes may represent a game-changer in resource-constrained settings.