Dataset Information

Automated COVID-19 diagnosis and prognosis with medical imaging and who is publishing: a systematic review.

ABSTRACT:

Objectives

To conduct a systematic survey of published techniques for automated diagnosis and prognosis of COVID-19 diseases using medical imaging, assessing the validity of reported performance and investigating the proposed clinical use-case. To conduct a scoping review into the authors publishing such work.

Methods

The Scopus database was queried and studies were screened for article type, and minimum source normalized impact per paper and citations, before manual relevance assessment and a bias assessment derived from a subset of the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). The number of failures of the full CLAIM was adopted as a surrogate for risk-of-bias. Methodological and performance measurements were collected from each technique. Each study was assessed by one author. Comparisons were evaluated for significance with a two-sided independent t-test.

Findings

Of 1002 studies identified, 390 remained after screening and 81 after relevance and bias exclusion. The ratio of exclusion for bias was 71%, indicative of a high level of bias in the field. The mean number of CLAIM failures per study was 8.3 ± 3.9 [1,17] (mean ± standard deviation [min,max]). 58% of methods performed diagnosis versus 31% prognosis. Of the diagnostic methods, 38% differentiated COVID-19 from healthy controls. For diagnostic techniques, area under the receiver operating curve (AUC) = 0.924 ± 0.074 [0.810,0.991] and accuracy = 91.7% ± 6.4 [79.0,99.0]. For prognostic techniques, AUC = 0.836 ± 0.126 [0.605,0.980] and accuracy = 78.4% ± 9.4 [62.5,98.0]. CLAIM failures did not correlate with performance, providing confidence that the highest results were not driven by biased papers. Deep learning techniques reported higher AUC (p < 0.05) and accuracy (p < 0.05), but no difference in CLAIM failures was identified.

Interpretation

A majority of papers focus on the less clinically impactful diagnosis task, contrasted with prognosis, with a significant portion performing a clinically unnecessary task of differentiating COVID-19 from healthy. Authors should consider the clinical scenario in which their work would be deployed when developing techniques. Nevertheless, studies report superb performance in a potentially impactful application. Future work is warranted in translating techniques into clinical tools.

SUBMITTER: Gillman AG

PROVIDER: S-EPMC8678975 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundAccurate and timely diagnosis and effective prognosis of the disease is important to provide the best possible care for patients with COVID-19 and reduce the burden on the health care system. Machine learning methods can play a vital role in the diagnosis of COVID-19 by processing chest x-ray images.ObjectiveThe aim of this study is to summarize information on the use of intelligent models for the diagnosis and prognosis of COVID-19 to help with early and timely diagnosis, minimize prolonged diagnosis, and improve overall health care.MethodsA systematic search of databases, including PubMed, Web of Science, IEEE, ProQuest, Scopus, bioRxiv, and medRxiv, was performed for COVID-19-related studies published up to May 24, 2020. This study was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. All original research articles describing the application of image processing for the prediction and diagnosis of COVID-19 were considered in the analysis. Two reviewers independently assessed the published papers to determine eligibility for inclusion in the analysis. Risk of bias was evaluated using the Prediction Model Risk of Bias Assessment Tool.ResultsOf the 629 articles retrieved, 44 articles were included. We identified 4 prognosis models for calculating prediction of disease severity and estimation of confinement time for individual patients, and 40 diagnostic models for detecting COVID-19 from normal or other pneumonias. Most included studies used deep learning methods based on convolutional neural networks, which have been widely used as a classification algorithm. The most frequently reported predictors of prognosis in patients with COVID-19 included age, computed tomography data, gender, comorbidities, symptoms, and laboratory findings. Deep convolutional neural networks obtained better results compared with non-neural network-based methods. Moreover, all of the models were found to be at high risk of bias due to the lack of information about the study population, intended groups, and inappropriate reporting.ConclusionsMachine learning models used for the diagnosis and prognosis of COVID-19 showed excellent discriminative performance. However, these models were at high risk of bias, because of various reasons such as inadequate information about study participants, randomization process, and the lack of external validation, which may have resulted in the optimistic reporting of these models. Hence, our findings do not recommend any of the current models to be used in practice for the diagnosis and prognosis of COVID-19.

Project description:BackgroundAirspace disease as seen on chest X-rays is an important point in triage for patients initially presenting to the emergency department with suspected COVID-19 infection. The purpose of this study is to evaluate a previously trained interpretable deep learning algorithm for the diagnosis and prognosis of COVID-19 pneumonia from chest X-rays obtained in the ED.MethodsThis retrospective study included 2456 (50% RT-PCR positive for COVID-19) adult patients who received both a chest X-ray and SARS-CoV-2 RT-PCR test from January 2020 to March of 2021 in the emergency department at a single U.S.InstitutionA total of 2000 patients were included as an additional training cohort and 456 patients in the randomized internal holdout testing cohort for a previously trained Siemens AI-Radiology Companion deep learning convolutional neural network algorithm. Three cardiothoracic fellowship-trained radiologists systematically evaluated each chest X-ray and generated an airspace disease area-based severity score which was compared against the same score produced by artificial intelligence. The interobserver agreement, diagnostic accuracy, and predictive capability for inpatient outcomes were assessed. Principal statistical tests used in this study include both univariate and multivariate logistic regression.ResultsOverall ICC was 0.820 (95% CI 0.790-0.840). The diagnostic AUC for SARS-CoV-2 RT-PCR positivity was 0.890 (95% CI 0.861-0.920) for the neural network and 0.936 (95% CI 0.918-0.960) for radiologists. Airspace opacities score by AI alone predicted ICU admission (AUC = 0.870) and mortality (0.829) in all patients. Addition of age and BMI into a multivariate log model improved mortality prediction (AUC = 0.906).ConclusionThe deep learning algorithm provides an accurate and interpretable assessment of the disease burden in COVID-19 pneumonia on chest radiographs. The reported severity scores correlate with expert assessment and accurately predicts important clinical outcomes. The algorithm contributes additional prognostic information not currently incorporated into patient management.

Project description:Evidence for the use of automated or partly automated contact-tracing tools to contain severe acute respiratory syndrome coronavirus 2 is scarce. We did a systematic review of automated or partly automated contact tracing. We searched PubMed, EMBASE, OVID Global Health, EBSCO Medical COVID Information Portal, Cochrane Library, medRxiv, bioRxiv, arXiv, and Google Advanced for articles relevant to COVID-19, severe acute respiratory syndrome, Middle East respiratory syndrome, influenza, or Ebola virus, published from Jan 1, 2000, to April 14, 2020. We also included studies identified through professional networks up to April 30, 2020. We reviewed all full-text manuscripts. Primary outcomes were the number or proportion of contacts (or subsequent cases) identified. Secondary outcomes were indicators of outbreak control, uptake, resource use, cost-effectiveness, and lessons learnt. This study is registered with PROSPERO (CRD42020179822). Of the 4036 studies identified, 110 full-text studies were reviewed and 15 studies were included in the final analysis and quality assessment. No empirical evidence of the effectiveness of automated contact tracing (regarding contacts identified or transmission reduction) was identified. Four of seven included modelling studies that suggested that controlling COVID-19 requires a high population uptake of automated contact-tracing apps (estimates from 56% to 95%), typically alongside other control measures. Studies of partly automated contact tracing generally reported more complete contact identification and follow-up compared with manual systems. Automated contact tracing could potentially reduce transmission with sufficient population uptake. However, concerns regarding privacy and equity should be considered. Well designed prospective studies are needed given gaps in evidence of effectiveness, and to investigate the integration and relative effects of manual and automated systems. Large-scale manual contact tracing is therefore still key in most contexts.

Project description:Chest computed tomography (CT) is frequently used in diagnosing coronavirus disease 2019 (COVID-19) for detecting abnormal changes in the lungs and monitoring disease progression during the treatment process. Furthermore, CT imaging appearances are correlated with patients presenting with different clinical scenarios, such as early versus advanced stages, asymptomatic versus symptomatic patients, and severe versus nonsevere situations. However, its role as a screening and diagnostic tool in COVID-19 remains to be clarified. This article provides a systematic review and meta-analysis of the current literature on chest CT imaging findings with the aim of highlighting the contribution and judicious use of CT in the diagnosis of COVID-19. A search of PubMed/Medline, Web of Science, ScienceDirect, Google Scholar and Scopus was performed to identify studies reporting chest imaging findings in COVID-19. Chest imaging abnormalities associated with COVID-19 were extracted from the eligible studies and diagnostic value of CT in detecting these abnormal changes was compared between studies consisting of both COVID-19 and non-COVID-19 patients. A random-effects model was used to perform meta-analysis for calculation of pooled mean values and 95% confidence intervals (95% CI) of abnormal imaging findings. Fifty-five studies met the selection criteria and were included in the analysis. Pulmonary lesions more often involved bilateral lungs (78%, 95% CI: 45-100%) and were more likely to have a peripheral (65.35%, 95% CI: 25.93-100%) and peripheral plus central distribution (31.12%, 95% CI: 1.96-74.07%), but less likely to have a central distribution (3.57%, 95% CI: 0.99-9.80%). Ground glass opacities (GGO) (58.05%, 95% CI: 16.67-100%), consolidation (44.18%, 95% CI: 1.61-71.46%) and GGO plus consolidation (52.99%, 95% CI: 19.05-76.79%) were the most common findings reported in 94.5% (52/55) of the studies, followed by air bronchogram (42.50%, 95% CI: 7.78-80.39%), linear opacities (41.29%, 95% CI: 7.44-65.06%), crazy-paving pattern (23.57%, 95% CI: 3.13-91.67%) and interlobular septal thickening (22.91%, 95% CI: 0.90-80.49%). CT has low specificity in differentiating pneumonia-related lung changes due to significant overlap between COVID-19 and non-COVID-19 patients with no significant differences in most of the imaging findings between these two groups (P>0.05). Furthermore, normal CT (13.31%, 95% CI: 0.74-38.36%) was reported in 26 (47.3%) studies. Despite widespread use of CT in the diagnosis of COVID-19 patients based on the current literature, CT findings are not pathognomonic as it lacks specificity in differentiating imaging appearances caused by different types of pneumonia. Further, there is a relatively high percentage of normal CT scans. Use of CT as a first-line diagnostic or screening tool in COVID-19 is not recommended.