Diagnostic Test Accuracy of Deep Learning Detection of COVID-19: A Systematic Review and Meta-Analysis.
Ontology highlight
ABSTRACT: Rationale and objective
To perform a meta-analysis to compare the diagnostic test accuracy (DTA) of deep learning (DL) in detecting coronavirus disease 2019 (COVID-19), and to investigate how network architecture and type of datasets affect DL performance.Materials and methods
We searched PubMed, Web of Science and Inspec from January 1, 2020, to December 3, 2020, for retrospective and prospective studies on deep learning detection with at least reported sensitivity and specificity. Pooled DTA was obtained using random-effect models. Sub-group analysis between studies was also carried out for data source and network architectures.Results
The pooled sensitivity and specificity were 91% (95% confidence interval [CI]: 88%, 93%; I2 = 69%) and 92% (95% CI: 88%, 94%; I2 = 88%), respectively for 19 studies. The pooled AUC and diagnostic odds ratio (DOR) were 0.95 (95% CI: 0.88, 0.92) and 112.5 (95% CI: 57.7, 219.3; I2 = 90%) respectively. The overall accuracy, recall, F1-score, LR+ and LR- are 89.5%, 89.5%, 89.7%, 23.13 and 0.13. Sub-group analysis shows that the sensitivity and DOR significantly vary with the type of network architectures and sources of data with low heterogeneity are (I2 = 0%) and (I2 = 18%) for ResNet architecture and single-source datasets, respectively.Conclusion
The diagnosis of COVID-19 via deep learning has achieved incredible performance, and the source of datasets, as well as network architectures, strongly affect DL performance.
SUBMITTER: Komolafe TE
PROVIDER: S-EPMC8445811 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectiveTo evaluate the diagnostic accuracy of deep learning algorithms to identify age-related macular degeneration and to explore factors impacting the results for future model training.MethodsDiagnostic accuracy studies published in PubMed, EMBASE, the Cochrane Library, and ClinicalTrails.gov before 11 August 2022 which employed deep learning for age-related macular degeneration detection were identified and extracted by two independent researchers. Sensitivity analysis, subgroup, and meta-regression were performed by Review Manager 5.4.1, Meta-disc 1.4, and Stata 16.0. The risk of bias was assessed using QUADAS-2. The review was registered (PROSPERO CRD42022352753).ResultsThe pooled sensitivity and specificity in this meta-analysis were 94% (P = 0, 95% CI 0.94-0.94, I2 = 99.7%) and 97% (P = 0, 95% CI 0.97-0.97, I2 = 99.6%), respectively. The pooled positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, and the area under the curve value were 21.77(95% CI 15.49-30.59), 0.06 (95% CI 0.04-0.09), 342.41 (95% CI 210.31-557.49), and 0.9925, respectively. Meta-regression indicated that types of AMD (P = 0.1882, RDOR = 36.03) and layers of the network (P = 0.4878, RDOR = 0.74) contributed to the heterogeneity.ConclusionsConvolutional neural networks are mostly adopted deep learning algorithms in age-related macular degeneration detection. Convolutional neural networks, especially ResNets, are effective in detecting age-related macular degeneration with high diagnostic accuracy. Types of age-related macular degeneration and layers of the network are the two essential factors that impact the model training process. Proper layers of the network will make the model more reliable. More datasets established by new diagnostic methods will be used to train deep learning models in the future, which will benefit for fundus application screening, long-range medical treatment, and reducing the workload of physicians.
| S-EPMC10079062 | biostudies-literature
Project description:BackgroundThe global COVID-19 pandemic has led to extensive development in many fields, including the diagnosis of COVID-19 infection by mass spectrometry. The aim of this systematic review and meta-analysis was to assess the accuracy of mass spectrometry diagnostic tests developed so far, across a wide range of biological matrices, and additionally to assess risks of bias and applicability in studies published to date.Method23 retrospective observational cohort studies were included in the systematic review using the PRISMA-DTA framework, with a total of 2858 COVID-19 positive participants and 2544 controls. Risks of bias and applicability were assessed via a QUADAS-2 questionnaire. A meta-analysis was also performed focusing on sensitivity, specificity, diagnostic accuracy and Youden's Index, in addition to assessing heterogeneity.FindingsSensitivity averaged 0.87 in the studies reviewed herein (interquartile range 0.81-0.96) and specificity 0.88 (interquartile range 0.82-0.98), with an area under the receiver operating characteristic summary curve of 0.93. By subgroup, the best diagnostic results were achieved by viral proteomic analyses of nasopharyngeal swabs and metabolomic analyses of plasma and serum. The performance of other sampling matrices (breath, sebum, saliva) was less good, indicating that these protocols are currently insufficiently mature for clinical application.ConclusionsThis systematic review and meta-analysis demonstrates the potential for mass spectrometry and 'omics in achieving accurate test results for COVID-19 diagnosis, but also highlights the need for further work to optimize and harmonize practice across laboratories before these methods can be translated to clinical applications.
| S-EPMC8548837 | biostudies-literature
Project description:BackgroundThis systematic review and meta-analysis were conducted to objectively evaluate the evidence of machine learning (ML) in the patient diagnosis of Intracranial Hemorrhage (ICH) on computed tomography (CT) scans.MethodsUntil May 2023, systematic searches were conducted in ISI Web of Science, PubMed, Scopus, Cochrane Library, IEEE Xplore Digital Library, CINAHL, Science Direct, PROSPERO, and EMBASE for studies that evaluated the diagnostic precision of ML model-assisted ICH detection. Patients with and without ICH as the target condition who were receiving CT-Scan were eligible for the research, which used ML algorithms based on radiologists' reports as the gold reference standard. For meta-analysis, pooled sensitivities, specificities, and a summary receiver operating characteristics curve (SROC) were used.ResultsAt last, after screening the title, abstract, and full paper, twenty-six retrospective and three prospective, and two retrospective/prospective studies were included. The overall (Diagnostic Test Accuracy) DTA of retrospective studies with a pooled sensitivity was 0.917 (95% CI 0.88-0.943, I2 = 99%). The pooled specificity was 0.945 (95% CI 0.918-0.964, I2 = 100%). The pooled diagnostic odds ratio (DOR) was 219.47 (95% CI 104.78-459.66, I2 = 100%). These results were significant for the specificity of the different network architecture models (p-value = 0.0289). However, the results for sensitivity (p-value = 0.6417) and DOR (p-value = 0.2187) were not significant. The ResNet algorithm has higher pooled specificity than other algorithms with 0.935 (95% CI 0.854-0.973, I2 = 93%).ConclusionThis meta-analysis on DTA of ML algorithms for detecting ICH by assessing non-contrast CT-Scans shows the ML has an acceptable performance in diagnosing ICH. Using ResNet in ICH detection remains promising prediction was improved via training in an Architecture Learning Network (ALN).
| S-EPMC10694901 | biostudies-literature
Project description:Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC's ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC's ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC's ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.
| S-EPMC8027892 | biostudies-literature
Project description:AimsThis meta-analysis aimed to assess the prognostic value of fasting hyperglycemia in patients with COVID-19.MethodsA systematic literature search on PubMed, Embase, and Scopus were performed up until February 18, 2021. Fasting hyperglycemia was defined as fasting plasma glucose level above the reference value. The outcome of interest was poor outcome, which was a composite of mortality and severe COVID-19. The effect estimate was in odds ratio (OR).ResultsThere were 9045 patients from 12 studies included in this systematic review and meta-analysis. The prevalence of fasting hyperglycemia was 29%. The incidence of poor outcome was 15%. Fasting hyperglycemia was associated with poor outcome in COVID-19 (OR 4.72 [3.32, 6.72], p < 0.001; I2: 69.8%, p < 0.001). Subgroup analysis in patients without prior history of diabetes showed that fasting hyperglycemia was associated with poor outcome in COVID-19 (OR 3.387 [2.433, 4.714], p < 0.001; I2: 0, p = 0.90). Fasting hyperglycemia has a sensitivity of 0.57 [0.45, 0.68], specificity of 0.78 [0.70, 0.84], PLR of 2.6 [2.0, 3.3], NLR of 0.55 [0.44, 0.69], DOR of 5 [3, 7], and AUC of 0.74 [0.70, 0.78] for predicting poor outcome. In this pooled analysis, fasting hyperglycemia has a 32% post-test probability for poor outcome, and absence of fasting hyperglycemia confers to a 9% post-test probability. Meta-regression and subgroup analysis showed that the sensitivity and specificity varies by chronic kidney disease but not by age, male (gender), hypertension, and chronic kidney disease.ConclusionFasting hyperglycemia was associated with mortality in COVID-19 patients, with or without diabetes.ProsperoCRD42021237997.
| S-EPMC8019489 | biostudies-literature
Project description:The accuracy of commercially available tests for COVID-19 in Brazil remains unclear. We aimed to perform a meta-analysis to describe the accuracy of available tests to detect COVID-19 in Brazil. We searched at the Brazilian Health Regulatory Agency (ANVISA) online platform to describe the pooled sensitivity (Se), specificity (Sp), diagnostic odds ratio (DOR) and summary receiver operating characteristic curves (SROC) for detection of IgM/IgG antibodies and for tests using naso/oropharyngeal swabs in the random-effects models. We identified 16 tests registered, mostly rapid-tests. Pooled diagnostic accuracy measures [95%CI] were: (i) for IgM antibodies Se=82% [76-87]; Sp=97% [96-98]; DOR=168 [92-305] and SROC=0.98 [0.96-0.99]; (ii) for IgG antibodies Se=97% [90-99]; Sp=98% [97-99]; DOR=1994 [385-10334] and SROC=0.99 [0.98-1.00]; and (iii) for detection of SARS-CoV-2 by antigen or molecular assays in naso/oropharyngeal swabs Se=97% [85-99]; Sp=99% [77-100]; DOR=2649 [30-233056] and SROC=0.99 [0.98-1.00]. These tests can be helpful for emergency testing during the COVID-19 pandemic in Brazil. However, it is important to highlight the high rate of false negative results from tests which detect SARS-CoV-2 IgM antibodies in the initial course of the disease and the scarce evidence-based validation results published in Brazil. Future studies addressing the diagnostic performance of tests for COVID-19 in the Brazilian population are urgently needed.
| S-EPMC7165277 | biostudies-literature
Project description:Melanoma has the highest mortality rate among skin cancers, and early-diagnosis is essential to maximize survival rate. The current procedure for melanoma diagnosis is based on dermoscopy, i.e., a qualitative visual inspection of lesions with intrinsic limited diagnostic reliability and reproducibility. Other non-invasive diagnostic techniques may represent valuable solutions to retrieve additional objective information of a lesion. This review aims to compare the diagnostic performance of non-invasive techniques, alternative to dermoscopy, for melanoma detection in clinical settings. A systematic review of the available literature was performed using PubMed, Scopus and Google scholar databases (2010-September 2020). All human, in-vivo, non-invasive studies using techniques, alternative to dermoscopy, for melanoma diagnosis were included with no restriction on the recruited population. The reference standard was histology but dermoscopy was accepted only in case of benign lesions. Attributes of the analyzed studies were compared, and the quality was evaluated using CASP Checklist. For studies in which the investigated technique was implemented as a diagnostic tool (DTA studies), the QUADAS-2 tool was applied. For DTA studies that implemented a melanoma vs. other skin lesions classification task, a meta-analysis was performed reporting the SROC curves. Sixty-two references were included in the review, of which thirty-eight were analyzed using QUADAS-2. Study designs were: clinical trials (13), retrospective studies (10), prospective studies (8), pilot studies (10), multitiered study (1); the remain studies were proof of concept or had undefined study type. Studies were divided in categories based on the physical principle employed by each diagnostic technique. Twenty-nine out of thirty-eight DTA studies were included in the meta-analysis. Heterogeneity of studies' types, testing strategy, and diagnostic task limited the systematic comparison of the techniques. Based on the SROC curves, spectroscopy achieved the best performance in terms of sensitivity (93%, 95% CI 92.8-93.2%) and specificity (85.2%, 95%CI 84.9-85.5%), even though there was high concern regarding robustness of metrics. Reflectance-confocal-microscopy, instead, demonstrated higher robustness and a good diagnostic performance (sensitivity 88.2%, 80.3-93.1%; specificity 65.2%, 55-74.2%). Best practice recommendations were proposed to reduce bias in future DTA studies. Particular attention should be dedicated to widen the use of alternative techniques to conventional dermoscopy.
| S-EPMC8103840 | biostudies-literature
Project description:ObjectiveTo determine the diagnostic accuracy of serological tests for coronavirus disease-2019 (covid-19).DesignSystematic review and meta-analysis.Data sourcesMedline, bioRxiv, and medRxiv from 1 January to 30 April 2020, using subject headings or subheadings combined with text words for the concepts of covid-19 and serological tests for covid-19.Eligibility criteria and data analysisEligible studies measured sensitivity or specificity, or both of a covid-19 serological test compared with a reference standard of viral culture or reverse transcriptase polymerase chain reaction. Studies were excluded with fewer than five participants or samples. Risk of bias was assessed using quality assessment of diagnostic accuracy studies 2 (QUADAS-2). Pooled sensitivity and specificity were estimated using random effects bivariate meta-analyses.Main outcome measuresThe primary outcome was overall sensitivity and specificity, stratified by method of serological testing (enzyme linked immunosorbent assays (ELISAs), lateral flow immunoassays (LFIAs), or chemiluminescent immunoassays (CLIAs)) and immunoglobulin class (IgG, IgM, or both). Secondary outcomes were stratum specific sensitivity and specificity within subgroups defined by study or participant characteristics, including time since symptom onset.Results5016 references were identified and 40 studies included. 49 risk of bias assessments were carried out (one for each population and method evaluated). High risk of patient selection bias was found in 98% (48/49) of assessments and high or unclear risk of bias from performance or interpretation of the serological test in 73% (36/49). Only 10% (4/40) of studies included outpatients. Only two studies evaluated tests at the point of care. For each method of testing, pooled sensitivity and specificity were not associated with the immunoglobulin class measured. The pooled sensitivity of ELISAs measuring IgG or IgM was 84.3% (95% confidence interval 75.6% to 90.9%), of LFIAs was 66.0% (49.3% to 79.3%), and of CLIAs was 97.8% (46.2% to 100%). In all analyses, pooled sensitivity was lower for LFIAs, the potential point-of-care method. Pooled specificities ranged from 96.6% to 99.7%. Of the samples used for estimating specificity, 83% (10 465/12 547) were from populations tested before the epidemic or not suspected of having covid-19. Among LFIAs, pooled sensitivity of commercial kits (65.0%, 49.0% to 78.2%) was lower than that of non-commercial tests (88.2%, 83.6% to 91.3%). Heterogeneity was seen in all analyses. Sensitivity was higher at least three weeks after symptom onset (ranging from 69.9% to 98.9%) compared with within the first week (from 13.4% to 50.3%).ConclusionHigher quality clinical studies assessing the diagnostic accuracy of serological tests for covid-19 are urgently needed. Currently, available evidence does not support the continued use of existing point-of-care serological tests.Study registrationPROSPERO CRD42020179452.
| S-EPMC7327913 | biostudies-literature
Project description:AimsTo validate the diagnostic accuracy of the Augurix SARS-CoV-2 IgM/IgG rapid immunoassay diagnostic test (RDT) for COVID-19.MethodsIn this unmatched 1:1 case-control study, blood samples from 46 real-time RT-PCR-confirmed SARS-CoV-2 hospitalized cases and 45 healthy donors (negative controls) were studied. Diagnostic accuracy of the IgG RDT was assessed against both an in-house recombinant spike-expressing immunofluorescence assay (rIFA), as an established reference method (primary endpoint), and the Euroimmun SARS-CoV-2 IgG enzyme-linked immunosorbent assays (ELISA) (secondary endpoint).ResultsCOVID-19 patients were more likely to be male (61% vs 20%; P = .0001) and older (median 66 vs 47 years old; P < .001) than controls. Whole blood IgG-RDT results showed 86% and 93% overall Kendall concordance with rIFA and IgG ELISA, respectively. IgG RDT performances were similar between plasma and whole blood. Overall, RDT sensitivity was 88% (95% confidence interval [95%CI]: 70-96), specificity 98% (95%CI: 90-100), PPV 97% (95%CI: 80-100) and NPV 94% (95%CI: 84-98). The IgG-RDT carried out from 0 to 6 days, 7 to 14 days and > 14 days after the SARS-CoV-2 RT-PCR test displayed 30%, 73% and 100% positivity rates in the COVID-19 group, respectively. When considering samples taken >14 days after RT-PCR diagnosis, NPV was 100% (95%CI:90-100), and PPV was 100% (95%CI:72-100).ConclusionsThe Augurix IgG-RDT done in whole blood displays a high diagnostic accuracy for SARS-CoV-2 IgG in high COVID-19 prevalence settings, where its use could be considered in the absence of routine diagnostic serology facilities.
| S-EPMC7404380 | biostudies-literature
Project description:PurposeEarly clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis.MethodsA systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance.ResultsAfter screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance.ConclusionThis systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.
| S-EPMC7067741 | biostudies-literature