Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Aggarwal R
PROVIDER: S-EPMC8027892 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:To perform a meta-analysis to compare the diagnostic test accuracy (DTA) of deep learning (DL) in detecting coronavirus disease 2019 (COVID-19), and to investigate how network architecture and type of datasets affect DL performance. We searched PubMed, Web of Science and Inspec from January 1, 2020, to December 3, 2020, for retrospective and prospective studies on deep learning detection with at least reported sensitivity and specificity. Pooled DTA was obtained using random-effect models. Sub-group analysis between studies was also carried out for data source and network architectures. The pooled sensitivity and specificity were 91% (95% confidence interval [CI]: 88%, 93%; I2 = 69%) and 92% (95% CI: 88%, 94%; I2 = 88%), respectively for 19 studies. The pooled AUC and diagnostic odds ratio (DOR) were 0.95 (95% CI: 0.88, 0.92) and 112.5 (95% CI: 57.7, 219.3; I2 = 90%) respectively. The overall accuracy, recall, F1-score, LR+ and LR- are 89.5%, 89.5%, 89.7%, 23.13 and 0.13. Sub-group analysis shows that the sensitivity and DOR significantly vary with the type of network architectures and sources of data with low heterogeneity are (I2 = 0%) and (I2 = 18%) for ResNet architecture and single-source datasets, respectively. The diagnosis of COVID-19 via deep learning has achieved incredible performance, and the source of datasets, as well as network architectures, strongly affect DL performance.
| S-EPMC8445811 | biostudies-literature
Project description:Tuberculosis (TB) remains one of the leading causes of death among infectious diseases worldwide. Early screening and diagnosis of pulmonary tuberculosis (PTB) is crucial in TB control, and tend to benefit from artificial intelligence. Here, we aimed to evaluate the diagnostic efficacy of a variety of artificial intelligence methods in medical imaging for PTB. We searched MEDLINE and Embase with the OVID platform to identify trials published update to November 2022 that evaluated the effectiveness of artificial-intelligence-based software in medical imaging of patients with PTB. After data extraction, the quality of studies was assessed using quality assessment of diagnostic accuracy studies 2 (QUADAS-2). Pooled sensitivity and specificity were estimated using a bivariate random-effects model. In total, 3987 references were initially identified and 61 studies were finally included, covering a wide range of 124,959 individuals. The pooled sensitivity and the specificity were 91% (95% confidence interval (CI), 89–93%) and 65% (54–75%), respectively, in clinical trials, and 94% (89–96%) and 95% (91–97%), respectively, in model-development studies. These findings have demonstrated that artificial-intelligence-based software could serve as an accurate tool to diagnose PTB in medical imaging. However, standardized reporting guidance regarding AI-specific trials and multicenter clinical trials is urgently needed to truly transform this cutting-edge technology into clinical practice.
| S-EPMC9820940 | biostudies-literature
Project description:ObjectiveTo evaluate the diagnostic accuracy of deep learning algorithms to identify age-related macular degeneration and to explore factors impacting the results for future model training.MethodsDiagnostic accuracy studies published in PubMed, EMBASE, the Cochrane Library, and ClinicalTrails.gov before 11 August 2022 which employed deep learning for age-related macular degeneration detection were identified and extracted by two independent researchers. Sensitivity analysis, subgroup, and meta-regression were performed by Review Manager 5.4.1, Meta-disc 1.4, and Stata 16.0. The risk of bias was assessed using QUADAS-2. The review was registered (PROSPERO CRD42022352753).ResultsThe pooled sensitivity and specificity in this meta-analysis were 94% (P = 0, 95% CI 0.94-0.94, I2 = 99.7%) and 97% (P = 0, 95% CI 0.97-0.97, I2 = 99.6%), respectively. The pooled positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, and the area under the curve value were 21.77(95% CI 15.49-30.59), 0.06 (95% CI 0.04-0.09), 342.41 (95% CI 210.31-557.49), and 0.9925, respectively. Meta-regression indicated that types of AMD (P = 0.1882, RDOR = 36.03) and layers of the network (P = 0.4878, RDOR = 0.74) contributed to the heterogeneity.ConclusionsConvolutional neural networks are mostly adopted deep learning algorithms in age-related macular degeneration detection. Convolutional neural networks, especially ResNets, are effective in detecting age-related macular degeneration with high diagnostic accuracy. Types of age-related macular degeneration and layers of the network are the two essential factors that impact the model training process. Proper layers of the network will make the model more reliable. More datasets established by new diagnostic methods will be used to train deep learning models in the future, which will benefit for fundus application screening, long-range medical treatment, and reducing the workload of physicians.
| S-EPMC10079062 | biostudies-literature
Project description:Introduction:The distinction between low-grade (grade 1) chondrosarcoma and its benign counterparts can be challenging. This systematic review aims to quantify the diagnostic accuracies of all functional imaging modalities used in the diagnosis of chondrosarcoma. Methods:Medline and Embase were searched in February 2019. We included studies of either retrospective or prospective design if the results of functional scans were compared with pre-determined reference standards. Studies had to be primary diagnostic reports on patients with chondral tumours at first diagnosis. Two review authors independently performed study selection, extracted data and assessed the methodological quality. We calculated diagnostic accuracy measures for each included study. Results:Four functional imaging modalities were identified across thirteen studies that met the inclusion criteria. 2-deoxy-2-[18F]fluoro-D-glucose positron emission tomography (FDG-PET) was a sensitive and specific test. Technetium-99 m with methylene diphosphonate (Tc-99 m MDP) had an overall low specificity of 4%. Thallium-201 scintigraphy demonstrated high positive predictive values across the studies. The negative predictive values of Technetium-99 m pentavalent dimercaptosuccinic acid (Tc-99 m DMSA (V)) were consistently 100%. Conclusions:Low-grade chondrosarcomas continue to pose a diagnostic dilemma. FDG-PET demonstrated superior diagnostic accuracy compared to Tc-99 m MDP, Thallium-201 and Tc-99 m DMSA (V). Characteristic uptake patterns of Thallium-201 and Tc-99 m DMSA (V) may provide additional metabolic information to guide the diagnosis in this challenging group of tumours.
| S-EPMC6811998 | biostudies-literature
Project description:ImportanceSystematic reviews of medical imaging diagnostic test accuracy (DTA) studies are affected by between-study heterogeneity due to a range of factors. Failure to appropriately assess the extent and causes of heterogeneity compromises the interpretability of systematic review findings.ObjectiveTo assess how heterogeneity has been examined in medical imaging DTA studies.Evidence reviewThe PubMed database was searched for systematic reviews of medical imaging DTA studies that performed a meta-analysis. The search was limited to the 40 journals with highest impact factor in the radiology, nuclear medicine, and medical imaging category in the InCites Journal Citation Reports of 2021 to reach a sample size of 200 to 300 included studies. Descriptive analysis was performed to characterize the imaging modality, target condition, type of meta-analysis model used, strategies for evaluating heterogeneity, and sources of heterogeneity identified. Multivariable logistic regression was performed to assess whether any factors were associated with at least 1 source of heterogeneity being identified in the included meta-analyses. Methodological quality evaluation was not performed. Data analysis occurred from October to December 2022.FindingsA total of 242 meta-analyses involving a median (range) of 987 (119-441 510) patients across a diverse range of disease categories and imaging modalities were included. The extent of heterogeneity was adequately described (ie, whether it was absent, low, moderate, or high) in 220 studies (91%) and was most commonly assessed using the I2 statistic (185 studies [76%]) and forest plots (181 studies [75%]). Heterogeneity was rated as moderate to high in 191 studies (79%). Of all included meta-analyses, 122 (50%) performed subgroup analysis and 87 (36%) performed meta-regression. Of the 242 studies assessed, 189 (78%) included 10 or more primary studies. Of these 189 studies, 60 (32%) did not perform meta-regression or subgroup analysis. Reasons for being unable to investigate sources of heterogeneity included inadequate reporting of primary study characteristics and a low number of included primary studies. Use of meta-regression was associated with identification of at least 1 source of variability (odds ratio, 1.90; 95% CI, 1.11-3.23; P = .02).Conclusions and relevanceIn this systematic review of assessment of heterogeneity in medical imaging DTA meta-analyses, most meta-analyses were impacted by a moderate to high level of heterogeneity, presenting interpretive challenges. These findings suggest that, despite the development and availability of more rigorous statistical models, heterogeneity appeared to be incomplete, inconsistently evaluated, or methodologically questionable in many cases, which lessened the interpretability of the analyses performed; comprehensive heterogeneity assessment should be addressed at the author level by improving personal familiarity with appropriate statistical methodology for assessing heterogeneity and involving biostatisticians and epidemiologists in study design, as well as at the editorial level, by mandating adherence to methodologic standards in primary DTA studies and DTA meta-analyses.
| S-EPMC10905313 | biostudies-literature
Project description:BackgroundCoronary artery disease (CAD) is a leading cause of death worldwide, and the diagnostic process comprises of invasive testing with coronary angiography and non-invasive imaging, in addition to history, clinical examination, and electrocardiography (ECG). A highly accurate assessment of CAD lies in perfusion imaging which is performed by myocardial perfusion scintigraphy (MPS) and magnetic resonance imaging (stress CMR). Recently deep learning has been increasingly applied on perfusion imaging for better understanding of the diagnosis, safety, and outcome of CAD.The aim of this review is to summarise the evidence behind deep learning applications in myocardial perfusion imaging.MethodsA systematic search was performed on MEDLINE and EMBASE databases, from database inception until September 29, 2020. This included all clinical studies focusing on deep learning applications and myocardial perfusion imaging, and excluded competition conference papers, simulation and animal studies, and studies which used perfusion imaging as a variable with different focus. This was followed by review of abstracts and full texts. A meta-analysis was performed on a subgroup of studies which looked at perfusion images classification. A summary receiver-operating curve (SROC) was used to compare the performance of different models, and area under the curve (AUC) was reported. Effect size, risk of bias and heterogeneity were tested.Results46 studies in total were identified, the majority were MPS studies (76%). The most common neural network was convolutional neural network (CNN) (41%). 13 studies (28%) looked at perfusion imaging classification using MPS, the pooled diagnostic accuracy showed AUC = 0.859. The summary receiver operating curve (SROC) comparison showed superior performance of CNN (AUC = 0.894) compared to MLP (AUC = 0.848). The funnel plot was asymmetrical, and the effect size was significantly different with p value < 0.001, indicating small studies effect and possible publication bias. There was no significant heterogeneity amongst studies according to Q test (p = 0.2184).ConclusionDeep learning has shown promise to improve myocardial perfusion imaging diagnostic accuracy, prediction of patients' events and safety. More research is required in clinical applications, to achieve better care for patients with known or suspected CAD.
| S-EPMC9514037 | biostudies-literature
Project description:ObjectivesAccurate subtype classification in primary aldosteronism (PA) is critical in assessing the optimal treatment options. This study aimed to evaluate the diagnostic accuracy of adrenal imaging for unilateral PA classification.MethodsSystematic searches of PubMed, EMBASE and the Cochrane databases were performed from 1 January 2000 to 1 February 2020, for all studies that used CT or MRI in determining unilateral PA and validated the results against invasive adrenal vein sampling (AVS). Summary diagnostic accuracies were assessed using a bivariate random-effects model. Subgroup analyses, meta-regression and sensitivity analysis were performed to explore the possible sources of heterogeneity.ResultA total of 25 studies, involving a total of 4669 subjects, were identified. The overall analysis revealed a pooled sensitivity of 68% (95% CI: 61% to 74%) and specificity of 57% (95% CI 50% to 65%) for CT/MRI in identifying unilateral PA. Sensitivity was higher in the contrast-enhanced (CT) group versus the traditional CT group (77% (95% CI 66% to 85%) vs 58% (95% CI 50% to 66%). Subgroup analysis stratified by screening test for PA showed that the sensitivity of the aldosterone-to-renin ratio (ARR) group was higher than that of the non-ARR group (78% (95% CI 69% to 84%) vs 66% (95% CI 58% to 72%)). The diagnostic accuracy of PA patients aged ≤40 years was reported in four studies, and the overall sensitivity was 71%, with 79% specificity. Meta-regression revealed a significant impact of sample size on sensitivity and of age and study quality on specificity.ConclusionCT/MRI is not a reliable alternative to invasive AVS without excellent sensitivity or specificity for correctly identifying unilateral PA. Even in young patients (≤40 years), 21% of patients would have undergone unnecessary adrenalectomy based on imaging results alone.
| S-EPMC7780716 | biostudies-literature
Project description:PurposeEarly clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis.MethodsA systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance.ResultsAfter screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance.ConclusionThis systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.
| S-EPMC7067741 | biostudies-literature
Project description:AimAs the completed studies have small sample sizes and different algorithms, a meta-analysis was conducted to assess the accuracy of WCE in identifying polyps using deep learning.MethodTwo independent reviewers searched PubMed, Embase, the Web of Science, and the Cochrane Library for potentially eligible studies published up to December 8, 2021, which were analysed on a per-image basis. STATA RevMan and Meta-DiSc were used to conduct this meta-analysis. A random effects model was used, and a subgroup and regression analysis was performed to explore sources of heterogeneity.ResultsEight studies published between 2017 and 2021 included 819 patients, and 18,414 frames were eventually included in the meta-analysis. The summary estimates for the WCE in identifying polyps by deep learning were sensitivity 0.97 (95% confidence interval (CI), 0.95-0.98); specificity 0.97 (95% CI, 0.94-0.98); positive likelihood ratio 27.19 (95% CI, 15.32-50.42); negative likelihood ratio 0.03 (95% CI 0.02-0.05); diagnostic odds ratio 873.69 (95% CI, 387.34-1970.74); and the area under the sROC curve 0.99.ConclusionWCE uses deep learning to identify polyps with high accuracy, but multicentre prospective randomized controlled studies are needed in the future.
| S-EPMC9159236 | biostudies-literature
Project description:BackgroundCardiovascular disease is the most common cause of death after kidney transplantation. Coronary artery disease (CAD) assessment is therefore mandatory in patients evaluated for transplantation. We aimed to assess the diagnostic accuracy for CAD of single-photon emission computed tomography (SPECT) compared to the standards invasive coronary angiography (ICA) and coronary computed tomography angiography (CCTA) in patients evaluated for kidney transplantation.MethodsWe performed a systematic literature search in PubMed, EMBASE, Web of Science, OvidSP (Medline), The Cochrane Library and Google Scholar. Studies investigating the diagnostic accuracy of myocardial perfusion imaging (MPI) SPECT in patients evaluated for kidney transplantation were retrieved. After a risk of bias assessment using QUADAS-2, a meta-analysis was conducted.ResultsOut of 1459 records, 13 MPI SPECT studies were included in the meta-analysis with a total of 1245 MPI SPECT scans. There were no studies available with CCTA as reference. Pooled sensitivity of MPI SPECT for CAD was 0.66 (95% CI 0.53 to 0.77), pooled specificity was 0.75 (95% CI 0.63 to 0.84) and the area under the curve (AUC) was 0.76. Positive likelihood ratio was 2.50 (95% CI 1.78 to 3.51) and negative likelihood ratio was 0.41 (95% CI 0.28 to 0.61). Pooled positive predictive value was 64.9% and pooled negative predictive value was 74.1%. Significant heterogeneity existed across the included studies.ConclusionsMPI SPECT had a moderate diagnostic accuracy in patients evaluated for kidney transplantation, with a high rate of false-negative findings. The use of an anatomical gold standard against a functional imaging test in the included studies is however suboptimal.
| S-EPMC9834355 | biostudies-literature