Dataset Information

Systematic review protocol to assess artificial intelligence diagnostic accuracy performance in detecting acute ischaemic stroke and large-vessel occlusions on CT and MR medical imaging.

ABSTRACT:

Introduction

The use of artificial intelligence (AI) to support the diagnosis of acute ischaemic stroke (AIS) could improve patient outcomes and facilitate accurate tissue and vessel assessment. However, the evidence in published AI studies is inadequate and difficult to interpret which reduces the accountability of the diagnostic results in clinical settings. This study protocol describes a rigorous systematic review of the accuracy of AI in the diagnosis of AIS and detection of large-vessel occlusions (LVOs).

Methods and analysis

We will perform a systematic review and meta-analysis of the performance of AI models for diagnosing AIS and detecting LVOs. We will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols guidelines. Literature searches will be conducted in eight databases. For data screening and extraction, two reviewers will use a modified Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist. We will assess the included studies using the Quality Assessment of Diagnostic Accuracy Studies guidelines. We will conduct a meta-analysis if sufficient data are available. We will use hierarchical summary receiver operating characteristic curves to estimate the summary operating points, including the pooled sensitivity and specificity, with 95% CIs, if pooling is appropriate. Furthermore, if sufficient data are available, we will use Grading of Recommendations, Assessment, Development and Evaluations profiler software to summarise the main findings of the systematic review, as a summary of results.

Ethics and dissemination

There are no ethical considerations associated with this study protocol, as the systematic review focuses on the examination of secondary data. The systematic review results will be used to report on the accuracy, completeness and standard procedures of the included studies. We will disseminate our findings by publishing our analysis in a peer-reviewed journal and, if required, we will communicate with the stakeholders of the studies and bibliographic databases.

Prospero registration number

CRD42020179652.

SUBMITTER: Kundeti SR

PROVIDER: S-EPMC7949439 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundArtificial intelligence (AI)-aided analysis of chest CT expedites the quantification of abnormalities and may facilitate the diagnosis and assessment of the prognosis of subjects with COVID-19.ObjectivesThis study investigates the performance of an AI-aided quantification model in predicting the clinical outcomes of hospitalized subjects with COVID-19 and compares it with radiologists' performance.Subjects and methodsA total of 90 subjects with COVID-19 (men, n = 59 [65.6%]; age, 52.9±16.7 years) were recruited in this cross-sectional study. Quantification of the total and compromised lung parenchyma was performed by two expert radiologists using a volumetric image analysis software and compared against an AI-assisted package consisting of a modified U-Net model for segmenting COVID-19 lesions and an off-the-shelf U-Net model augmented with COVID-19 data for segmenting lung volume. The fraction of compromised lung parenchyma (%CL) was calculated. Based on clinical results, the subjects were divided into two categories: critical (n = 45) and noncritical (n = 45). All admission data were compared between the two groups.ResultsThere was an excellent agreement between the radiologist-obtained and AI-assisted measurements (intraclass correlation coefficient = 0.88, P < 0.001). Both the AI-assisted and radiologist-obtained %CLs were significantly higher in the critical subjects (P = 0.009 and 0.02, respectively) than in the noncritical subjects. In the multivariate logistic regression analysis to distinguish the critical subjects, an AI-assisted %CL ≥35% (odds ratio [OR] = 17.0), oxygen saturation level of <88% (OR = 33.6), immunocompromised condition (OR = 8.1), and other comorbidities (OR = 15.2) independently remained as significant variables in the models. Our proposed model obtained an accuracy of 83.9%, a sensitivity of 79.1%, and a specificity of 88.6% in predicting critical outcomes.ConclusionsAI-assisted measurements are similar to quantitative radiologist-obtained measurements in determining lung involvement in COVID-19 subjects.

Project description:Background and Purpose- Accurate and rapid detection of anterior circulation large vessel occlusion (LVO) is of paramount importance in patients with acute stroke due to the potentially rapid infarction of at-risk tissue and the limited therapeutic window for endovascular clot retrieval. Hence, the optimal threshold of a new, fully automated software-based approach for LVO detection was determined, and its diagnostic performance evaluated in a large cohort study. Methods- For this retrospective study, data were pooled from: 2 stroke trials, DEFUSE 2 (n=62; 07/08-09/11) and DEFUSE 3 (n=213; 05/17-05/18); a cohort of endovascular clot retrieval candidates (n=82; August 2, 2014-August 30, 2015) and normals (n=111; June 6, 2017-January 28, 2019) from a single quaternary center; and code stroke patients (n=501; January 1, 2017-December 31, 2018) from a single regional hospital. All CTAs were assessed by the automated algorithm. Consensus reads by 2 neuroradiologists served as the reference standard. ROC analysis was used to assess diagnostic performance of the algorithm for detection of (1) anterior circulation LVOs involving the intracranial internal carotid artery or M1 segment middle cerebral artery (M1-MCA); (2) anterior circulation LVOs and proximal M2 segment MCA (M2-MCA) occlusions; and (3) individual segment occlusions. Results- CTAs from 926 patients (median age 70 years, interquartile range: 58-80; 422 females) were analyzed. Three hundred ninety-five patients had an anterior circulation LVO or M2-MCA occlusion (National Institutes of Health Stroke Scale 14 [median], interquartile range: 9-19). Sensitivity and specificity were 97% and 74%, respectively, for LVO detection, and 95% and 79%, respectively, when M2 occlusions were included. On analysis by occlusion site, sensitivities were 90% (M2-MCA), 97% (M1-MCA), and 97% (intracranial internal carotid artery) with corresponding area-under-the-ROC-curves of 0.874 (M2), 0.962 (M1), and 0.997 (intracranial internal carotid artery). Conclusions- Intracranial anterior circulation LVOs and proximal M2 occlusions can be rapidly and reliably detected by an automated detection tool, which may facilitate intra- and inter-instutional workflows and emergent imaging triage in the care of patients with stroke.

Project description:PurposeTo assess the accuracy of e-Paarvai, an artificial intelligence-based smartphone application (app) that detects and grades cataracts using images taken with a smartphone by comparing with slit lamp-based diagnoses by trained ophthalmologists.MethodsIn this prospective diagnostic study conducted between January and April 2022 at a large tertiary-care eye hospital in South India, two screeners were trained to use the app. Patients aged >40 years and with a best-corrected visual acuity <20/40 were recruited for the study. The app is intended to determine whether the eye has immature cataract, mature cataract, posterior chamber intra-ocular lens, or no cataract. The diagnosis of the app was compared with that of trained ophthalmologists based on slit-lamp examinations, the gold standard, and a receiver operating characteristic (ROC) curve was estimated. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were computed.ResultsThe two screeners used the app to screen 2,619 eyes of 1,407 patients. In detecting cataracts, the app showed high sensitivity (96%) but low specificity (25%), an overall accuracy of 88%, a PPV of 92.3%, and an NPV of 57.8%. In terms of cataract grading, the accuracy of the app was high in detecting immature cataracts (1,875 eyes, 94.2%), but its accuracy was poor in detecting mature cataracts (73 eyes, 22%), posterior chamber intra-ocular lenses (55 eyes, 29.3%), and clear lenses (2 eyes, 2%). We found that the area under the curve in predicting ophthalmologists' cataract diagnosis could potentially be improved beyond the app's diagnosis based on using images only by incorporating information about patient sex and age (P < 0.0001) and best-corrected visual acuity (P < 0.0001).ConclusionsAlthough there is room for improvement, e-Paarvai app is a promising approach for diagnosing cataracts in difficult-to-reach populations. Integrating this with existing outreach programs can enhance the case detection rate.

Project description:BackgroundArtificial Intelligence (AI) holds considerable promise for diagnostics in the field of gastroenterology. This systematic review and meta-analysis aims to assess the diagnostic accuracy of AI models compared with the gold standard of experts and histopathology for the diagnosis of various gastrointestinal (GI) luminal pathologies including polyps, neoplasms, and inflammatory bowel disease.MethodsWe searched PubMed, CINAHL, Wiley Cochrane Library, and Web of Science electronic databases to identify studies assessing the diagnostic performance of AI models for GI luminal pathologies. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. We performed a meta-analysis and hierarchical summary receiver operating characteristic curves (HSROC). The risk of bias was assessed using Quality Assessment for Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Subgroup analyses were conducted based on the type of GI luminal disease, AI model, reference standard, and type of data used for analysis. This study is registered with PROSPERO (CRD42021288360).FindingsWe included 73 studies, of which 31 were externally validated and provided sufficient information for inclusion in the meta-analysis. The overall sensitivity of AI for detecting GI luminal pathologies was 91.9% (95% CI: 89.0-94.1) and specificity was 91.7% (95% CI: 87.4-94.7). Deep learning models (sensitivity: 89.8%, specificity: 91.9%) and ensemble methods (sensitivity: 95.4%, specificity: 90.9%) were the most commonly used models in the included studies. Majority of studies (n = 56, 76.7%) had a high risk of selection bias while 74% (n = 54) studies were low risk on reference standard and 67% (n = 49) were low risk for flow and timing bias.InterpretationThe review suggests high sensitivity and specificity of AI models for the detection of GI luminal pathologies. There is a need for large, multi-center trials in both high income countries and low- and middle- income countries to assess the performance of these AI models in real clinical settings and its impact on diagnosis and prognosis.Systematic review registration[https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=288360], identifier [CRD42021288360].

Project description:ObjectiveTo develop a dynamic 3D radiomics analysis method using artificial intelligence technique for automatically assessing four disease stages (i.e., early, progressive, peak, and absorption stages) of COVID-19 patients on CT images.MethodsThe dynamic 3D radiomics analysis method was composed of three AI algorithms (the lung segmentation, lesion segmentation, and stage-assessing AI algorithms) that were trained and tested on 313,767 CT images from 520 COVID-19 patients. This proposed method used 3D lung lesion that was segmented by the lung and lesion segmentation algorithms to extract radiomics features, and then combined with clinical metadata to assess the possible stage of COVID-19 patients using stage-assessing algorithm. Area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity were used to evaluate diagnostic performance.ResultsOf 520 patients, 66 patients (mean age, 57 years ± 15 [standard deviation]; 35 women), including 203 CT scans, were tested. The dynamic 3D radiomics analysis method used 30 features, including 27 radiomics features and 3 clinical features to assess the possible disease stage of COVID-19 with an accuracy of 90%. For the prediction of each stage, the AUC of stage 1 was 0.965 (95% CI: 0.934, 0.997), AUC of stage 2 was 0.958 (95% CI: 0.931, 0.984), AUC of stage 3 was 0.998 (95% CI: 0.994, 1.000), and AUC of stage 4 was 0.975 (95% CI: 0.956, 0.994).ConclusionWith high diagnostic performance, the dynamic 3D radiomics analysis using artificial intelligence could represent a potential tool for helping hospitals make appropriate resource allocations and follow-up of treatment response.Key points• The AI segmentation algorithms were able to accurately segment the lung and lesion of COVID-19 patients of different stages. • The dynamic 3D radiomics analysis method successfully extracted the radiomics features from the 3D lung lesion. • The stage-assessing AI algorithm combining with clinical metadata was able to assess the four stages with an accuracy of 90%, a macro-average AUC of 0.975.

Project description:BackgroundThe integration of artificial intelligence (AI) into medicine is growing, with some experts predicting its standalone use soon. However, skepticism remains due to limited positive outcomes from independent validations. This research evaluates AI software's effectiveness in analyzing chest X-rays (CXR) to identify lung nodules, a possible lung cancer indicator.MethodsThis retrospective study analyzed 7,670,212 record pairs from radiological exams conducted between 2020 and 2022 during the Moscow Computer Vision Experiment, focusing on CXR and computed tomography (CT) scans. All images were acquired during clinical routine. The final dataset comprised 100 CXR images (50 with lung nodules, 50 without), selected consecutively and based on inclusion and exclusion criteria, to evaluate the performance of all five AI-based solutions, participating in the Moscow Computer Vision Experiment and analyzing CXR. The evaluation was performed in 3 stages. In the first stage, the probability of a nodule in the lung obtained from AI services was compared with the Ground Truth (1-there is a nodule, 0-there is no nodule). In the second stage, 3 radiologists evaluated the segmentation of nodules performed by the AI services (1-nodule correctly segmented, 0-nodule incorrectly segmented or not segmented at all). In the third stage, the same radiologists additionally evaluated the classification of the nodules (1-nodule correctly segmented and classified, 0-all other cases). The results obtained in stages 2 and 3 were compared with Ground Truth, which was common to all three stages. For each stage, diagnostic accuracy metrics were calculated for each AI service.ResultsThree software solutions (Celsus, Lunit INSIGHT CXR, and qXR) demonstrated diagnostic metrics that matched or surpassed the vendor specifications, and achieved the highest area under the receiver operating characteristic curve (AUC) of 0.956 [95% confidence interval (CI): 0.918 to 0.994]. However, when evaluated by three radiologists for accurate nodule segmentation and classification, all solutions performed below the vendor-declared metrics, with the highest AUC reaching 0.812 (95% CI: 0.744 to 0.879). Meanwhile, all AI services demonstrated 100% specificity at stages 2 and 3 of the study.ConclusionsTo ensure the reliability and applicability of AI-based software, it is crucial to validate performance metrics using high-quality datasets and engage radiologists in the evaluation process. Developers are recommended to improve the accuracy of the underlying models before allowing the standalone use of the software for lung nodule detection. The dataset created during the study may be accessed at https://mosmed.ai/datasets/mosmeddatargogksnalichiemiotsutstviemlegochnihuzlovtipvii/.