Dataset Information

A multicenter study of artificial intelligence-aided software for detecting visible clinically significant prostate cancer on mpMRI.

ABSTRACT:

Background

AI-based software may improve the performance of radiologists when detecting clinically significant prostate cancer (csPCa). This study aims to compare the performance of radiologists in detecting MRI-visible csPCa on MRI with and without AI-based software.

Materials and methods

In total, 480 multiparametric MRI (mpMRI) images were retrospectively collected from eleven different MR devices, with 349 csPCa lesions in 180 (37.5%) cases. The csPCa areas were annotated based on pathology. Sixteen radiologists from four hospitals participated in reading. Each radiologist was randomly assigned to 30 cases and diagnosed twice. Half cases were interpreted without AI, and the other half were interpreted with AI. After four weeks, the cases were read again in switched mode. The mean diagnostic performance was compared using sensitivity and specificity on lesion level and patient level. The median reading time and diagnostic confidence were assessed.

Results

On lesion level, AI-aided improved the sensitivity from 40.1% to 59.0% (18.9% increased; 95% confidence interval (CI) [11.5, 26.1]; p < .001). On patient level, AI-aided improved the specificity from 57.7 to 71.7% (14.0% increase, 95% CI [6.4, 21.4]; p < .001) while preserving the sensitivity (88.3% vs. 93.9%, p = 0.06). AI-aided reduced the median reading time of one case by 56.3% from 423 to 185 s (238-s decrease, 95% CI [219, 260]; p < .001), and the median diagnostic confidence score was increased by 10.3% from 3.9 to 4.3 (0.4-score increase, 95% CI [0.3, 0.5]; p < .001).

Conclusions

AI software improves the performance of radiologists by reducing false positive detection of prostate cancer patients and also improving reading times and diagnostic confidence.

Clinical relevance statement

This study involves the process of data collection, randomization and crossover reading procedure.

SUBMITTER: Sun Z

PROVIDER: S-EPMC10149551 | biostudies-literature | 2023 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A multicenter study of artificial intelligence-aided software for detecting visible clinically significant prostate cancer on mpMRI.

Sun Zhaonan Z Wang Kexin K Kong Zixuan Z Xing Zhangli Z Chen Yuntian Y Luo Ning N Yu Yang Y Song Bin B Wu Pengsheng P Wang Xiangpeng X Zhang Xiaodong X Wang Xiaoying X

Insights into imaging 20230430 1

<h4>Background</h4>AI-based software may improve the performance of radiologists when detecting clinically significant prostate cancer (csPCa). This study aims to compare the performance of radiologists in detecting MRI-visible csPCa on MRI with and without AI-based software.<h4>Materials and methods</h4>In total, 480 multiparametric MRI (mpMRI) images were retrospectively collected from eleven different MR devices, with 349 csPCa lesions in 180 (37.5%) cases. The csPCa areas were annotated base ...[more]

PMID: 37121983

Similar Datasets

Project description:BackgroundAn increasing number of patients with suspected clinically significant prostate cancer (csPCa) are undergoing prostate multiparametric magnetic resonance imaging (mpMRI). The role of artificial intelligence (AI) algorithms in interpreting prostate mpMRI needs to be tested with multicenter external data. This study aimed to investigate the diagnostic efficacy of an AI model in detecting and localizing visible csPCa on mpMRI a multicenter external data set.MethodsThe data of 2,105 patients suspected of having prostate cancer from four hospitals were retrospectively collected to develop an AI model to detect and localize suspicious csPCa. The lesions were annotated based on pathology records by two radiologists. Diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) values were used as the input for the three-dimensional U-Net framework. Subsequently, the model was validated using an external data set comprising the data of 557 patients from three hospitals. Sensitivity, specificity, and accuracy were employed to evaluate the diagnostic efficacy of the model.ResultsAt the lesion level, the model had a sensitivity of 0.654. At the overall sextant level, the model had a sensitivity, specificity, and accuracy of 0.846, 0.884, and 0.874, respectively. At the patient level, the model had a sensitivity, specificity, and accuracy of 0.943, 0.776, and 0.849, respectively. The AI-predicted accuracy for the csPCa patients (231/245, 0.943) was significantly higher than that for the non-csPCa patients (242/312, 0.776) (P<0.001). The lesion number and tumor volume were greater in the correctly diagnosed patients than the incorrectly diagnosed patients (both P<0.001). Among the positive patients, those with lower average ADC values had a higher rate of correct diagnosis than those with higher average ADC values (P=0.01).ConclusionsThe AI model exhibited acceptable accuracy in detecting and localizing visible csPCa at the patient and sextant levels. However, further improvements need to be made to enhance the sensitivity of the model at the lesion level.

Project description:BackgroundThe introduction of multiparameter MRI and novel biomarkers has greatly improved the prediction of clinically significant prostate cancer (csPCa). However, decision-making regarding prostate biopsy and prebiopsy examinations is still difficult. We aimed to establish a quick and economic tool to improve the detection of csPCa based on routinely performed clinical examinations through an automated machine learning platform (AutoML).MethodsThis study included a multicenter retrospective cohort and two prospective cohorts with 4747 cases from 9 hospitals across China. The multimodal data, including demographics, clinical characteristics, laboratory tests, and ultrasound reports, of consecutive participants were retrieved using extract-transform-load tools. AutoML was applied to explore potential data processing patterns and the most suitable algorithm to build the Prostate Cancer Artificial Intelligence Diagnostic System (PCAIDS). The diagnostic performance was determined by the receiver operating characteristic curve (ROC) for discriminating csPCa from insignificant prostate cancer (PCa) and benign disease. The clinical utility was evaluated by decision curve analysis (DCA) and waterfall plots.ResultsThe random forest algorithm was applied in the feature selection, and the AutoML algorithm was applied for model establishment. The area under the curve (AUC) value in identifying csPCa was 0.853 in the training cohort, 0.820 in the validation cohort, 0.807 in the Changhai prospective cohort, and 0.850 in the Zhongda prospective cohort. DCA showed that the PCAIDS was superior to PSA or fPSA/tPSA for diagnosing csPCa with a higher net benefit for all threshold probabilities in all cohorts. Setting a fixed sensitivity of 95%, a total of 32.2%, 17.6%, and 26.3% of unnecessary biopsies could be avoided with less than 5% of csPCa missed in the validation cohort, Changhai and Zhongda prospective cohorts, respectively.ConclusionsThe PCAIDS was an effective tool to inform decision-making regarding the need for prostate biopsy and prebiopsy examinations such as mpMRI. Further prospective and international studies are warranted to validate the findings of this study.Trial registrationChinese Clinical Trial Registry ChiCTR2100048428. Registered on 06 July 2021.

Project description:Summary Background MRI is the routine examination to surveil the recurrence of nasopharyngeal carcinoma, but it has relatively lower sensitivity than PET/CT. We aimed to find if artificial intelligence (AI) could be competent pre-inspector for MRI radiologists and whether AI-aided MRI could perform better or even equal to PET/CT. Methods This multicenter study enrolled 6916 patients from five hospitals between September 2009 and October 2020. A 2.5D convolutional neural network diagnostic model and a nnU-Net contouring model were developed in the training and test cohorts and used to independently predict and visualize the recurrence of patients in the internal and external validation cohorts. We evaluated the area under the ROC curve (AUC) of AI and compared AI with MRI and PET/CT in sensitivity and specificity using the McNemar test. The prospective cohort was randomized into the AI and non-AI groups, and their sensitivity and specificity were compared using the Chi-square test. Findings The AI model achieved AUCs of 0.92 and 0.88 in the internal and external validation cohorts, corresponding to the sensitivity of 79.5% and 74.3% and specificity of 91.0% and 92.8%. It had comparable sensitivity to MRI (e.g., 74.3% vs. 74.7%, P = 0.89) but lower sensitivity than PET/CT (77.9% vs. 92.0%, P < 0.0001) at the same individual-specificities. The AI model achieved moderate precision with a median dice similarity coefficient of 0.67. AI-aided MRI improved specificity (92.5% vs. 85.0%, P = 0.034), equaled PET/CT in the internal validation subcohort, and increased sensitivity (81.9% vs. 70.8%, P = 0.021) in the external validation subcohort. In the prospective cohort of 1248 patients, the AI group had higher sensitivity than the non-AI group (78.6% vs. 67.3%, P = 0.23), albeit nonsignificant. In future randomized controlled trials, a sample size of 3943 patients in each arm would be required to demonstrate the statistically significant difference. Interpretation The AI model equaled MRI by expert radiologists, and AI-aided MRI by expert radiologists equaled PET/CT. A larger randomized controlled trial is warranted to demonstrate the AI's benefit sufficiently. Funding The Sun Yat-sen University Clinical Research 5010 Program (2015020), 10.13039/501100021171Guangdong Basic and Applied Basic Research Foundation (2022A1515110356), and Guangzhou Science and Technology Program (2023A04J1788).

Project description:BackgroundThe integration of artificial intelligence (AI) into medicine is growing, with some experts predicting its standalone use soon. However, skepticism remains due to limited positive outcomes from independent validations. This research evaluates AI software's effectiveness in analyzing chest X-rays (CXR) to identify lung nodules, a possible lung cancer indicator.MethodsThis retrospective study analyzed 7,670,212 record pairs from radiological exams conducted between 2020 and 2022 during the Moscow Computer Vision Experiment, focusing on CXR and computed tomography (CT) scans. All images were acquired during clinical routine. The final dataset comprised 100 CXR images (50 with lung nodules, 50 without), selected consecutively and based on inclusion and exclusion criteria, to evaluate the performance of all five AI-based solutions, participating in the Moscow Computer Vision Experiment and analyzing CXR. The evaluation was performed in 3 stages. In the first stage, the probability of a nodule in the lung obtained from AI services was compared with the Ground Truth (1-there is a nodule, 0-there is no nodule). In the second stage, 3 radiologists evaluated the segmentation of nodules performed by the AI services (1-nodule correctly segmented, 0-nodule incorrectly segmented or not segmented at all). In the third stage, the same radiologists additionally evaluated the classification of the nodules (1-nodule correctly segmented and classified, 0-all other cases). The results obtained in stages 2 and 3 were compared with Ground Truth, which was common to all three stages. For each stage, diagnostic accuracy metrics were calculated for each AI service.ResultsThree software solutions (Celsus, Lunit INSIGHT CXR, and qXR) demonstrated diagnostic metrics that matched or surpassed the vendor specifications, and achieved the highest area under the receiver operating characteristic curve (AUC) of 0.956 [95% confidence interval (CI): 0.918 to 0.994]. However, when evaluated by three radiologists for accurate nodule segmentation and classification, all solutions performed below the vendor-declared metrics, with the highest AUC reaching 0.812 (95% CI: 0.744 to 0.879). Meanwhile, all AI services demonstrated 100% specificity at stages 2 and 3 of the study.ConclusionsTo ensure the reliability and applicability of AI-based software, it is crucial to validate performance metrics using high-quality datasets and engage radiologists in the evaluation process. Developers are recommended to improve the accuracy of the underlying models before allowing the standalone use of the software for lung nodule detection. The dataset created during the study may be accessed at https://mosmed.ai/datasets/mosmeddatargogksnalichiemiotsutstviemlegochnihuzlovtipvii/.

Dataset Information

A multicenter study of artificial intelligence-aided software for detecting visible clinically significant prostate cancer on mpMRI.

Background

Materials and methods

Results

Conclusions

Clinical relevance statement

Publications

A multicenter study of artificial intelligence-aided software for detecting visible clinically significant prostate cancer on mpMRI.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets