Dataset Information

Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: Comparison with the performance of double reading by radiologists.

ABSTRACT: BACKGROUND:The study was conducted to evaluate the performance of a state-of-the-art commercial deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing pulmonary nodules. METHODS:Pulmonary nodules in 346 healthy subjects (male: female = 221:125, mean age 51 years) from a lung cancer screening program conducted from March to November 2017 were screened using a DL-CAD system and double reading independently, and their performance in nodule detection and characterization were evaluated. An expert panel combined the results of the DL-CAD system and double reading as the reference standard. RESULTS:The DL-CAD system showed a higher detection rate than double reading, regardless of nodule size (86.2% vs. 79.2%; P < 0.001): nodules ? 5 mm (96.5% vs. 88.0%; P = 0.008); nodules < 5 mm (84.3% vs. 77.5%; P < 0.001). However, the false positive rate (per computed tomography scan) of the DL-CAD system (1.53, 529/346) was considerably higher than that of double reading (0.13, 44/346; P < 0.001). Regarding nodule characterization, the sensitivity and specificity of the DL-CAD system for distinguishing solid nodules > 5 mm (90.3% and 100.0%, respectively) and ground-glass nodules (100.0% and 96.1%, respectively) were close to that of double reading, but dropped to 55.5% and 93%, respectively, when discriminating part solid nodules. CONCLUSION:Our DL-CAD system detected significantly more nodules than double reading. In the future, false positive findings should be further reduced and characterization accuracy improved.

SUBMITTER: Li L

PROVIDER: S-EPMC6360226 | biostudies-literature | 2019 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: Comparison with the performance of double reading by radiologists.

Li Li L Liu Zhou Z Huang Hua H Lin Meng M Luo Dehong D

Thoracic cancer 20181208 2

<h4>Background</h4>The study was conducted to evaluate the performance of a state-of-the-art commercial deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing pulmonary nodules.<h4>Methods</h4>Pulmonary nodules in 346 healthy subjects (male: female = 221:125, mean age 51 years) from a lung cancer screening program conducted from March to November 2017 were screened using a DL-CAD system and double reading independently, and their performance in nodule detec ...[more]

PMID: 30536611

Similar Datasets

Project description:To evaluate the interobserver variability in descriptions of breast masses by dedicated breast imagers and radiology residents and determine how any differences in lesion description affect the performance of a computer-aided diagnosis (CAD) computer classification system.Institutional review board approval was obtained for this HIPAA-compliant study, and the requirement to obtain informed consent was waived. Images of 50 breast lesions were individually interpreted by seven dedicated breast imagers and 10 radiology residents, yielding 850 lesion interpretations. Lesions were described with use of 11 descriptors from the Breast Imaging Reporting and Data System, and interobserver variability was calculated with the Cohen κ statistic. Those 11 features were selected, along with patient age, and merged together by a linear discriminant analysis (LDA) classification model trained by using 1005 previously existing cases. Variability in the recommendations of the computer model for different observers was also calculated with the Cohen κ statistic.A significant difference was observed for six lesion features, and radiology residents had greater interobserver variability in their selection of five of the six features than did dedicated breast imagers. The LDA model accurately classified lesions for both sets of observers (area under the receiver operating characteristic curve = 0.94 for residents and 0.96 for dedicated imagers). Sensitivity was maintained at 100% for residents and improved from 98% to 100% for dedicated breast imagers. For residents, the computer model could potentially improve the specificity from 20% to 40% (P < .01) and the κ value from 0.09 to 0.53 (P < .001). For dedicated breast imagers, the computer model could increase the specificity from 34% to 43% (P = .16) and the κ value from 0.21 to 0.61 (P < .001).Among findings showing a significant difference, there was greater interobserver variability in lesion descriptions among residents; however, an LDA model using data from either dedicated breast imagers or residents yielded a consistently high performance in the differentiation of benign from malignant breast lesions, demonstrating potential for improving specificity and decreasing interobserver variability in biopsy recommendations.

Project description:BackgroundAlthough miniscrews are widely used in orthodontic treatment as temporary anchorage devices, their correct and safe placement has attracted little attention. This study aimed to introduce a novel cone beam CT (CBCT) image-based computer aided design and computer aided manufacturing (CAD-CAM) template for orthodontic miniscrew implantation and to evaluate the effectiveness of miniscrews implanted under the guidance of this template.MethodsThe CBCT scans of ten patients requiring miniscrews as anchorage were analyzed in NNT software to predetermine the insertion sites of miniscrew implants. The DICOM data of the scans, along with virtual miniscrews acquired in Solidworks software, were imported into Mimics software to construct three-dimensional (3D) images of teeth and bone and to determine the virtual position of miniscrews and 3D virtual templates were designed following consideration of the virtual implantation plans. A STL (Stereolithography) file of the virtual template was output, and the resin template was then fabricated with a stereolithographic appliance (SLA). 24 Miniscrews were then implanted guided by the template and clinical evaluation of their safety and stability, as well as their placement deviations, were made. A dental casts model and cephalometric analysis before and after orthodontic treatment were made to assess the dentomaxillofacial changes.ResultsAll 24 miniscrews had no contact with adjacent roots. 18 miniscrews had a grade I safety score and six had a grade II. The miniscrews were stable at 1, 3, 6, and 9 months after implantation, although there was mild inflammation around two miniscrews. Implantation deviation of miniscrew in the crown was (1.03±0.65) and (1.26±0.72) mm in the apex, on average. Satisfactory dentomaxillofacial changes in 10 patients with these 24 miniscrews as anchorage were acquired.ConclusionsMiniscrews could be implanted in the targeted position safely and precisely when guided by the novel templates, and remained stable during orthodontic treatment. Patients treated with these miniscrews as anchorage in orthodontic treatment acquired satisfactory dentomaxillofacial changes.

Project description:Radiologists' diagnostic capabilities for breast mass lesions depend on their experience. Junior radiologists may underestimate or overestimate Breast Imaging Reporting and Data System (BI-RADS) categories of mass lesions owing to a lack of diagnostic experience. The computer-aided diagnosis (CAD) method assists in improving diagnostic performance by providing a breast mass classification reference to radiologists. This study aims to evaluate the impact of a CAD method based on perceptive features learned from quantitative BI-RADS descriptions on breast mass diagnosis performance. We conducted a retrospective multi-reader multi-case (MRMC) study to assess the perceptive feature-based CAD method. A total of 416 digital mammograms of patients with breast masses were obtained from 2014 through 2017, including 231 benign and 185 malignant masses, from which we randomly selected 214 cases (109 benign, 105 malignant) to train the CAD model for perceptive feature extraction and classification. The remaining 202 cases were enrolled as the test set for evaluation, of which 51 patients (29 benign and 22 malignant) participated in the MRMC study. In the MRMC study, we categorized six radiologists into three groups: junior, middle-senior, and senior. They diagnosed 51 patients with and without support from the CAD model. The BI-RADS category, benign or malignant diagnosis, malignancy probability, and diagnosis time during the two evaluation sessions were recorded. In the MRMC evaluation, the average area under the curve (AUC) of the six radiologists with CAD support was slightly higher than that without support (0.896 vs. 0.850, p = 0.0209). Both average sensitivity and specificity increased (p = 0.0253). Under CAD assistance, junior and middle-senior radiologists adjusted the assessment categories of more BI-RADS 4 cases. The diagnosis time with and without CAD support was comparable for five radiologists. The CAD model improved the radiologists' diagnostic performance for breast masses without prolonging the diagnosis time and assisted in a better BI-RADS assessment, especially for junior radiologists.

Project description:Computer-aided detection identifies suspicious findings on mammograms to assist radiologists. Since the Food and Drug Administration approved the technology in 1998, it has been disseminated into practice, but its effect on the accuracy of interpretation is unclear.We determined the association between the use of computer-aided detection at mammography facilities and the performance of screening mammography from 1998 through 2002 at 43 facilities in three states. We had complete data for 222,135 women (a total of 429,345 mammograms), including 2351 women who received a diagnosis of breast cancer within 1 year after screening. We calculated the specificity, sensitivity, and positive predictive value of screening mammography with and without computer-aided detection, as well as the rates of biopsy and breast-cancer detection and the overall accuracy, measured as the area under the receiver-operating-characteristic (ROC) curve.Seven facilities (16%) implemented computer-aided detection during the study period. Diagnostic specificity decreased from 90.2% before implementation to 87.2% after implementation (P<0.001), the positive predictive value decreased from 4.1% to 3.2% (P=0.01), and the rate of biopsy increased by 19.7% (P<0.001). The increase in sensitivity from 80.4% before implementation of computer-aided detection to 84.0% after implementation was not significant (P=0.32). The change in the cancer-detection rate (including invasive breast cancers and ductal carcinomas in situ) was not significant (4.15 cases per 1000 screening mammograms before implementation and 4.20 cases after implementation, P=0.90). Analyses of data from all 43 facilities showed that the use of computer-aided detection was associated with significantly lower overall accuracy than was nonuse (area under the ROC curve, 0.871 vs. 0.919; P=0.005).The use of computer-aided detection is associated with reduced accuracy of interpretation of screening mammograms. The increased rate of biopsy with the use of computer-aided detection is not clearly associated with improved detection of invasive breast cancer.

Dataset Information

Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: Comparison with the performance of double reading by radiologists.

Publications

Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: Comparison with the performance of double reading by radiologists.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets