DR-MIL: deep represented multiple instance learning distinguishes COVID-19 from community-acquired pneumonia in CT images.
Ontology highlight
ABSTRACT: Background and objective
Given that the novel coronavirus disease 2019 (COVID-19) has become a pandemic, a method to accurately distinguish COVID-19 from community-acquired pneumonia (CAP) is urgently needed. However, the spatial uncertainty and morphological diversity of COVID-19 lesions in the lungs, and subtle differences with respect to CAP, make differential diagnosis non-trivial.Methods
We propose a deep represented multiple instance learning (DR-MIL) method to fulfill this task. A 3D volumetric CT scan of one patient is treated as one bag and ten CT slices are selected as the initial instances. For each instance, deep features are extracted from the pre-trained ResNet-50 with fine-tuning and represented as one deep represented instance score (DRIS). Each bag with a DRIS for each initial instance is then input into a citation k-nearest neighbor search to generate the final prediction. A total of 141 COVID-19 and 100 CAP CT scans were used. The performance of DR-MIL is compared with other potential strategies and state-of-the-art models.Results
DR-MIL displayed an accuracy of 95% and an area under curve of 0.943, which were superior to those observed for comparable methods. COVID-19 and CAP exhibited significant differences in both the DRIS and the spatial pattern of lesions (p<0.001). As a means of content-based image retrieval, DR-MIL can identify images used as key instances, references, and citers for visual interpretation.Conclusions
DR-MIL can effectively represent the deep characteristics of COVID-19 lesions in CT images and accurately distinguish COVID-19 from CAP in a weakly supervised manner. The resulting DRIS is a useful supplement to visual interpretation of the spatial pattern of lesions when screening for COVID-19.
SUBMITTER: Qi S
PROVIDER: S-EPMC8426140 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectivesTo utilize a deep learning model for automatic detection of abnormalities in chest CT images from COVID-19 patients and compare its quantitative determination performance with radiological residents.MethodsA deep learning algorithm consisted of lesion detection, segmentation, and location was trained and validated in 14,435 participants with chest CT images and definite pathogen diagnosis. The algorithm was tested in a non-overlapping dataset of 96 confirmed COVID-19 patients in three hospitals across China during the outbreak. Quantitative detection performance of the model was compared with three radiological residents with two experienced radiologists' reading reports as reference standard by assessing the accuracy, sensitivity, specificity, and F1 score.ResultsOf 96 patients, 88 had pneumonia lesions on CT images and 8 had no abnormities on CT images. For per-patient basis, the algorithm showed superior sensitivity of 1.00 (95% confidence interval (CI) 0.95, 1.00) and F1 score of 0.97 in detecting lesions from CT images of COVID-19 pneumonia patients. While for per-lung lobe basis, the algorithm achieved a sensitivity of 0.96 (95% CI 0.94, 0.98) and a slightly inferior F1 score of 0.86. The median volume of lesions calculated by algorithm was 40.10 cm3. An average running speed of 20.3 s ± 5.8 per case demonstrated the algorithm was much faster than the residents in assessing CT images (all p < 0.017). The deep learning algorithm can also assist radiologists make quicker diagnosis (all p < 0.0001) with superior diagnostic performance.ConclusionsThe algorithm showed excellent performance in detecting COVID-19 pneumonia on chest CT images compared with resident radiologists.Key points• The higher sensitivity of deep learning model in detecting COVID-19 pneumonia were found compared with radiological residents on a per-lobe and per-patient basis. • The deep learning model improves diagnosis efficiency by shortening processing time. • The deep learning model can automatically calculate the volume of the lesions and whole lung.
| S-EPMC7331494 | biostudies-literature
Project description:Computed tomography (CT) is one of the most efficient diagnostic methods for rapid diagnosis of the widespread COVID-19. However, reading CT films brings a lot of concentration and time for doctors. Therefore, it is necessary to develop an automatic CT image diagnosis system to assist doctors in diagnosis. Previous studies devoted to COVID-19 in the past months focused mostly on discriminating COVID-19 infected patients from healthy persons and/or bacterial pneumonia patients, and have ignored typical viral pneumonia since it is hard to collect samples for viral pneumonia that is less frequent in adults. In addition, it is much more challenging to discriminate COVID-19 from typical viral pneumonia as COVID-19 is also a kind of virus. In this study, we have collected CT images of 262, 100, 219, and 78 persons for COVID-19, bacterial pneumonia, typical viral pneumonia, and healthy controls, respectively. To the best of our knowledge, this was the first study of quaternary classification to include also typical viral pneumonia. To effectively capture the subtle differences in CT images, we have constructed a new model by combining the ResNet50 backbone with SE blocks that was recently developed for fine image analysis. Our model was shown to outperform commonly used baseline models, achieving an overall accuracy of 0.94 with AUC of 0.96, recall of 0.94, precision of 0.95, and F1-score of 0.94. The model is available in https://github.com/Zhengfudan/COVID-19-Diagnosis-and-Pneumonia-Classification .
| S-EPMC7914048 | biostudies-literature
Project description:Coronavirus Disease 2019 (COVID-19) still presents a pandemic trend globally. Detecting infected individuals and analyzing their status can provide patients with proper healthcare while protecting the normal population. Chest CT (computed tomography) is an effective tool for screening of COVID-19. It displays detailed pathology-related information. To achieve automated COVID-19 diagnosis and lung CT image segmentation, convolutional neural networks (CNNs) have become mainstream methods. However, most of the previous works consider automated diagnosis and image segmentation as two independent tasks, in which some focus on lung fields segmentation and the others focus on single-lesion segmentation. Moreover, lack of clinical explainability is a common problem for CNN-based methods. In such context, we develop a multi-task learning framework in which the diagnosis of COVID-19 and multi-lesion recognition (segmentation of CT images) are achieved simultaneously. The core of the proposed framework is an explainable multi-instance multi-task network. The network learns task-related features adaptively with learnable weights, and gives explicable diagnosis results by suggesting local CT images with lesions as additional evidence. Then, severity assessment of COVID-19 and lesion quantification are performed to analyze patient status. Extensive experimental results on real-world datasets show that the proposed framework outperforms all the compared approaches for COVID-19 diagnosis and multi-lesion segmentation.
| S-EPMC9235304 | biostudies-literature
Project description:BackgroundTo develop a machine learning-based CT radiomics model is critical for the accurate diagnosis of the rapid spreading coronavirus disease 2019 (COVID-19).MethodsIn this retrospective study, a total of 326 chest CT exams from 134 patients (63 confirmed COVID-19 patients and 71 non-COVID-19 patients) were collected from January 20 to February 8, 2020. A semi-automatic segmentation procedure was used to delineate the volume of interest (VOI), and radiomic features were extracted. The Support Vector Machine (SVM) model was built on the combination of 4 groups of features, including radiomic features, traditional radiological features, quantifying features, and clinical features. By repeating cross-validation procedure, the performance on the time-independent testing cohort was evaluated by the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.ResultsFor the SVM model built on the combination of 4 groups of features (integrated model), the per-exam AUC was 0.925 (95% CI 0.856 to 0.994) for differentiating COVID-19 on the testing cohort, and the sensitivity and specificity were 0.816 (95% CI 0.651 to 0.917) and 0.923 (95% CI 0.621 to 0.996), respectively. As for the SVM models built on radiomic features, radiological features, quantifying features, and clinical features, individually, the AUC on the testing cohort reached 0.765, 0.818, 0.607, and 0.739, respectively, significantly lower than the integrated model, except for the radiomic model.ConclusionThe machine learning-based CT radiomics models may accurately classify COVID-19, helping clinicians and radiologists to identify COVID-19 positive cases.
| S-EPMC8424152 | biostudies-literature
Project description:Tumor histology is an important predictor of therapeutic response and outcomes in lung cancer. Tissue sampling for pathologist review is the most reliable method for histology classification, however, recent advances in deep learning for medical image analysis allude to the utility of radiologic data in further describing disease characteristics and for risk stratification. In this study, we propose a radiomics approach to predicting non-small cell lung cancer (NSCLC) tumor histology from non-invasive standard-of-care computed tomography (CT) data. We trained and validated convolutional neural networks (CNNs) on a dataset comprising 311 early-stage NSCLC patients receiving surgical treatment at Massachusetts General Hospital (MGH), with a focus on the two most common histological types: adenocarcinoma (ADC) and Squamous Cell Carcinoma (SCC). The CNNs were able to predict tumor histology with an AUC of 0.71(p = 0.018). We also found that using machine learning classifiers such as k-nearest neighbors (kNN) and support vector machine (SVM) on CNN-derived quantitative radiomics features yielded comparable discriminative performance, with AUC of up to 0.71 (p = 0.017). Our best performing CNN functioned as a robust probabilistic classifier in heterogeneous test sets, with qualitatively interpretable visual explanations to its predictions. Deep learning based radiomics can identify histological phenotypes in lung cancer. It has the potential to augment existing approaches and serve as a corrective aid for diagnosticians.
| S-EPMC7943565 | biostudies-literature
Project description:Even with over 80% of the population being vaccinated against COVID-19, the disease continues to claim victims. Therefore, it is crucial to have a secure Computer-Aided Diagnostic system that can assist in identifying COVID-19 and determining the necessary level of care. This is especially important in the Intensive Care Unit to monitor disease progression or regression in the fight against this epidemic. To accomplish this, we merged public datasets from the literature to train lung and lesion segmentation models with five different distributions. We then trained eight CNN models for COVID-19 and Common-Acquired Pneumonia classification. If the examination was classified as COVID-19, we quantified the lesions and assessed the severity of the full CT scan. To validate the system, we used Resnetxt101 Unet++ and Mobilenet Unet for lung and lesion segmentation, respectively, achieving accuracy of 98.05%, F1-score of 98.70%, precision of 98.7%, recall of 98.7%, and specificity of 96.05%. This was accomplished in just 19.70 s per full CT scan, with external validation on the SPGC dataset. Finally, when classifying these detected lesions, we used Densenet201 and achieved accuracy of 90.47%, F1-score of 93.85%, precision of 88.42%, recall of 100.0%, and specificity of 65.07%. The results demonstrate that our pipeline can correctly detect and segment lesions due to COVID-19 and Common-Acquired Pneumonia in CT scans. It can differentiate these two classes from normal exams, indicating that our system is efficient and effective in identifying the disease and assessing the severity of the condition.
| S-EPMC10215490 | biostudies-literature
Project description:ObjectiveTo develop a fully automated AI system to quantitatively assess the disease severity and disease progression of COVID-19 using thick-section chest CT images.MethodsIn this retrospective study, an AI system was developed to automatically segment and quantify the COVID-19-infected lung regions on thick-section chest CT images. Five hundred thirty-one CT scans from 204 COVID-19 patients were collected from one appointed COVID-19 hospital. The automatically segmented lung abnormalities were compared with manual segmentation of two experienced radiologists using the Dice coefficient on a randomly selected subset (30 CT scans). Two imaging biomarkers were automatically computed, i.e., the portion of infection (POI) and the average infection HU (iHU), to assess disease severity and disease progression. The assessments were compared with patient status of diagnosis reports and key phrases extracted from radiology reports using the area under the receiver operating characteristic curve (AUC) and Cohen's kappa, respectively.ResultsThe dice coefficient between the segmentation of the AI system and two experienced radiologists for the COVID-19-infected lung abnormalities was 0.74 ± 0.28 and 0.76 ± 0.29, respectively, which were close to the inter-observer agreement (0.79 ± 0.25). The computed two imaging biomarkers can distinguish between the severe and non-severe stages with an AUC of 0.97 (p value < 0.001). Very good agreement (κ = 0.8220) between the AI system and the radiologists was achieved on evaluating the changes in infection volumes.ConclusionsA deep learning-based AI system built on the thick-section CT imaging can accurately quantify the COVID-19-associated lung abnormalities and assess the disease severity and its progressions.Key points• A deep learning-based AI system was able to accurately segment the infected lung regions by COVID-19 using the thick-section CT scans (Dice coefficient ≥ 0.74). • The computed imaging biomarkers were able to distinguish between the non-severe and severe COVID-19 stages (area under the receiver operating characteristic curve 0.97). • The infection volume changes computed by the AI system were able to assess the COVID-19 progression (Cohen's kappa 0.8220).
| S-EPMC7368602 | biostudies-literature
Project description:Mitochondria are extremely pleomorphic organelles. Automatically annotating each one accurately and precisely in any 2D or volume electron microscopy (EM) image is an unsolved computational challenge. Current deep learning-based approaches train models on images that provide limited cellular contexts, precluding generality. To address this, we amassed a highly heterogeneous ∼1.5 × 106 image 2D unlabeled cellular EM dataset and segmented ∼135,000 mitochondrial instances therein. MitoNet, a model trained on these resources, performs well on challenging benchmarks and on previously unseen volume EM datasets containing tens of thousands of mitochondria. We release a Python package and napari plugin, empanada, to rapidly run inference, visualize, and proofread instance segmentations. A record of this paper's transparent peer review process is included in the supplemental information.
| S-EPMC9883049 | biostudies-literature
Project description:BACKGROUND:Multiplex immunohistochemistry (mIHC) permits the labeling of six or more distinct cell types within a single histologic tissue section. The classification of each cell type requires detection of the unique colored chromogens localized to cells expressing biomarkers of interest. The most comprehensive and reproducible method to evaluate such slides is to employ digital pathology and image analysis pipelines to whole-slide images (WSIs). Our suite of deep learning tools quantitatively evaluates the expression of six biomarkers in mIHC WSIs. These methods address the current lack of readily available methods to evaluate more than four biomarkers and circumvent the need for specialized instrumentation to spectrally separate different colors. The use case application for our methods is a study that investigates tumor immune interactions in pancreatic ductal adenocarcinoma (PDAC) with a customized mIHC panel. METHODS:Six different colored chromogens were utilized to label T-cells (CD3, CD4, CD8), B-cells (CD20), macrophages (CD16), and tumor cells (K17) in formalin-fixed paraffin-embedded (FFPE) PDAC tissue sections. We leveraged pathologist annotations to develop complementary deep learning-based methods: (1) ColorAE is a deep autoencoder which segments stained objects based on color; (2) U-Net is a convolutional neural network (CNN) trained to segment cells based on color, texture and shape; and ensemble methods that employ both ColorAE and U-Net, collectively referred to as (3) ColorAE:U-Net. We assessed the performance of our methods using: structural similarity and DICE score to evaluate segmentation results of ColorAE against traditional color deconvolution; F1 score, sensitivity, positive predictive value, and DICE score to evaluate the predictions from ColorAE, U-Net, and ColorAE:U-Net ensemble methods against pathologist-generated ground truth. We then used prediction results for spatial analysis (nearest neighbor). RESULTS:We observed that (1) the performance of ColorAE is comparable to traditional color deconvolution for single-stain IHC images (note: traditional color deconvolution cannot be used for mIHC); (2) ColorAE and U-Net are complementary methods that detect 6 different classes of cells with comparable performance; (3) combinations of ColorAE and U-Net into ensemble methods outperform using either ColorAE and U-Net alone; and (4) ColorAE:U-Net ensemble methods can be employed for detailed analysis of the tumor microenvironment (TME). We developed a suite of scalable deep learning methods to analyze 6 distinctly labeled cell populations in mIHC WSIs. We evaluated our methods and found that they reliably detected and classified cells in the PDAC tumor microenvironment. We also present a use case, wherein we apply the ColorAE:U-Net ensemble method across 3 mIHC WSIs and use the predictions to quantify all stained cell populations and perform nearest neighbor spatial analysis. Thus, we provide proof of concept that these methods can be employed to quantitatively describe the spatial distribution immune cells within the tumor microenvironment. These complementary deep learning methods are readily deployable for use in clinical research studies.
| S-EPMC7385962 | biostudies-literature
Project description:Pulmonary fissure detection in computed tomography (CT) is a critical component for automatic lobar segmentation. The majority of fissure detection methods use feature descriptors that are hand-crafted, low-level, and have local spatial extent. The design of such feature detectors is typically targeted toward normal fissure anatomy, yielding low sensitivity to weak, and abnormal fissures that are common in clinical data sets. Furthermore, local features commonly suffer from low specificity, as the complex textures in the lung can be indistinguishable from the fissure when the global context is not considered. We propose a supervised discriminative learning framework for simultaneous feature extraction and classification. The proposed framework, called FissureNet, is a coarse-to-fine cascade of two convolutional neural networks. The coarse-to-fine strategy alleviates the challenges associated with training a network to segment a thin structure that represents a small fraction of the image voxels. FissureNet was evaluated on a cohort of 3706 subjects with inspiration and expiration 3DCT scans from the COPDGene clinical trial and a cohort of 20 subjects with 4DCT scans from a lung cancer clinical trial. On both data sets, FissureNet showed superior performance compared with a deep learning approach using the U-Net architecture and a Hessian-based fissure detection method in terms of area under the precision-recall curve (PR-AUC). The overall PR-AUC for FissureNet, U-Net, and Hessian on the COPDGene (lung cancer) data set was 0.980 (0.966), 0.963 (0.937), and 0.158 (0.182), respectively. On a subset of 30 COPDGene scans, FissureNet was compared with a recently proposed advanced fissure detection method called derivative of sticks (DoS) and showed superior performance with a PR-AUC of 0.991 compared with 0.668 for DoS.
| S-EPMC6318012 | biostudies-literature