CTH-Net: A CNN and Transformer hybrid network for skin lesion segmentation.
Ontology highlight
ABSTRACT:
SUBMITTER: Ding Y
PROVIDER: S-EPMC10957498 | biostudies-literature | 2024 Apr
REPOSITORIES: biostudies-literature
Publications
CTH-Net: A CNN and Transformer hybrid network for skin lesion segmentation.
Ding Yuhan Y Yi Zhenglin Z Xiao Jiatong J Hu Minghui M Guo Yu Y Liao Zhifang Z Wang Yongjie Y
iScience 20240306 4
Automatically and accurately segmenting skin lesions can be challenging, due to factors such as low contrast and fuzzy boundaries. This paper proposes a hybrid encoder-decoder model (CTH-Net) based on convolutional neural network (CNN) and Transformer, capitalizing on the advantages of these approaches. We propose three modules for skin lesion segmentation and seamlessly connect them with carefully designed model architecture. Better segmentation performance is achieved by introducing SoftPool i ...[more]
Similar Datasets
Project description:Melanoma is a malignant skin tumor that threatens human life and health. Early detection is essential for effective treatment. However, the low contrast between melanoma lesions and normal skin and the irregularity in size and shape make skin lesions difficult to detect with the naked eye in the early stages, making the task of skin lesion segmentation challenging. Traditional encoder-decoder built with U-shaped networks using convolutional neural network (CNN) networks have limitations in establishing long-term dependencies and global contextual connections, while the Transformer architecture is limited in its application to small medical datasets. To address these issues, we propose a new skin lesion segmentation network, SUTrans-NET, which combines CNN and Transformer in a parallel fashion to form a dual encoder, where both CNN and Transformer branches perform dynamic interactive fusion of image information in each layer. At the same time, we introduce our designed multi-grouping module SpatialGroupAttention (SGA) to complement the spatial and texture information of the Transformer branch, and utilize the Focus idea of YOLOV5 to construct the Patch Embedding module in the Transformer to prevent the loss of pixel accuracy. In addition, we design a decoder with full-scale information fusion capability to fully fuse shallow and deep features at different stages of the encoder. The effectiveness of our method is demonstrated on the ISIC 2016, ISIC 2017, ISIC 2018 and PH2 datasets and its advantages over existing methods are verified.
| S-EPMC11042008 | biostudies-literature
Project description:Convolutional neural networks (CNNs) have achieved remarkable success in automatically segmenting organs or lesions on 3D medical images. Recently, vision transformer networks have exhibited exceptional performance in 2D image classification tasks. Compared with CNNs, transformer networks have an appealing advantage of extracting long-range features due to their self-attention algorithm. Therefore, we propose a CNN-Transformer combined model, called BiTr-Unet, with specific modifications for brain tumor segmentation on multi-modal MRI scans. Our BiTr-Unet achieves good performance on the BraTS2021 validation dataset with median Dice score 0.9335, 0.9304 and 0.8899, and median Hausdor_ distance 2.8284, 2.2361 and 1.4142 for the whole tumor, tumor core, and enhancing tumor, respectively. On the BraTS2021 testing dataset, the corresponding results are 0.9257, 0.9350 and 0.8874 for Dice score, and 3, 2.2361 and 1.4142 for Hausdorff distance. The code is publicly available at https://github.com/JustaTinyDot/BiTr-Unet.
| S-EPMC9396958 | biostudies-literature
Project description:Cervical lesions pose a significant threat to women's health worldwide. Colposcopy is essential for screening and treating cervical lesions, but its effectiveness depends on the doctor's experience. Artificial intelligence-based solutions via colposcopy images have shown great potential in cervical lesions screening. However, some challenges still need to be addressed, such as low algorithm performance and lack of high-quality multi-modal datasets. Here, we established a multi-modal colposcopy dataset of 2,273 HPV+ patients, comprising original colposcopy images, acetic acid reactions at 60s and 120s, iodine staining, diagnostic reports, and pathological results. Utilizing this dataset, we developed CerviFusionNet, a hybrid architecture that merges convolutional neural networks and vision transformers to learn robust representations. We designed a temporal module to capture dynamic changes in acetic acid sequences, which can boost the model performance without sacrificing inference speed. Compared with several existing methods, CerviFusionNet demonstrated excellent accuracy and efficiency.
| S-EPMC11615576 | biostudies-literature
Project description:BackgroundThe coronavirus disease 2019 (COVID-19) spreads rapidly across the globe, seriously threatening the health of people all over the world. To reduce the diagnostic pressure of front-line doctors, an accurate and automatic lesion segmentation method is highly desirable in clinic practice.PurposeMany proposed two-dimensional (2D) methods for sliced-based lesion segmentation cannot take full advantage of spatial information in the three-dimensional (3D) volume data, resulting in limited segmentation performance. Three-dimensional methods can utilize the spatial information but suffer from long training time and slow convergence speed. To solve these problems, we propose an end-to-end hybrid-feature cross fusion network (HFCF-Net) to fuse the 2D and 3D features at three scales for the accurate segmentation of COVID-19 lesions.MethodsThe proposed HFCF-Net incorporates 2D and 3D subnets to extract features within and between slices effectively. Then the cross fusion module is designed to bridge 2D and 3D decoders at the same scale to fuse both types of features. The module consists of three cross fusion blocks, each of which contains a prior fusion path and a context fusion path to jointly learn better lesion representations. The former aims to explicitly provide the 3D subnet with lesion-related prior knowledge, and the latter utilizes the 3D context information as the attention guidance of the 2D subnet, which promotes the precise segmentation of the lesion regions. Furthermore, we explore an imbalance-robust adaptive learning loss function that includes image-level loss and pixel-level loss to tackle the problems caused by the apparent imbalance between the proportions of the lesion and non-lesion voxels, providing a learning strategy to dynamically adjust the learning focus between 2D and 3D branches during the training process for effective supervision.ResultExtensive experiments conducted on a publicly available dataset demonstrate that the proposed segmentation network significantly outperforms some state-of-the-art methods for the COVID-19 lesion segmentation, yielding a Dice similarity coefficient of 74.85%. The visual comparison of segmentation performance also proves the superiority of the proposed network in segmenting different-sized lesions.ConclusionsIn this paper, we propose a novel HFCF-Net for rapid and accurate COVID-19 lesion segmentation from chest computed tomography volume data. It innovatively fuses hybrid features in a cross manner for lesion segmentation, aiming to utilize the advantages of 2D and 3D subnets to complement each other for enhancing the segmentation performance. Benefitting from the cross fusion mechanism, the proposed HFCF-Net can segment the lesions more accurately with the knowledge acquired from both subnets.
| S-EPMC9088496 | biostudies-literature
Project description:Imaging speed is critical for photoacoustic microscopy as it affects the capability to capture dynamic biological processes and support real-time clinical applications. Conventional approaches for increasing imaging speed typically involve high-repetition-rate lasers, which pose a risk of thermal damage to samples. Here, we propose a deep-learning-driven optical-scanning undersampling method for photoacoustic remote sensing (PARS) microscopy, accelerating imaging acquisition while maintaining a constant laser repetition rate and reducing laser dosage. We develop a hybrid Transformer-Convolutional Neural Network, HTC-GAN, to address the challenges of both nonuniform sampling and motion misalignment inherent in optical-scanning undersampling. A mouse ear vasculature image dataset is created through our customized galvanometer-scanned PARS system to train and validate HTC-GAN. The network successfully restores high-quality images from 1/2-undersampled and 1/4-undersampled data, closely approximating the ground truth images. A series of performance experiments demonstrate that HTC-GAN surpasses the basic misalignment compensation algorithm, and standalone CNN or Transformer networks in terms of perceptual quality and quantitative metrics. Moreover, three-dimensional imaging results validate the robustness and versatility of the proposed optical-scanning undersampling imaging method across multiscale scanning modes. Our method achieves a fourfold improvement in PARS imaging speed without hardware upgrades, offering an available solution for enhancing imaging speed in other optical-scanning microscopic systems.
| S-EPMC11889609 | biostudies-literature
Project description:ObjectivesAccurate segmentation of focal cortical dysplasia (FCD) lesions from MR images plays an important role in surgical planning and decision but is still challenging for radiologists and clinicians. In this study, we introduce a novel transformer-based model, designed for the end-to-end segmentation of FCD lesions from multi-channel MR images.MethodsThe core innovation of our proposed model is the integration of a convolutional neural network-based encoder-decoder structure with a multiscale transformer to augment the feature representation of lesions in the global field of view. Transformer pathways, composed of memory- and computation-efficient dual-self-attention modules, leverage feature maps from varying depths of the encoder to discern long-range interdependencies among feature positions and channels, thereby emphasizing areas and channels relevant to lesions. The proposed model was trained and evaluated on a public-open dataset including MR images of 85 patients using both subject-level and voxel-level metrics.ResultsExperimental results indicate that our model offers superior performance both quantitatively and qualitatively. It successfully identified lesions in 82.4% of patients, with a low false-positive lesion cluster rate of 0.176 ± 0.381 per patient. Furthermore, the model achieved an average Dice coefficient of 0.410 ± 0.288, outperforming five established methods.ConclusionIntegration of the transformer could enhance the feature presentation and segmentation performance of FCD lesions. The proposed model has the potential to serve as a valuable assistive tool for physicians, enabling rapid and accurate identification of FCD lesions. The source code and pre-trained model weights are available at https://github.com/zhangxd0530/MS-DSA-NET .Critical relevance statementThis multiscale transformer-based model performs segmentation of focal cortical dysplasia lesions, aiming to help radiologists and clinicians make accurate and efficient preoperative evaluations of focal cortical dysplasia patients from MR images.Key pointsThe first transformer-based model was built to explore focal cortical dysplasia lesion segmentation. Integration of global and local features enhances the segmentation performance of lesions. A valuable benchmark for model development and comparative analyses was provided.
| S-EPMC11393231 | biostudies-literature
Project description:The robust segmentation of organs from the medical image is the key technique in medical image analysis for disease diagnosis. U-Net is a robust structure for medical image segmentation. However, U-Net adopts consecutive downsampling encoders to capture multiscale features, resulting in the loss of contextual information and insufficient recovery of high-level semantic features. In this paper, we present a new multibranch hybrid attention network (MHA-Net) to capture more contextual information and high-level semantic features. The main idea of our proposed MHA-Net is to use the multibranch hybrid attention feature decoder to recover more high-level semantic features. The lightweight pyramid split attention (PSA) module is used to connect the encoder and decoder subnetwork to obtain a richer multiscale feature map. We compare the proposed MHA-Net to state-of-art approaches on the DRIVE dataset, the fluoroscopic roentgenographic stereophotogrammetric analysis X-ray dataset, and the polyp dataset. The experimental results on different modal images reveal that our proposed MHA-Net provides better segmentation results than other segmentation approaches.
| S-EPMC9560845 | biostudies-literature
Project description:BackgroundSkin lesion segmentation plays a significant role in skin cancer diagnosis. However, due to the complex shapes, varying sizes, and different color depths, precise segmentation of skin lesions is a challenging task. Therefore, the aim of this study was to design a customized deep learning (DL) model for the precise segmentation of skin lesions, particularly for complex shapes and small target lesions.MethodsIn this study, an adaptive deformable fusion convolutional network (Seg-SkiNet) was proposed. Seg-SkiNet integrated dual-channel convolution encoder (Dual-Conv encoder), Multi-Scale-Multi-Receptive Field Extraction and Refinement (Multi2ER) module, and local-global information interaction fusion decoder (LGI-FSN decoder). In the Dual-Conv encoder, a Dual-Conv module was proposed and cascaded with max pooling in each layer to capture the features of complex-shaped skin lesions. The design of the Dual-Conv module not only effectively captured edge features of the lesions but also learned deep internal features of the lesions. The Multi2ER module was composed of an Atrous Spatial Pyramid Pooling (ASPP) module and an Attention Refinement Module (ARM), and integrated multi-scale features of small target lesions by expanding the receptive field of the convolutional kernel, thereby improving the learning and accurately segmentation of small target lesions. In the LGI-FSN decoder, we integrated convolution and Local-Global Attention Fusion (LGAF) module in each layer to enable interactive fusion of local-global information in feature maps while eliminating redundant feature information. Additionally, we designed a densely connected architecture that fuses the feature maps from a specific layer of the Dual-Conv encoder and all of its preceding layers into the corresponding layer of the LGI-FSN decoder, preventing information loss caused by pooling operations.ResultsWe validated the performance of Seg-SkiNet for skin lesion segmentation on three public datasets: International Skin Imaging Collaboration (ISIC)-2016, ISIC-2017, and ISIC-2018. The experimental results demonstrated that Seg-SkiNet achieved a Dice coefficient (DICE) of 93.66%, 89.44% and 92.29%, respectively.ConclusionsThe Seg-SkiNet model performed excellently in segmenting complex-shaped lesions and small target lesions.
| S-EPMC11744141 | biostudies-literature
Project description:Wide-field optical coherence tomography (OCT) imaging can enable monitoring of peripheral changes in the retina, beyond the conventional fields of view used in current clinical OCT imaging systems. However, wide-field scans can present significant challenges for retinal layer segmentation. Deep Convolutional Neural Networks (CNNs) have shown strong performance in medical imaging segmentation but typically require large-scale, high-quality, pixel-level annotated datasets to be effectively developed. To address this challenge, we propose an advanced semi-supervised learning framework that combines the detailed capabilities of convolutional networks with the broader perspective of transformers. This method efficiently leverages labelled and unlabelled data to reduce dependence on extensive, manually annotated datasets. We evaluated the model performance on a dataset of 74 volumetric OCT scans, each performed using a prototype swept-source OCT system following a wide-field scan protocol with a 15 × 9 mm field of view, comprising 11,750 labelled and 29,016 unlabelled images. Wide-field retinal layer segmentation using the semi-supervised approach show significant improvements (P-value < 0.001) of up to 11% against a UNet baseline model. Comparisons with a clinical spectral-domain-OCT system revealed significant correlations of up to 0.91 (P-value < 0.001) in retinal layer thickness measurements. These findings highlight the effectiveness of semi-supervised learning with cross-teaching between CNNs and transformers for automated OCT layer segmentation.
| S-EPMC11850926 | biostudies-literature
Project description:Various segmentation networks based on Swin Transformer have shown promise in medical segmentation tasks. Nonetheless, challenges such as lower accuracy and slower training convergence have persisted. To tackle these issues, we introduce a novel approach that combines the Swin Transformer and Deformable Transformer to enhance overall model performance. We leverage the Swin Transformer's window attention mechanism to capture local feature information and employ the Deformable Transformer to adjust sampling positions dynamically, accelerating model convergence and aligning it more closely with object shapes and sizes. By amalgamating both Transformer modules and incorporating additional skip connections to minimize information loss, our proposed model excels at rapidly and accurately segmenting CT or X-ray lung images. Experimental results demonstrate the remarkable, showcasing the significant prowess of our model. It surpasses the performance of the standalone Swin Transformer's Swin Unet and converges more rapidly under identical conditions, yielding accuracy improvements of 0.7% (resulting in 88.18%) and 2.7% (resulting in 98.01%) on the COVID-19 CT scan lesion segmentation dataset and Chest X-ray Masks and Labels dataset, respectively. This advancement has the potential to aid medical practitioners in early diagnosis and treatment decision-making.
| S-EPMC10909362 | biostudies-literature