Browse
Submit Data
Databases
API
Help

Dataset Information

27 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Mask then classify: multi-instance segmentation for surgical instruments.

ABSTRACT:

Purpose

The detection and segmentation of surgical instruments has been a vital step for many applications in minimally invasive surgical robotics. Previously, the problem was tackled from a semantic segmentation perspective, yet these methods fail to provide good segmentation maps of instrument types and do not contain any information on the instance affiliation of each pixel. We propose to overcome this limitation by using a novel instance segmentation method which first masks instruments and then classifies them into their respective type.

Methods

We introduce a novel method for instance segmentation where a pixel-wise mask of each instance is found prior to classification. An encoder-decoder network is used to extract instrument instances, which are then separately classified using the features of the previous stages. Furthermore, we present a method to incorporate instrument priors from surgical robots.

Results

Experiments are performed on the robotic instrument segmentation dataset of the 2017 endoscopic vision challenge. We perform a fourfold cross-validation and show an improvement of over 18% to the previous state-of-the-art. Furthermore, we perform an ablation study which highlights the importance of certain design choices and observe an increase of 10% over semantic segmentation methods.

Conclusions

We have presented a novel instance segmentation method for surgical instruments which outperforms previous semantic segmentation-based methods. Our method further provides a more informative output of instance level information, while retaining a precise segmentation mask. Finally, we have shown that robotic instrument priors can be used to further increase the performance.

SUBMITTER: Kurmann T

PROVIDER: S-EPMC8260538 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Instance Segmentation of Multiple Myeloma Cells Using Deep-Wise Data Augmentation and Mask R-CNN.

Project description:Multiple myeloma is a condition of cancer in the bone marrow that can lead to dysfunction of the body and fatal expression in the patient. Manual microscopic analysis of abnormal plasma cells, also known as multiple myeloma cells, is one of the most commonly used diagnostic methods for multiple myeloma. However, as it is a manual process, it consumes too much effort and time. Besides, it has a higher chance of human errors. This paper presents a computer-aided detection and segmentation of myeloma cells from microscopic images of the bone marrow aspiration. Two major contributions are presented in this paper. First, different Mask R-CNN models using different images, including original microscopic images, contrast-enhanced images and stained cell images, are developed to perform instance segmentation of multiple myeloma cells. As a second contribution, a deep-wise augmentation, a deep learning-based data augmentation method, is applied to increase the performance of Mask R-CNN models. Based on the experimental findings, the Mask R-CNN model using contrast-enhanced images combined with the proposed deep-wise data augmentation provides a superior performance compared to other models. It achieves a mean precision of 0.9973, mean recall of 0.8631, and mean intersection over union (IOU) of 0.9062.

| S-EPMC8774909 | biostudies-literature

Instance segmentation convolutional neural network based on multi-scale attention mechanism.

Project description:Instance segmentation is more challenging and difficult than object detection and semantic segmentation. It paves the way for the realization of a complete scene understanding, and has been widely used in robotics, automatic driving, medical care, and other aspects. However, there are some problems in instance segmentation methods, such as the low detection efficiency for low-resolution objects and the slow detection speed of images with complex backgrounds. To solve these problems, this paper proposes an instance segmentation method with multi-scale attention, which is called a Hybrid Kernel Mask R-CNN. Firstly, the hybrid convolution kernel is constructed by combining different kernels and groups, which can complement each other to extract rich information. Secondly, a multi-scale attention mechanism is designed by assign weights to different convolution kernels, which can retain more important information. After the introduction of our strategy, the network is more inclined to focus on the low-resolution objects in the image. The proposed method achieves the best accuracy over the anchor-based method. To verify the universality of the model, we test Hybrid Kernel Mask R-CNN on Balloon, xBD and COCO datasets. The test results exceed the state of art methods. And the visualization results show our method can extract low-resolution objects effectively.

| S-EPMC8794127 | biostudies-literature

Explainable multi-instance and multi-task learning for COVID-19 diagnosis and lesion segmentation in CT images.

Project description:Coronavirus Disease 2019 (COVID-19) still presents a pandemic trend globally. Detecting infected individuals and analyzing their status can provide patients with proper healthcare while protecting the normal population. Chest CT (computed tomography) is an effective tool for screening of COVID-19. It displays detailed pathology-related information. To achieve automated COVID-19 diagnosis and lung CT image segmentation, convolutional neural networks (CNNs) have become mainstream methods. However, most of the previous works consider automated diagnosis and image segmentation as two independent tasks, in which some focus on lung fields segmentation and the others focus on single-lesion segmentation. Moreover, lack of clinical explainability is a common problem for CNN-based methods. In such context, we develop a multi-task learning framework in which the diagnosis of COVID-19 and multi-lesion recognition (segmentation of CT images) are achieved simultaneously. The core of the proposed framework is an explainable multi-instance multi-task network. The network learns task-related features adaptively with learnable weights, and gives explicable diagnosis results by suggesting local CT images with lesions as additional evidence. Then, severity assessment of COVID-19 and lesion quantification are performed to analyze patient status. Extensive experimental results on real-world datasets show that the proposed framework outperforms all the compared approaches for COVID-19 diagnosis and multi-lesion segmentation.

| S-EPMC9235304 | biostudies-literature

Nanopore basecalling from a perspective of instance segmentation.

Project description:BackgroundNanopore sequencing is a rapidly developing third-generation sequencing technology, which can generate long nucleotide reads of molecules within a portable device in real-time. Through detecting the change of ion currency signals during a DNA/RNA fragment's pass through a nanopore, genotypes are determined. Currently, the accuracy of nanopore basecalling has a higher error rate than the basecalling of short-read sequencing. Through utilizing deep neural networks, the-state-of-the art nanopore basecallers achieve basecalling accuracy in a range from 85% to 95%.ResultIn this work, we proposed a novel basecalling approach from a perspective of instance segmentation. Different from previous approaches of doing typical sequence labeling, we formulated the basecalling problem as a multi-label segmentation task. Meanwhile, we proposed a refined U-net model which we call UR-net that can model sequential dependencies for a one-dimensional segmentation task. The experiment results show that the proposed basecaller URnano achieves competitive results on the in-species data, compared to the recently proposed CTC-featured basecallers.ConclusionOur results show that formulating the basecalling problem as a one-dimensional segmentation task is a promising approach, which does basecalling and segmentation jointly.

| S-EPMC7178565 | biostudies-literature

A lightweight segmentation network for endoscopic surgical instruments based on edge refinement and efficient self-attention.

Project description:In robot-assisted surgical systems, surgical instrument segmentation is a critical task that provides important information for surgeons to make informed decisions and ensure surgical safety. However, current mainstream models often lack precise segmentation edges and suffer from an excess of parameters, rendering their deployment challenging. To address these issues, this article proposes a lightweight semantic segmentation model based on edge refinement and efficient self-attention. The proposed model utilizes a lightweight densely connected network for feature extraction, which is able to extract high-quality semantic information with fewer parameters. The decoder combines a feature pyramid module with an efficient criss-cross self-attention module. This fusion integrates multi-scale data, strengthens focus on surgical instrument details, and enhances edge segmentation accuracy. To train and evaluate the proposed model, the authors developed a private dataset of endoscopic surgical instruments. It containing 1,406 images for training, 469 images for validation and 469 images for testing. The proposed model performs well on this dataset with only 466 K parameters, achieving a mean Intersection over Union (mIoU) of 97.11%. In addition, the model was trained on public datasets Kvasir-instrument and Endovis2017. Excellent results of 93.24% and 95.83% were achieved on the indicator mIoU, respectively. The superiority and effectiveness of the method are proved. Experimental results show that the proposed model has lower parameters and higher accuracy than other state-of-the-art models. The proposed model thus lays the foundation for further research in the field of surgical instrument segmentation.

| S-EPMC10803021 | biostudies-literature

WormSwin: Instance segmentation of C. elegans using vision transformer.

Project description:The possibility to extract motion of a single organism from video recordings at a large-scale provides means for the quantitative study of its behavior, both individual and collective. This task is particularly difficult for organisms that interact with one another, overlap, and occlude parts of their bodies in the recording. Here we propose WormSwin-an approach to extract single animal postures of Caenorhabditis elegans (C. elegans) from recordings of many organisms in a single microscope well. Based on transformer neural network architecture our method segments individual worms across a range of videos and images generated in different labs. Our solutions offers accuracy of 0.990 average precision ([Formula: see text]) and comparable results on the benchmark image dataset BBBC010. Finally, it allows to segment challenging overlapping postures of mating worms with an accuracy sufficient to track the organisms with a simple tracking heuristic. An accurate and efficient method for C. elegans segmentation opens up new opportunities for studying of its behaviors previously inaccessible due to the difficulty in the worm extraction from the video frames.

| S-EPMC10328995 | biostudies-literature

CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation.

Project description:Automated cellular instance segmentation is a process utilized for accelerating biological research for the past two decades, and recent advancements have produced higher quality results with less effort from the biologist. Most current endeavors focus on completely cutting the researcher out of the picture by generating highly generalized models. However, these models invariably fail when faced with novel data, distributed differently than the ones used for training. Rather than approaching the problem with methods that presume the availability of large amounts of target data and computing power for retraining, in this work we address the even greater challenge of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently. A large set of results show that 3 to 5 annotations lead to models with accuracy that: 1) significantly mitigate the covariate shift effects; 2) matches or surpasses other adaptation methods; 3) even approaches methods that have been fully retrained on the target distribution. The adaptation training is only a few minutes, paving a path towards a balance between model performance, computing requirements and expert-level annotation needs.

| S-EPMC10760785 | biostudies-literature

YOLOv5-LiNet: A lightweight network for fruits instance segmentation.

Project description:To meet the goals of computer vision-based understanding of images adopted in agriculture for improved fruit production, it is expected of a recognition model to be robust against complex and changeable environment, fast, accurate and lightweight for a low power computing platform deployment. For this reason, a lightweight YOLOv5-LiNet model for fruit instance segmentation to strengthen fruit detection was proposed based on the modified YOLOv5n. The model included Stem, Shuffle_Block, ResNet and SPPF as backbone network, PANet as neck network, and EIoU loss function to enhance detection performance. YOLOv5-LiNet was compared to YOLOv5n, YOLOv5-GhostNet, YOLOv5-MobileNetv3, YOLOv5-LiNetBiFPN, YOLOv5-LiNetC, YOLOv5-LiNet, YOLOv5-LiNetFPN, YOLOv5-Efficientlite, YOLOv4-tiny and YOLOv5-ShuffleNetv2 lightweight model including Mask-RCNN. The obtained results show that YOLOv5-LiNet having the box accuracy of 0.893, instance segmentation accuracy of 0.885, weight size of 3.0 MB and real-time detection of 2.6 ms combined together outperformed other lightweight models. Therefore, the YOLOv5-LiNet model is robust, accurate, fast, applicable to low power computing devices and extendable to other agricultural products for instance segmentation.

| S-EPMC9980778 | biostudies-literature

Automatic tooth periodontal ligament segmentation of cone beam computed tomography based on instance segmentation network.

Project description:ObjectiveThe three-dimensional morphological structures of periodontal ligaments (PDLs) are important data for periodontal, orthodontic, prosthodontic, and implant interventions. This study aimed to employ a deep learning (DL) algorithm to segment the PDL automatically in cone-beam computed tomography (CBCT).MethodThis was a retrospective study. We randomly selected 389 patients and 1734 axial CBCT images from the CBCT database, and designed a fully automatic PDL segmentation computer-aided model based on instance segmentation Mask R-CNN network. The labels of the model training were 'teeth' and 'alveolar bone', and the 'PDL' is defined as the region where the 'teeth' and 'alveolar bone' overlap. The model's segmentation performance was evaluated using CBCT data from eight patients outside the database.ResultsQualitative evaluation indicates that the PDL segmentation accuracy of incisors, canines, premolars, wisdom teeth, and implants reached 100%. The segmentation accuracy of molars was 96.4%. Quantitative evaluation indicates that the mIoU and mDSC of PDL segmentation were 0.667 ± 0.015 (>0.6) and 0.799 ± 0.015 (>0.7) respectively.ConclusionThis study analysed a unique approach to AI-driven automatic segmentation of PDLs on CBCT imaging, possibly enabling chair-side measurements of PDLs to facilitate periodontists, orthodontists, prosthodontists, and implantologists in more efficient and accurate diagnosis and treatment planning.

| S-EPMC10827460 | biostudies-literature

RC-Net: Regression Correction for End-To-End Chromosome Instance Segmentation.

Project description:Precise segmentation of chromosome in the real image achieved by a microscope is significant for karyotype analysis. The segmentation of image is usually achieved by a pixel-level classification task, which considers different instances as different classes. Many instance segmentation methods predict the Intersection over Union (IoU) through the head branch to correct the classification confidence. Their effectiveness is based on the correlation between branch tasks. However, none of these methods consider the correlation between input and output in branch tasks. Herein, we propose a chromosome instance segmentation network based on regression correction. First, we adopt two head branches to predict two confidences that are more related to localization accuracy and segmentation accuracy to correct the classification confidence, which reduce the omission of predicted boxes in NMS. Furthermore, a NMS algorithm is further designed to screen the target segmentation mask with the IoU of the overlapping instance, which reduces the omission of predicted masks in NMS. Moreover, given the fact that the original IoU loss function is not sensitive to the wrong segmentation, K-IoU loss function is defined to strengthen the penalty of the wrong segmentation, which rationalizes the loss of mis-segmentation and effectively prevents wrong segmentation. Finally, an ablation experiment is designed to evaluate the effectiveness of the chromosome instance segmentation network based on regression correction, which shows that our proposed method can effectively enhance the performance in automatic chromosome segmentation tasks and provide a guarantee for end-to-end karyotype analysis.

| S-EPMC9158129 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data