Dataset Information

Artificial Intelligence for Intraoperative Guidance: Using Semantic Segmentation to Identify Surgical Anatomy During Laparoscopic Cholecystectomy.

ABSTRACT:

Objective

The aim of this study was to develop and evaluate the performance of artificial intelligence (AI) models that can identify safe and dangerous zones of dissection, and anatomical landmarks during laparoscopic cholecystectomy (LC).

Summary background data

Many adverse events during surgery occur due to errors in visual perception and judgment leading to misinterpretation of anatomy. Deep learning, a subfield of AI, can potentially be used to provide real-time guidance intraoperatively.

Methods

Deep learning models were developed and trained to identify safe (Go) and dangerous (No-Go) zones of dissection, liver, gallbladder, and hepatocystic triangle during LC. Annotations were performed by 4 high-volume surgeons. AI predictions were evaluated using 10-fold cross-validation against annotations by expert surgeons. Primary outcomes were intersection- over-union (IOU) and F1 score (validated spatial correlation indices), and secondary outcomes were pixel-wise accuracy, sensitivity, specificity, ± standard deviation.

Results

AI models were trained on 2627 random frames from 290 LC videos, procured from 37 countries, 136 institutions, and 153 surgeons. Mean IOU, F1 score, accuracy, sensitivity, and specificity for the AI to identify Go zones were 0.53 (±0.24), 0.70 (±0.28), 0.94 (±0.05), 0.69 (±0.20). and 0.94 (±0.03), respectively. For No-Go zones, these metrics were 0.71 (±0.29), 0.83 (±0.31), 0.95 (±0.06), 0.80 (±0.21), and 0.98 (±0.05), respectively. Mean IOU for identification of the liver, gallbladder, and hepatocystic triangle were: 0.86 (±0.12), 0.72 (±0.19), and 0.65 (±0.22), respectively.

Conclusions

AI can be used to identify anatomy within the surgical field. This technology may eventually be used to provide real-time guidance and minimize the risk of adverse events.

SUBMITTER: Madani A

PROVIDER: S-EPMC8186165 | biostudies-literature | 2022 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Artificial Intelligence for Intraoperative Guidance: Using Semantic Segmentation to Identify Surgical Anatomy During Laparoscopic Cholecystectomy.

Madani Amin A Namazi Babak B Altieri Maria S MS Hashimoto Daniel A DA Rivera Angela Maria AM Pucher Philip H PH Navarrete-Welton Allison A Sankaranarayanan Ganesh G Brunt L Michael LM Okrainec Allan A Alseidi Adnan A

Annals of surgery 20201113 2

<h4>Objective</h4>The aim of this study was to develop and evaluate the performance of artificial intelligence (AI) models that can identify safe and dangerous zones of dissection, and anatomical landmarks during laparoscopic cholecystectomy (LC).<h4>Summary background data</h4>Many adverse events during surgery occur due to errors in visual perception and judgment leading to misinterpretation of anatomy. Deep learning, a subfield of AI, can potentially be used to provide real-time guidance intr ...[more]

PMID: 33196488

Similar Datasets

Project description:ImportanceComplications that arise from phacoemulsification procedures can lead to worse visual outcomes. Real-time image processing with artificial intelligence tools can extract data to deliver surgical guidance, potentially enhancing the surgical environment.ObjectiveTo evaluate the ability of a deep neural network to track the pupil, identify the surgical phase, and activate specific computer vision tools to aid the surgeon during phacoemulsification cataract surgery by providing visual feedback in real time.Design, setting, and participantsThis cross-sectional study evaluated deidentified surgical videos of phacoemulsification cataract operations performed by faculty and trainee surgeons in a university-based ophthalmology department between July 1, 2020, and January 1, 2021, in a population-based cohort of patients.ExposuresA region-based convolutional neural network was used to receive frames from the video source and, in real time, locate the pupil and in parallel identify the surgical phase being performed. Computer vision-based algorithms were applied according to the phase identified, providing visual feedback to the surgeon.Main outcomes and measuresOutcomes were area under the receiver operator characteristic curve and area under the precision-recall curve for surgical phase classification and Dice score (harmonic mean of the precision and recall [sensitivity]) for detection of the pupil boundary. Network performance was assessed as video output in frames per second. A usability survey was administered to volunteer cataract surgeons previously unfamiliar with the platform.ResultsThe region-based convolutional neural network model achieved area under the receiver operating characteristic curve values of 0.996 for capsulorhexis, 0.972 for phacoemulsification, 0.997 for cortex removal, and 0.880 for idle phase recognition. The final algorithm reached a Dice score of 90.23% for pupil segmentation and a mean (SD) processing speed of 97 (34) frames per second. Among the 11 cataract surgeons surveyed, 8 (72%) were mostly or extremely likely to use the current platform during surgery for complex cataract.Conclusions and relevanceA computer vision approach using deep neural networks was able to pupil track, identify the surgical phase being executed, and activate surgical guidance tools. These results suggest that an artificial intelligence-based surgical guidance platform has the potential to enhance the surgeon experience in phacoemulsification cataract surgery. This proof-of-concept investigation suggests that a pipeline from a surgical microscope could be integrated with neural networks and computer vision tools to provide surgical guidance in real time.

Project description:To assess the role of laparoscopic ultrasound (LUS) as a substitute for intraoperative cholangiography (IOC) during cholecystectomy.We present a MEDLINE and PubMed literature search, having used the key-words "laparoscopic intraoperative ultrasound" and "laparoscopic cholecystectomy". All relevant English language publications from 2000 to 2016 were identified, with data extracted for the role of LUS in the anatomical delineation of the biliary tract, detection of common bile duct stones (CBDS), prevention or early detection of biliary duct injury (BDI), and incidental findings during laparoscopic cholecystectomy. Data for the role of LUS vs IOC in complex situations (i.e., inflammatory disease/fibrosis) were specifically analyzed.We report data from eighteen reports, 13 prospective non-randomized trials, 5 retrospective trials, and two meta-analyses assessing diagnostic accuracy, with one analysis also assessing costs, duration of the examination, and anatomical mapping. Overall, LUS was shown to provide highly sensitive mapping of the extra-pancreatic biliary anatomy in 92%-100% of patients, with more difficulty encountered in delineation of the intra-pancreatic segment of the biliary tract (73.8%-98%). Identification of vascular and biliary variations has been documented in two studies. Although inflammatory disease hampered accuracy, LUS was still advantageous vs IOC in patients with obscured anatomy. LUS can be performed before any dissection and repeated at will to guide the surgeon especially when hilar mapping is difficult due to fibrosis and inflammation. In two studies LUS prevented conversion in 91% of patients with difficult scenarios. Considering CBDS detection, LUS sensitivity and specificity were 76%-100% and 96.2%-100%, respectively. LUS allowed the diagnosis/treatment of incidental findings of adjacent organs. No valuable data for BDI prevention or detection could be retrieved, even if no BDI was documented in the reports analyzed. Literature analysis proved LUS as a safe, quick, non-irradiating, cost-effective technique, which is comparatively well known although largely under-utilized, probably due to the perception of a difficult learning curve.We highlight the advantages and limitations of laparoscopic ultrasound during cholecystectomy, and underline its value in difficult scenarios when the anatomy is obscured.

Project description:Background and objectiveRecently, deep learning algorithms, including convolutional neural networks (CNNs), have shown remarkable progress in medical imaging analysis. Semantic segmentation, which segments an unknown image into different parts and objects, has potential applications in robotic surgery in areas where artificial intelligence (AI) can be applied, such as in AI-assisted surgery, surgeon training, and skill assessment. We aimed to investigate the performance of a CNN-based deep learning model in real-time segmentation in robot-assisted radical prostatectomy (RALP).MethodsIntraoperative videos of RALP procedures were obtained. The reinforcement U-Net model was used for segmentation. Segmentation of the images of instruments, bladder, prostate, and seminal vesicle-vas deferens was performed. The dataset was preprocessed and split randomly into training, validation, and test data in a 7:2:1 ratio. Dice coefficient, intersection over union (IoU), and accuracy by class, which are commonly used in medical image segmentation, were calculated to evaluate the performance of the model.Key findings and limitationsFrom 120 patient videos, 2400 images were selected for RALP procedures. The mean Dice scores for the identification of the instruments, bladder, prostate, and seminal vesicle-vas deferens were 0.96, 0.74, 0.85, and 0.84, respectively. Overall, when applied to the test data, the model had a mean Dice coefficient value of 0.85, IoU of 0.77, and accuracy of 0.85. Limitations included the sample size, lack of diversity in the methods of surgery, incomplete surgical processes, and lack of external validation.Conclusions and clinical implicationsThe CNN-based segmentation provides accurate real-time recognition of surgical instruments and anatomy in RALP. Deep learning algorithms can be used to identify anatomy within the surgical field and could potentially be used to provide real-time guidance in robotic surgery.Patient summaryWe demonstrate the potential effectiveness of deep learning segmentation in robotic prostatectomy procedures. Deep learning algorithms could be used to identify anatomical structures within the surgical field and may provide real-time guidance in robotic surgery.

Project description:The recent developments in artificial intelligence have the potential to facilitate new research methods in ecology. Especially Deep Convolutional Neural Networks (DCNNs) have been shown to outperform other approaches in automatic image analyses. Here we apply a DCNN to facilitate quantitative wood anatomical (QWA) analyses, where the main challenges reside in the detection of a high number of cells, in the intrinsic variability of wood anatomical features, and in the sample quality. To properly classify and interpret features within the images, DCNNs need to undergo a training stage. We performed the training with images from transversal wood anatomical sections, together with manually created optimal outputs of the target cell areas. The target species included an example for the most common wood anatomical structures: four conifer species; a diffuse-porous species, black alder (Alnus glutinosa L.); a diffuse to semi-diffuse-porous species, European beech (Fagus sylvatica L.); and a ring-porous species, sessile oak (Quercus petraea Liebl.). The DCNN was created in Python with Pytorch, and relies on a Mask-RCNN architecture. The developed algorithm detects and segments cells, and provides information on the measurement accuracy. To evaluate the performance of this tool we compared our Mask-RCNN outputs with U-Net, a model architecture employed in a similar study, and with ROXAS, a program based on traditional image analysis techniques. First, we evaluated how many target cells were correctly recognized. Next, we assessed the cell measurement accuracy by evaluating the number of pixels that were correctly assigned to each target cell. Overall, the "learning process" defining artificial intelligence plays a key role in overcoming the issues that are usually manually solved in QWA analyses. Mask-RCNN is the model that better detects which are the features characterizing a target cell when these issues occur. In general, U-Net did not attain the other algorithms' performance, while ROXAS performed best for conifers, and Mask-RCNN showed the highest accuracy in detecting target cells and segmenting lumen areas of angiosperms. Our research demonstrates that future software tools for QWA analyses would greatly benefit from using DCNNs, saving time during the analysis phase, and providing a flexible approach that allows model retraining.

Dataset Information

Artificial Intelligence for Intraoperative Guidance: Using Semantic Segmentation to Identify Surgical Anatomy During Laparoscopic Cholecystectomy.

Objective

Summary background data

Methods

Results

Conclusions

Publications

Artificial Intelligence for Intraoperative Guidance: Using Semantic Segmentation to Identify Surgical Anatomy During Laparoscopic Cholecystectomy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets