A learning robot for cognitive camera control in minimally invasive surgery.
Ontology highlight
ABSTRACT: Background
We demonstrate the first self-learning, context-sensitive, autonomous camera-guiding robot applicable to minimally invasive surgery. The majority of surgical robots nowadays are telemanipulators without autonomous capabilities. Autonomous systems have been developed for laparoscopic camera guidance, however following simple rules and not adapting their behavior to specific tasks, procedures, or surgeons.Methods
The herein presented methodology allows different robot kinematics to perceive their environment, interpret it according to a knowledge base and perform context-aware actions. For training, twenty operations were conducted with human camera guidance by a single surgeon. Subsequently, we experimentally evaluated the cognitive robotic camera control. A VIKY EP system and a KUKA LWR 4 robot were trained on data from manual camera guidance after completion of the surgeon's learning curve. Second, only data from VIKY EP were used to train the LWR and finally data from training with the LWR were used to re-train the LWR.Results
The duration of each operation decreased with the robot's increasing experience from 1704 s ± 244 s to 1406 s ± 112 s, and 1197 s. Camera guidance quality (good/neutral/poor) improved from 38.6/53.4/7.9 to 49.4/46.3/4.1% and 56.2/41.0/2.8%.Conclusions
The cognitive camera robot improved its performance with experience, laying the foundation for a new generation of cognitive surgical robots that adapt to a surgeon's needs.
SUBMITTER: Wagner M
PROVIDER: S-EPMC8346448 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectiveTo develop a deep learning algorithm for anatomy recognition in thoracoscopic video frames from robot-assisted minimally invasive esophagectomy (RAMIE) procedures using deep learning.BackgroundRAMIE is a complex operation with substantial perioperative morbidity and a considerable learning curve. Automatic anatomy recognition may improve surgical orientation and recognition of anatomical structures and might contribute to reducing morbidity or learning curves. Studies regarding anatomy recognition in complex surgical procedures are currently lacking.MethodsEighty-three videos of consecutive RAMIE procedures between 2018 and 2022 were retrospectively collected at University Medical Center Utrecht. A surgical PhD candidate and an expert surgeon annotated the azygos vein and vena cava, aorta, and right lung on 1050 thoracoscopic frames. 850 frames were used for training of a convolutional neural network (CNN) to segment the anatomical structures. The remaining 200 frames of the dataset were used for testing the CNN. The Dice and 95% Hausdorff distance (95HD) were calculated to assess algorithm accuracy.ResultsThe median Dice of the algorithm was 0.79 (IQR = 0.20) for segmentation of the azygos vein and/or vena cava. A median Dice coefficient of 0.74 (IQR = 0.86) and 0.89 (IQR = 0.30) were obtained for segmentation of the aorta and lung, respectively. Inference time was 0.026 s (39 Hz). The prediction of the deep learning algorithm was compared with the expert surgeon annotations, showing an accuracy measured in median Dice of 0.70 (IQR = 0.19), 0.88 (IQR = 0.07), and 0.90 (0.10) for the vena cava and/or azygos vein, aorta, and lung, respectively.ConclusionThis study shows that deep learning-based semantic segmentation has potential for anatomy recognition in RAMIE video frames. The inference time of the algorithm facilitated real-time anatomy recognition. Clinical applicability should be assessed in prospective clinical studies.
| S-EPMC10322962 | biostudies-literature
Project description:Effective force modulation during tissue manipulation is important for ensuring safe, robot-assisted, minimally invasive surgery (RMIS). Strict requirements for in vivo applications have led to prior sensor designs that trade off ease of manufacture and integration against force measurement accuracy along the tool axis. Due to this trade-off, there are no commercial, off-the-shelf, 3-degrees-of-freedom (3DoF) force sensors for RMIS available to researchers. This makes it challenging to develop new approaches to indirect sensing and haptic feedback for bimanual telesurgical manipulation. We present a modular 3DoF force sensor that integrates easily with an existing RMIS tool. We achieve this by relaxing biocompatibility and sterilizability requirements and by using commercial load cells and common electromechanical fabrication techniques. The sensor has a range of ±5 N axially and ±3 N laterally with errors of below 0.15 N and maximum errors below 11% of the sensing range in all directions. During telemanipulation, a pair of jaw-mounted sensors achieved average errors below 0.15 N in all directions. It achieved an average grip force error of 0.156 N. The sensor is for bimanual haptic feedback and robotic force control in delicate tissue telemanipulation. As an open-source design, the sensors can be adapted to suit other non-RMIS robotic applications.
| S-EPMC10255999 | biostudies-literature
Project description:BackgroundAdequate pain control after open esophagectomy is associated with reduced complications, earlier recovery and higher patient satisfaction. While further developing surgical procedures like robot-assisted minimally invasive esophagectomy (RAMIE) it is relevant to adapt postoperative pain management. The primary question of this observational survey was whether one of the two standard treatments, thoracic epidural analgesia (TEA) or intravenous patient-controlled analgesia (PCA), is superior for pain control after RAMIE as the optimal pain management for these patients still remains unclear. Use of additional analgesics, changes in forced expiratory volume in 1 s (FEV1), postoperative complications and duration of intensive care and hospital stay were also analyzed.MethodsThis prospective observational pilot study analyzed 50 patients undergoing RAMIE (postoperative PCA with piritramide or TEA using bupivacaine; each n = 25). Patient reported pain using the numeric rating scale score and differences in FEV1 using a micro spirometer were measured at postoperative day 1, 3 and 7. Additional data of secondary endpoints were collected from patient charts.ResultsKey demographics, comorbidity, clinical and operative variables were equivalently distributed. Patients receiving TEA had lower pain scores and a longer-lasting pain relief. Moreover, TEA was an independent predictive variable for reduced length of hospital stay (HR -3.560 (95% CI: -6.838 to -0.282), p = 0.034).ConclusionsAlthough RAMIE leads to reduced surgical trauma, a less invasive pain therapy with PCA appears to be inferior compared to TEA in case of sufficient postoperative analgesia and length of hospital stay. According to the results of this observational pilot study analgesia with TEA provided better and longer-lasting pain relief compared to PCA. Further randomized controlled trials should be conducted to evaluate the optimal postoperative analgesic treatment for RAMIE.
| S-EPMC9988548 | biostudies-literature
Project description:BackgroundRobot-assisted open surgery (RA-OS) is now commonly used in traditional open-exposure spinal screw placement surgery. With the help of robots, robot-assisted minimally invasive surgery (RA-MIS) can achieve less bleeding and less tissue damage in percutaneous screw insertion. While the research comparing the safety and accuracy of screw placement between RA-MIS and RA-OS is insufficient. This study aims to compare the effects of RA-MIS and RA-OS in thoracic and lumbar spine.MethodsThis was a prospective cohort study evaluating 208 patients undergoing robot-assisted screw insertions from July 2020 to September 2021. Age, BMI, gender, screws accuracy, screws Gertzbein-Robbins grade, small joint invasion and perioperative outcomes (operation time, blood loss, postoperative hospital stay, comorbidity) were collected. A subgroup analysis was also performed according to disease, namely fracture, spondylolisthesis, and disc herniation. Data were analyzed using Stata/MP 14.0. Wilcoxon's signed rank test, Kruskal-Wallis test and Fisher's exact test were used for statistical tests and p < 0.05 was considered statistically significant.ResultsA total of 1030 screws were inserted; 368 minimally invasive screws and 662 open screws. The acceptability of screw insertion in the RA-MIS and RA-OS was 97.3% and 95.6% respectively. There was no statistical difference between the RA-MIS group and RA-OS group in age (p = 0.106), gender (p = 0.074), BMI (p = 0.181) and comorbidity (p = 0.203). Compared with RA-OS, RA-MIS had less blood loss (p < 0.001) and shorter postoperative hospital stay (p = 0.008). In the minimally invasive surgery group, the fracture subgroup had smaller screw deviation, less blood loss, and shorter operation time compared with the other subgroups (p < 0.01). Specifically, RA-MIS significantly reduced the postoperative hospital stay of patients with spondylolisthesis compared with RA-OS (p < 0.01).ConclusionRA-OS and RA-MIS had equal accuracy and safety. Compared with open surgery, minimally invasive surgery reduced blood loss in each subgroup and shortened the postoperative hospital stay in the spondylolisthesis subgroup. Compared with the other subgroups under minimally invasive surgery, the fracture subgroup had less blood loss and shorter operation time.Clinical trial registrationNCT04040868. Registered 1 March 2019, https://clinicaltrials.gov/ct2/show/NCT04040868?cond=Accuracy+Study+of+Robot-assisted+Screw+Insertion+in+Spinal+Surgery&draw=2&rank=1 .
| S-EPMC8832639 | biostudies-literature
Project description:BackgroundFor patients with esophageal adenocarcinoma or cancer of the gastroesophageal junction, radical esophagectomy with 2-field lymphadenectomy is the cornerstone of the multimodality treatment with curative intent. Both conventional minimally invasive esophagectomy (MIE) and robot assisted minimally invasive esophagectomy (RAMIE) were shown to be superior compared to open transthoracic esophagectomy considering postoperative complications. However, no randomized comparison exists between MIE and RAMIE in the Western World for patients with esophageal adenocarcinoma.MethodsThis is an investigator-initiated and investigator-driven multicenter randomized controlled parallel-group superiority trial. All adult patients (age ≥ 18 and ≤ 90 years) with histologically proven, surgically resectable (cT1-4a, N0-3, M0) esophageal adenocarcinoma of the intrathoracic esophagus or adenocarcinoma of the gastroesophageal junction and with European Clinical Oncology Group performance status 0, 1 or 2 will be assessed for eligibility and included after obtaining informed consent. Patients (n = 218) with resectable esophageal adenocarcinoma of the intrathoracic esophagus or adenocarcinoma of the gastroesophageal junction are randomized to either RAMIE (n = 109) or MIE (n = 109). The primary outcome of this study is the total number of resected abdominal and mediastinal lymph nodes specified per lymph node station.ConclusionThis is the first randomized controlled trial designed to compare RAMIE to MIE as surgical treatment for resectable esophageal adenocarcinoma or adenocarcinoma of the gastroesophageal junction in the Western World. The hypothesis of the proposed study is that RAMIE will result in a higher abdominal and mediastinal lymph node yield specified per station compared to conventional MIE. Short-term results and the primary endpoint (total number of resected abdominal and mediastinal lymph nodes per lymph node station) will be analyzed and published after discharge of the last randomized patient within this trial.Trial registrationClinicalTrials.gov Identifier: NCT04306458 . Registered 13th March 2020, https://clinicaltrials.gov/ct2/show/NCT04306458; Date of first enrolment 18.01.2021; Target sample size 218; Recruitment status: Recruiting; Protocol version 2; Issue date 10.03.2020; Rev. 02.02.2021; Authors ET, PCvdS, PPG.
| S-EPMC8474742 | biostudies-literature
Project description:This article reviews the principles and different techniques used to perform minimally invasive strabismus surgery (MISS). This term is used for strabismus surgeries minimizing tissue disruption. Muscles are not accessed through one large opening, but using several keyhole openings placed where needed for the surgical steps. If necessary, tunnels are created between cuts, which will allow performing additional surgical steps. To keep the keyhole openings small, transconjunctival suturing techniques are used. The cuts are always placed as far away from the limbus as feasible. This will reduce the risk for postoperative corneal complications and it will ensure that all cuts will be covered by the eyelids, minimizing postoperative visibility of surgery and patient discomfort. Benefits from minimizing anatomical disruption between the muscle and the surrounding tissue are a better preservation of muscle function, less swelling, and pain, and more ease to perform reoperations. MISS openings allow to perform all types of strabismus surgeries, namely rectus muscle recessions, resections, plications, reoperations, retroequatorial myopexias, transpositions, oblique muscle recessions, or plications, and adjustable sutures, even in the presence of restricted motility.
| S-EPMC4330290 | biostudies-other
Project description:Since their introduction, it has been demonstrated that minimally invasive aortic valve replacement (MIAVR) approaches are safe and effective for the treatment of aortic valve diseases. To date, the main advantage of these approaches is represented by the reduced surgical trauma, with a subsequent reduced complication rate and faster recovery. This makes such approaches an appealing choice also for frail patients [obese, aged, chronic obstructive pulmonary disease (COPD)]. The standardization of the minimally invasive techniques, together with the implementation of preoperative workup and anesthesiological intra- and post-operative care, led to an amelioration of surgical results and reduction of surgical times. Moreover, the improvement of surgical technology and the introduction of new devices such as sutureless and rapid deployment (SURD) valves, has helped the achievement of comparable results to traditional surgery. However, transcatheter technologies are nowadays more and more important in the treatment of aortic valve disease, also in low risk patients. For this reason surgeons should put new efforts for further reducing the surgical trauma in the future, even taking inspiration from other disciplines. In this review, we aim to present a review of literature evidences regarding minimally invasive treatment of aortic diseases, also reflecting our personal experience with MIAVR techniques. This review could represent a tool for a well-structured patient assessment and preoperative planning, in order to safely carrying out an MIAVR procedure with satisfactory outcomes.
| S-EPMC8024826 | biostudies-literature
Project description:Tricuspid valve disease carries a very unfavorable prognosis when medically treated. Despite that, surgical intervention is still underperformed for tricuspid valve disease due to the reported high morbidity and mortality from a sternotomy approach. This had led to a shift towards maximizing medical therapy for right ventricular failure and, as a result, a more significant delay in surgical referrals with surgical risks when patients are finally referred. Tricuspid valve patients usually have other co-morbidities resulting from their systemic venous congestion and low flow cardiac output. Minimally invasive tricuspid valve surgery provides less tissue injury and, as a result, less trauma during surgery. This provides a hope for both patients and treating doctors to be more open for providing this procedure with less complications. Isolated minimally invasive tricuspid valve surgery is still not performed as widely as expected. This can be partly due to the adverse outcomes historically labelled to tricuspid valve surgery or by the long journey of learning the surgical team would need to commit to with a minimal access approach. In this article we will review the perioperative pathway, and outcomes of isolated minimally invasive tricuspid valve surgery in the available English literature.
| S-EPMC8024798 | biostudies-literature
Project description:BackgroundWith Surgomics, we aim for personalized prediction of the patient's surgical outcome using machine-learning (ML) on multimodal intraoperative data to extract surgomic features as surgical process characteristics. As high-quality annotations by medical experts are crucial, but still a bottleneck, we prospectively investigate active learning (AL) to reduce annotation effort and present automatic recognition of surgomic features.MethodsTo establish a process for development of surgomic features, ten video-based features related to bleeding, as highly relevant intraoperative complication, were chosen. They comprise the amount of blood and smoke in the surgical field, six instruments, and two anatomic structures. Annotation of selected frames from robot-assisted minimally invasive esophagectomies was performed by at least three independent medical experts. To test whether AL reduces annotation effort, we performed a prospective annotation study comparing AL with equidistant sampling (EQS) for frame selection. Multiple Bayesian ResNet18 architectures were trained on a multicentric dataset, consisting of 22 videos from two centers.ResultsIn total, 14,004 frames were tag annotated. A mean F1-score of 0.75 ± 0.16 was achieved for all features. The highest F1-score was achieved for the instruments (mean 0.80 ± 0.17). This result is also reflected in the inter-rater-agreement (1-rater-kappa > 0.82). Compared to EQS, AL showed better recognition results for the instruments with a significant difference in the McNemar test comparing correctness of predictions. Moreover, in contrast to EQS, AL selected more frames of the four less common instruments (1512 vs. 607 frames) and achieved higher F1-scores for common instruments while requiring less training frames.ConclusionWe presented ten surgomic features relevant for bleeding events in esophageal surgery automatically extracted from surgical video using ML. AL showed the potential to reduce annotation effort while keeping ML performance high for selected features. The source code and the trained models are published open source.
| S-EPMC10615926 | biostudies-literature
Project description:Background:The development of minimally invasive surgery has initiated many changes in the surgical treatment of esophageal cancer (EC) patients. The aim of this study was to compare the short-term outcomes of robotic-assisted minimally invasive esophagectomy (RAMIE), video-assisted minimally invasive esophagectomy (VAMIE), and open esophagectomy (OE). Methods:Our study included patients who had undergone McKeown esophagectomy at Tianjin Medical University Cancer Institute and Hospital between January 2016 and December 2018. We analyzed clinical baseline data, as well as perioperative and pathological outcomes. Results:A total of 312 cases met the inclusion criteria (OE: 77, VAMIE: 144, RAMIE: 91). The OE group had a greater number of late-stage patients as well as those who received the neo-adjuvant therapy, compared with the other two groups (P=0.001). The procedure time in the OE group was also shorter by approximately 20 minutes (P=0.021). Total blood loss was significantly lower in the two MIE groups (P=0.004) than in the OE group. There were no differences in the total number of dissected lymph nodes between the three groups (OE: 24.09±10.77, VAMIE: 23.07±10.18, RAMIE: 22.84±8.37, P=0.680). Both the lymph node number (P=0.155) and achievement rate (P=0.190) in the right recurrent laryngeal nerve (RLN) area were comparable between the three groups. However, in the left RLN area, minimally invasive approaches resulted in a higher number of harvested lymph nodes (P=0.032) and greater achievement rate (P=0.018). Neither MIE procedure increased the incidence of postoperative complications. Conclusions:Minimally invasive surgery could guarantee the quality of bilateral RLN lymphadenectomy without increasing postoperative complications, especially in RAMIE patients. The rational choice of different surgical approaches would improve both safety and oncological outcomes for patients.
| S-EPMC7139097 | biostudies-literature