Dataset Information

Co-axial Projective Imaging for Augmented Reality Telementoring in Skin Cancer Surgery.

ABSTRACT: Telemedicine has the potential to overcome the unequal distribution of medical resources worldwide. In this study, we report the second-generation co-axial projective imaging (CPI-2) system featured with orthotopic image projection for augmented reality surgical telementoring. The CPI-2 system can acquire surgical scene images from the local site, transmit them wirelessly to the remote site, and project the virtual annotations drawn by a remote expert with great accuracy to the surgical field. The performance characteristics of the CPI-2 system are quantitatively verified in benchtop experiments. The ex vivo study that compares the CPI-2 system and a monitor-based telementoring system shows that the CPI-2 system can reduce the focus shift and avoid subjective mapping of the instructions from a monitor to the real-world scene, thereby saving operation time and achieving precise teleguidance. The clinical feasibility of the CPI-2 system is validated in teleguided skin cancer surgery. Our ex vivo and in vivo experiment results imply the improved performance of surgical telementoring, and the clinical utility of deploying the CPI-2 system for surgical interventions in resource-limited settings. The CPI-2 system has the potential to reduce healthcare disparities in remote areas with limited resources.

SUBMITTER: Zhang F

PROVIDER: S-EPMC9255459 | biostudies-literature | 2022 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Co-axial Projective Imaging for Augmented Reality Telementoring in Skin Cancer Surgery.

Zhang Fan F Contreras Carlo M CM Shao Pengfei P Zhao Liping L Wu Bingxuan B Li Chenmeng C Lin Feng F Zhong Xiaohong X Lang Zhongliang Z Liu Peng P Xu Ronald X RX

Annals of biomedical engineering 20220705 12

Telemedicine has the potential to overcome the unequal distribution of medical resources worldwide. In this study, we report the second-generation co-axial projective imaging (CPI-2) system featured with orthotopic image projection for augmented reality surgical telementoring. The CPI-2 system can acquire surgical scene images from the local site, transmit them wirelessly to the remote site, and project the virtual annotations drawn by a remote expert with great accuracy to the surgical field. T ...[more]

PMID: 35788468

Similar Datasets

Project description:BackgroundOver the last decade, augmented reality (AR) has emerged in health care as a tool for visualizing data and enhancing simulation learning. AR, which has largely been explored for communication and collaboration in nonhealth contexts, could play a role in shaping future remote medical services and training. This review summarized existing studies implementing AR in real-time telemedicine and telementoring to create a foundation for health care providers and technology developers to understand future opportunities in remote care and education.ObjectiveThis review described devices and platforms that use AR for real-time telemedicine and telementoring, the tasks for which AR was implemented, and the ways in which these implementations were evaluated to identify gaps in research that provide opportunities for further study.MethodsWe searched PubMed, Scopus, Embase, and MEDLINE to identify English-language studies published between January 1, 2012, and October 18, 2022, implementing AR technology in a real-time interaction related to telemedicine or telementoring. The search terms were "augmented reality" OR "AR" AND "remote" OR "telemedicine" OR "telehealth" OR "telementoring." Systematic reviews, meta-analyses, and discussion-based articles were excluded from analysis.ResultsA total of 39 articles met the inclusion criteria and were categorized into themes of patient evaluation, medical intervention, and education. In total, 20 devices and platforms using AR were identified, with common features being the ability for remote users to annotate, display graphics, and display their hands or tools in the local user's view. Common themes across the studies included consultation and procedural education, with surgery, emergency, and hospital medicine being the most represented specialties. Outcomes were most often measured using feedback surveys and interviews. The most common objective measures were time to task completion and performance. Long-term outcome and resource cost measurements were rare. Across the studies, user feedback was consistently positive for perceived efficacy, feasibility, and acceptability. Comparative trials demonstrated that AR-assisted conditions had noninferior reliability and performance and did not consistently extend procedure times compared with in-person controls.ConclusionsStudies implementing AR in telemedicine and telementoring demonstrated the technology's ability to enhance access to information and facilitate guidance in multiple health care settings. However, AR's role as an alternative to current telecommunication platforms or even in-person interactions remains to be validated, with many disciplines and provider-to-nonprovider uses still lacking robust investigation. Additional studies comparing existing methods may offer more insight into this intersection, but the early stage of technical development and the lack of standardized tools and adoption have hindered the conduct of larger longitudinal and randomized controlled trials. Overall, AR has the potential to complement and advance the capabilities of remote medical care and learning, creating unique opportunities for innovator, provider, and patient involvement.

Project description:BackgroundRemote surgeons use telementoring technologies to provide real-time guidance during minimally invasive surgeries (MIS). Such technologies are continuously improving with the integration of dynamic augmented reality (AR) cues. This includes virtual overlays of hand gestures, pointers, and surgical tools onto the operating surgeon's view. The operating surgeon comprehends this augmented information to operate on the patient. Thus, understanding these dynamic AR cues (either during surgical training or live surgery) is crucial.ObjectiveIn this paper, we aimed to review the existing telementoring technologies that use dynamic AR cues during MIS. This review describes the MIS (including surgery type, specialty, procedure, and clinical trial), the telementoring system, the dynamic AR cues generated by these systems, and evaluation of the technology in terms of technical aspects, user perceptions, skills gained, and patient outcomes.MethodsA scoping review was conducted using PubMed, Web of Science, Scopus, IEEE Xplore, and ACM Digital Library databases. The search terms included "telementoring," "minimally invasive surgery," and "augmented reality" without restrictions imposed on the publication year. Articles covering telementoring using dynamic AR cues during MIS, including laparoscopic and robot-assisted, were identified.ResultsA total of 21 articles were included and categorized based on type of surgery, the telementoring technology used, and evaluation of the technology. Most of the articles reported on laparoscopic suturing performed using synthetic phantoms. Hand gestures and surgical tools were the most frequently used dynamic AR cues (10 articles on each cue), while the mentors and mentees primarily consisted of experienced surgeons and medical students. The studies assessing the telementoring technologies were either descriptive (7 articles) or analytical (14 articles) where it was compared against no cue, prerecorded visual cue, in-person guidance, audio cue, or static AR cue. Outcomes were measured mostly using skills gained (13 articles) and user perception about the telementoring system.ConclusionsIn general, telementoring using dynamic AR cues resulted in positive outcomes during MIS. In some cases, they were considered on par with conventional methods such as audio cues and in-person guidance. Further randomized controlled trials are required to objectively assess its clinical benefits.

Project description:BackgroundTechnological advances have transformed vascular intervention in recent decades. In particular, improvements in imaging and data processing have allowed for the development of increasingly complex endovascular and hybrid interventions. Augmented reality (AR) is a subject of growing interest in surgery, with the potential to improve clinicians' understanding of 3D anatomy and aid in the processing of real-time information. This study hopes to elucidate the potential impact of AR technology in the rapidly evolving fields of vascular and endovascular surgery.ObjectiveThe aim of this review is to summarize the fundamental concepts of AR technologies and conduct a scoping review of the impact of AR and mixed reality in vascular and endovascular surgery.MethodsA systematic search of MEDLINE, Scopus, and Embase was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. All studies written in English from inception until January 8, 2021, were included in the search. Combinations of the following keywords were used in the systematic search string: ("augmented reality" OR "hololens" OR "image overlay" OR "daqri" OR "magic leap" OR "immersive reality" OR "extended reality" OR "mixed reality" OR "head mounted display") AND ("vascular surgery" OR "endovascular"). Studies were selected through a blinded process between 2 investigators (JE and AS) and assessed using data quality tools.ResultsAR technologies have had a number of applications in vascular and endovascular surgery. Most studies (22/32, 69%) used 3D imaging of computed tomography angiogram-derived images of vascular anatomy to augment clinicians' anatomical understanding during procedures. A wide range of AR technologies were used, with heads up fusion imaging and AR head-mounted displays being the most commonly applied clinically. AR applications included guiding open, robotic, and endovascular surgery while minimizing dissection, improving procedural times, and reducing radiation and contrast exposure.ConclusionsAR has shown promising developments in the field of vascular and endovascular surgery, with potential benefits to surgeons and patients alike. These include reductions in patient risk and operating times as well as in contrast and radiation exposure for radiological interventions. Further technological advances are required to overcome current limitations, including processing capacity and vascular deformation by instrumentation.

Project description:Purpose: Augmented reality (AR) may allow vitreoretinal surgeons to leverage microscope-integrated digital imaging systems to analyze and highlight key retinal anatomic features in real time, possibly improving safety and precision during surgery. By employing convolutional neural networks (CNNs) for retina vessel segmentation, a retinal coordinate system can be created that allows pre-operative images of capillary non-perfusion or retinal breaks to be digitally aligned and overlayed upon the surgical field in real time. Such technology may be useful in assuring thorough laser treatment of capillary non-perfusion or in using pre-operative optical coherence tomography (OCT) to guide macular surgery when microscope-integrated OCT (MIOCT) is not available. Methods: This study is a retrospective analysis involving the development and testing of a novel image-registration algorithm for vitreoretinal surgery. Fifteen anonymized cases of pars plana vitrectomy with epiretinal membrane peeling, along with corresponding preoperative fundus photographs and optical coherence tomography (OCT) images, were retrospectively collected from the Mayo Clinic database. We developed a TPU (Tensor-Processing Unit)-accelerated CNN for semantic segmentation of retinal vessels from fundus photographs and subsequent real-time image registration in surgical video streams. An iterative patch-wise cross-correlation (IPCC) algorithm was developed for image registration, with a focus on optimizing processing speeds and maintaining high spatial accuracy. The primary outcomes measured were processing speed in frames per second (FPS) and the spatial accuracy of image registration, quantified by the Dice coefficient between registered and manually aligned images. Results: When deployed on an Edge TPU, the CNN model combined with our image-registration algorithm processed video streams at a rate of 14 FPS, which is superior to processing rates achieved on other standard hardware configurations. The IPCC algorithm efficiently aligned pre-operative and intraoperative images, showing high accuracy in comparison to manual registration. Conclusions: This study demonstrates the feasibility of using TPU-accelerated CNNs for enhanced AR in vitreoretinal surgery.

Dataset Information

Co-axial Projective Imaging for Augmented Reality Telementoring in Skin Cancer Surgery.

Publications

Co-axial Projective Imaging for Augmented Reality Telementoring in Skin Cancer Surgery.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets