Dataset Information

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective.

ABSTRACT: Image-based navigation is widely considered the next frontier of minimally invasive surgery. It is believed that image-based navigation will increase the access to reproducible, safe, and high-precision surgery as it may then be performed at acceptable costs and effort. This is because image-based techniques avoid the need of specialized equipment and seamlessly integrate with contemporary workflows. Furthermore, it is expected that image-based navigation techniques will play a major role in enabling mixed reality environments, as well as autonomous and robot-assisted workflows. A critical component of image guidance is 2D/3D registration, a technique to estimate the spatial relationships between 3D structures, e.g., preoperative volumetric imagery or models of surgical instruments, and 2D images thereof, such as intraoperative X-ray fluoroscopy or endoscopy. While image-based 2D/3D registration is a mature technique, its transition from the bench to the bedside has been restrained by well-known challenges, including brittleness with respect to optimization objective, hyperparameter selection, and initialization, difficulties in dealing with inconsistencies or multiple objects, and limited single-view performance. One reason these challenges persist today is that analytical solutions are likely inadequate considering the complexity, variability, and high-dimensionality of generic 2D/3D registration problems. The recent advent of machine learning-based approaches to imaging problems that, rather than specifying the desired functional mapping, approximate it using highly expressive parametric models holds promise for solving some of the notorious challenges in 2D/3D registration. In this manuscript, we review the impact of machine learning on 2D/3D registration to systematically summarize the recent advances made by introduction of this novel technology. Grounded in these insights, we then offer our perspective on the most pressing needs, significant open problems, and possible next steps.

SUBMITTER: Unberath M

PROVIDER: S-EPMC8436154 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Surgical reduction of pelvic dislocation is a challenging procedure with poor long-term prognosis if reduction does not accurately restore natural morphology. The procedure often requires long fluoroscopic exposure times and trial-and-error to achieve accurate reduction. We report a method to automatically compute the target pose of dislocated bones in preoperative CT and provide 3D guidance of reduction using routine 2D fluoroscopy. A pelvic statistical shape model (SSM) and a statistical pose model (SPM) were formed from an atlas of 40 pelvic CT images. Multi-body bone segmentation was achieved by mapping the SSM to a preoperative CT via an active shape model. The target reduction pose for the dislocated bone is estimated by fitting the poses of undislocated bones to the SPM. Intraoperatively, multiple bones are registered to fluoroscopy images via 3D-2D registration to obtain 3D pose estimates from 2D images. The method was examined in three studies: (1) a simulation study of 40 CT images simulating a range of dislocation patterns; (2) a pelvic phantom study with controlled dislocation of the left innominate bone; (3) a clinical case study investigating feasibility in images acquired during pelvic reduction surgery. Experiments investigated the accuracy of registration as a function of initialization error (capture range), image quality (radiation dose and image noise), and field of view (FOV) size. The simulation study achieved target pose estimation with translational error of median 2.3 mm (1.4 mm interquartile range, IQR) and rotational error of 2.1° (1.3° IQR). 3D-2D registration yielded 0.3 mm (0.2 mm IQR) in-plane and 0.3 mm (0.2 mm IQR) out-of-plane translational error, with in-plane capture range of ±50 mm and out-of-plane capture range of ±120 mm. The phantom study demonstrated 3D-2D target registration error of 2.5 mm (1.5 mm IQR), and the method was robust over a large dose range, down to 5 [Formula: see text]Gy/frame (an order of magnitude lower than the nominal fluoroscopic dose). The clinical feasibility study demonstrated accurate registration with both preoperative and intraoperative radiographs, yielding 3.1 mm (1.0 mm IQR) projection distance error with robust performance for FOV ranging from 340 × 340 mm2 to 170 × 170 mm2 (at the image plane). The method demonstrated accurate estimation of the target reduction pose in simulation, phantom, and a clinical feasibility study for a broad range of dislocation patterns, initialization error, dose levels, and FOV size. The system provides a novel means of guidance and assessment of pelvic reduction from routinely acquired preoperative CT and intraoperative fluoroscopy. The method has the potential to reduce radiation dose by minimizing trial-and-error and to improve outcomes by guiding more accurate reduction of joint dislocations.

Project description:In adaptive radiation therapy of prostate cancer, fast and accurate registration between the planning image and treatment images of the patient is of essential importance. With the authors' recently developed deformable surface model, prostate boundaries in each treatment image can be rapidly segmented and their correspondences (or relative deformations) to the prostate boundaries in the planning image are also established automatically. However, the dense correspondences on the nonboundary regions, which are important especially for transforming the treatment plan designed in the planning image space to each treatment image space, are remained unresolved. This paper presents a novel approach to learn the statistical correlation between deformations of prostate boundary and nonboundary regions, for rapidly estimating deformations of the nonboundary regions when given the deformations of the prostate boundary at a new treatment image.The main contributions of the proposed method lie in the following aspects. First, the statistical deformation correlation will be learned from both current patient and other training patients, and further updated adaptively during the radiotherapy. Specifically, in the initial treatment stage when the number of treatment images collected from the current patient is small, the statistical deformation correlation is mainly learned from other training patients. As more treatment images are collected from the current patient, the patient-specific information will play a more important role in learning patient-specific statistical deformation correlation to effectively reflect prostate deformation of the current patient during the treatment. Eventually, only the patient-specific statistical deformation correlation is used to estimate dense correspondences when a sufficient number of treatment images have been acquired from the current patient. Second, the statistical deformation correlation will be learned by using a multiple linear regression (MLR) model, i.e., ridge regression (RR) model, which has the best prediction accuracy than other MLR models such as canonical correlation analysis (CCA) and principal component regression (PCR).To demonstrate the performance of the proposed method, we first evaluate its registration accuracy by comparing the deformation field predicted by our method with the deformation field estimated by the thin plate spline (TPS) based correspondence interpolation method on 306 serial prostate CT images of 24 patients. The average predictive error on the voxels around 5 mm of prostate boundary is 0.38 mm for our method of RR-based correlation model. Also, the corresponding maximum error is 2.89 mm. We then compare the speed for deformation interpolation by different methods. When considering the larger region of interest (ROI) with the size of 512 × 512 × 61, our method takes 24.41 seconds to interpolate the dense deformation field while TPS method needs 6.7 minutes; when considering a small ROI (surrounding prostate) with size of 112 × 110 × 93, our method takes 1.80 seconds, while TPS method needs 25 seconds.Experimental results show that the proposed method can achieve much faster registration speed yet with comparable registration accuracy, compared to the TPS-based correspondence (or deformation) interpolation approach.

Project description:Our purpose was to develop a neural network-based registration quality evaluator (RQE) that can improve the 2D/3D image registration robustness for pediatric patient setup in external beam radiotherapy. Orthogonal daily setup X-ray images of six pediatric patients with brain tumors receiving proton therapy treatments were retrospectively registered with their treatment planning computed tomography (CT) images. A neural network-based pattern classifier was used to determine whether a registration solution was successful based on geometric features of the similarity measure values near the point-of-solution. Supervised training and test datasets were generated by rigidly registering a pair of orthogonal daily setup X-ray images to the treatment planning CT. The best solution for each registration task was selected from 50 optimizing attempts that differed only by the randomly generated initial transformation parameters. The distance from each individual solution to the best solution in the normalized parametrical space was compared to a user-defined error tolerance to determine whether that solution was acceptable. A supervised training was then used to train the RQE. Performance of the RQE was evaluated using test dataset consisting of registration results that were not used in training. The RQE was integrated with our in-house 2D/3D registration system and its performance was evaluated using the same patient dataset. With an optimized sampling step size (i.e., 5 mm) in the feature space, the RQE has the sensitivity and the specificity in the ranges of 0.865-0.964 and 0.797-0.990, respectively, when used to detect registration error with mean voxel displacement (MVD) greater than 1 mm. The trial-to-acceptance ratio of the integrated 2D/3D registration system, for all patients, is equal to 1.48. The final acceptance ratio is 92.4%. The proposed RQE can potentially be used in a 2D/3D rigid image registration system to improve the overall robustness by rejecting unsuccessful registration solutions. The RQE is not patient-specific, so a single RQE can be constructed and used for a particular application (e.g., the registration for images acquired on the same anatomical site). Implementation of the RQE in a 2D/3D registration system is clinically feasible.

Project description:To determine the localization uncertainties associated with 2-dimensional/3-dimensional image registration in comparison to 3-dimensional/3-dimensional image registration in 6 dimensions on a Varian Edge Linac under various imaging conditions.The systematic errors in 6 dimensions were assessed by comparing automatic 2-dimensional/3-dimensional (kV/MV vs computed tomography) with 3-dimensional/3-dimensional (cone beam computed tomography vs computed tomography) image registrations under various conditions encountered in clinical applications. The 2-dimensional/3-dimensional image registration uncertainties for 88 patients with different treatment sites including intracranial and extracranial were evaluated by statistically analyzing 2-dimensional/3-dimensional pretreatment verification shifts of 192 fractions in stereotactic radiosurgery and stereotactic body radiotherapy.The systematic errors of 2-dimensional/3-dimensional image registration using kV-kV, MV-kV, and MV-MV image pairs were within 0.3 mm and 0.3° for the translational and rotational directions within a 95% confidence interval. No significant difference ( P > .05) in target localization was observed with various computed tomography slice thicknesses (0.8, 1, 2, and 3 mm). Two-dimensional/3-dimensional registration had the best accuracy when pattern intensity and content filter were used. For intracranial sites, means ± standard deviations of translational errors were -0.20 ± 0.70 mm, 0.04 ± 0.50 mm, and 0.10 ± 0.40 mm for the longitudinal, lateral, and vertical directions, respectively. For extracranial sites, means ± standard deviations of translational errors were -0.04 ± 1.00 mm, 0.2 ± 1.0 mm, and 0.1 ± 1.0 mm for the longitudinal, lateral, and vertical directions, respectively. Two-dimensional/3-dimensional image registration for intracranial and extracranial sites had comparable systematic errors that were approximately 0.2 mm in the translational direction and 0.08° in the rotational direction.The standard 2-dimensional/3-dimensional image registration tool available on the Varian Edge radiosurgery device, a state-of-the-art system, is helpful for robust and accurate target positioning for image-guided stereotactic radiosurgery.

Dataset Information

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets