Metric Learning for Image Registration.
Ontology highlight
ABSTRACT:
SUBMITTER: Niethammer M
PROVIDER: S-EPMC7286567 | biostudies-literature | 2019 Jun
REPOSITORIES: biostudies-literature
Publications
Metric Learning for Image Registration.
Niethammer Marc M Kwitt Roland R Vialard François-Xavier FX
Proceedings. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 20190601
Image registration is a key technique in medical image analysis to estimate deformations between image pairs. A good deformation model is important for high-quality estimates. However, most existing approaches use ad-hoc deformation models chosen for mathematical convenience rather than to capture observed data variation. Recent deep learning approaches learn deformation models directly from data. However, they provide limited control over the spatial regularity of transformations. Instead of le ...[more]
Similar Datasets
Project description:The United States National Institutes of Health (NIH) commit significant support to open-source data and software resources in order to foment reproducibility in the biomedical imaging sciences. Here, we report and evaluate a recent product of this commitment: Advanced Neuroimaging Tools (ANTs), which is approaching its 2.0 release. The ANTs open source software library consists of a suite of state-of-the-art image registration, segmentation and template building tools for quantitative morphometric analysis. In this work, we use ANTs to quantify, for the first time, the impact of similarity metrics on the affine and deformable components of a template-based normalization study. We detail the ANTs implementation of three similarity metrics: squared intensity difference, a new and faster cross-correlation, and voxel-wise mutual information. We then use two-fold cross-validation to compare their performance on openly available, manually labeled, T1-weighted MRI brain image data of 40 subjects (UCLA's LPBA40 dataset). We report evaluation results on cortical and whole brain labels for both the affine and deformable components of the registration. Results indicate that the best ANTs methods are competitive with existing brain extraction results (Jaccard=0.958) and cortical labeling approaches. Mutual information affine mapping combined with cross-correlation diffeomorphic mapping gave the best cortical labeling results (Jaccard=0.669±0.022). Furthermore, our two-fold cross-validation allows us to quantify the similarity of templates derived from different subgroups. Our open code, data and evaluation scripts set performance benchmark parameters for this state-of-the-art toolkit. This is the first study to use a consistent transformation framework to provide a reproducible evaluation of the isolated effect of the similarity metric on optimal template construction and brain labeling.
| S-EPMC3065962 | biostudies-literature
Project description:Groupwise image registration tackles biases that can potentially arise from inappropriate template selection. It typically involves simultaneous registration of a cohort of images to a common space that is not specified a priori. Existing groupwise registration methods are computationally complex and are only effective for image populations without large anatomical variations. In this paper, we propose a deep learning framework to rapidly estimate large deformations between images to significantly reduce structural variability. Specifically, we employ a multi-level graph coarsening method to agglomerate similar images into clusters, each represented by an exemplar image. We then use a deep learning framework to predict the initial deformations between images. Warping with the estimated deformations brings the images closer in the image manifold and their alignment can be further refined using conventional groupwise registration algorithms. We evaluated the effectiveness of our method in groupwise registration of MR brain images and compared it against state-of-the-art groupwise registration methods. Experimental results indicate that deformation initialization enables groupwise registration to converge significantly faster with competitive accuracy, therefore facilitates large-scale imaging studies.
| S-EPMC7373822 | biostudies-literature
Project description:PURPOSE:In cardiac interventions, such as cardiac resynchronization therapy (CRT), image guidance can be enhanced by involving preoperative models. Multimodality 3D/2D registration for image guidance, however, remains a significant research challenge for fundamentally different image data, i.e., MR to X-ray. Registration methods must account for differences in intensity, contrast levels, resolution, dimensionality, field of view. Furthermore, same anatomical structures may not be visible in both modalities. Current approaches have focused on developing modality-specific solutions for individual clinical use cases, by introducing constraints, or identifying cross-modality information manually. Machine learning approaches have the potential to create more general registration platforms. However, training image to image methods would require large multimodal datasets and ground truth for each target application. METHODS:This paper proposes a model-to-image registration approach instead, because it is common in image-guided interventions to create anatomical models for diagnosis, planning or guidance prior to procedures. An imitation learning-based method, trained on 702 datasets, is used to register preoperative models to intraoperative X-ray images. RESULTS:Accuracy is demonstrated on cardiac models and artificial X-rays generated from CTs. The registration error was [Formula: see text] on 1000 test cases, superior to that of manual ([Formula: see text]) and gradient-based ([Formula: see text]) registration. High robustness is shown in 19 clinical CRT cases. CONCLUSION:Besides the proposed methods feasibility in a clinical environment, evaluation has shown good accuracy and high robustness indicating that it could be applied in image-guided interventions.
| S-EPMC6096758 | biostudies-literature
Project description:Medical image registration is a vital component of many medical procedures, such as image-guided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research. The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.
| S-EPMC8611468 | biostudies-literature
Project description:In adaptive radiation therapy of prostate cancer, fast and accurate registration between the planning image and treatment images of the patient is of essential importance. With the authors' recently developed deformable surface model, prostate boundaries in each treatment image can be rapidly segmented and their correspondences (or relative deformations) to the prostate boundaries in the planning image are also established automatically. However, the dense correspondences on the nonboundary regions, which are important especially for transforming the treatment plan designed in the planning image space to each treatment image space, are remained unresolved. This paper presents a novel approach to learn the statistical correlation between deformations of prostate boundary and nonboundary regions, for rapidly estimating deformations of the nonboundary regions when given the deformations of the prostate boundary at a new treatment image.The main contributions of the proposed method lie in the following aspects. First, the statistical deformation correlation will be learned from both current patient and other training patients, and further updated adaptively during the radiotherapy. Specifically, in the initial treatment stage when the number of treatment images collected from the current patient is small, the statistical deformation correlation is mainly learned from other training patients. As more treatment images are collected from the current patient, the patient-specific information will play a more important role in learning patient-specific statistical deformation correlation to effectively reflect prostate deformation of the current patient during the treatment. Eventually, only the patient-specific statistical deformation correlation is used to estimate dense correspondences when a sufficient number of treatment images have been acquired from the current patient. Second, the statistical deformation correlation will be learned by using a multiple linear regression (MLR) model, i.e., ridge regression (RR) model, which has the best prediction accuracy than other MLR models such as canonical correlation analysis (CCA) and principal component regression (PCR).To demonstrate the performance of the proposed method, we first evaluate its registration accuracy by comparing the deformation field predicted by our method with the deformation field estimated by the thin plate spline (TPS) based correspondence interpolation method on 306 serial prostate CT images of 24 patients. The average predictive error on the voxels around 5 mm of prostate boundary is 0.38 mm for our method of RR-based correlation model. Also, the corresponding maximum error is 2.89 mm. We then compare the speed for deformation interpolation by different methods. When considering the larger region of interest (ROI) with the size of 512 × 512 × 61, our method takes 24.41 seconds to interpolate the dense deformation field while TPS method needs 6.7 minutes; when considering a small ROI (surrounding prostate) with size of 112 × 110 × 93, our method takes 1.80 seconds, while TPS method needs 25 seconds.Experimental results show that the proposed method can achieve much faster registration speed yet with comparable registration accuracy, compared to the TPS-based correspondence (or deformation) interpolation approach.
| S-EPMC3215689 | biostudies-other
Project description:Convolutional neural networks (CNNs) offer a promising means to achieve fast deformable image registration with accuracy comparable to conventional, physics-based methods. A persistent question with CNN methods, however, is whether they will be able to generalize to data outside of the training set. We investigated this question of mismatch between train and test data with respect to first- and second-order image statistics (e.g., spatial resolution, image noise, and power spectrum). A UNet-based architecture was built and trained on simulated CT images for various conditions of image noise (dose), spatial resolution, and deformation magnitude. Target registration error was measured as a function of the difference in statistical properties between the test and training data. Generally, registration error is minimized when the training data exactly match the statistics of the test data; however, networks trained with data exhibiting a diversity in statistical characteristics generalized well across the range of statistical conditions considered. Furthermore, networks trained on simulated image content with first- and second-order statistics selected to match that of real anatomical data were shown to provide reasonable registration performance on real anatomical content, offering potential new means for data augmentation. Characterizing the behavior of a CNN in the presence of statistical mismatch is an important step in understanding how these networks behave when deployed on new, unobserved data. Such characterization can inform decisions on whether retraining is necessary and can guide the data collection and/or augmentation process for training.
| S-EPMC6916745 | biostudies-literature
Project description:Neuroimage registration is crucial for brain morphometric analysis and treatment efficacy evaluation. However, existing advanced registration algorithms such as FLIRT and ANTs are not efficient enough for clinical use. In this paper, a GPU implementation of FLIRT with the correlation ratio (CR) as the similarity metric and a GPU accelerated correlation coefficient (CC) calculation for the symmetric diffeomorphic registration of ANTs have been developed. The comparison with their corresponding original tools shows that our accelerated algorithms can greatly outperform the original algorithm in terms of computational efficiency. This paper demonstrates the great potential of applying these registration tools in clinical applications.
| S-EPMC4564209 | biostudies-literature
Project description:PURPOSE:Probe-based confocal laser endomicroscopy (pCLE) is a recent imaging modality that allows performing in vivo optical biopsies. The design of pCLE hardware, and its reliance on an optical fibre bundle, fundamentally limits the image quality with a few tens of thousands fibres, each acting as the equivalent of a single-pixel detector, assembled into a single fibre bundle. Video registration techniques can be used to estimate high-resolution (HR) images by exploiting the temporal information contained in a sequence of low-resolution (LR) images. However, the alignment of LR frames, required for the fusion, is computationally demanding and prone to artefacts. METHODS:In this work, we propose a novel synthetic data generation approach to train exemplar-based Deep Neural Networks (DNNs). HR pCLE images with enhanced quality are recovered by the models trained on pairs of estimated HR images (generated by the video registration algorithm) and realistic synthetic LR images. Performance of three different state-of-the-art DNNs techniques were analysed on a Smart Atlas database of 8806 images from 238 pCLE video sequences. The results were validated through an extensive image quality assessment that takes into account different quality scores, including a Mean Opinion Score (MOS). RESULTS:Results indicate that the proposed solution produces an effective improvement in the quality of the obtained reconstructed image. CONCLUSION:The proposed training strategy and associated DNNs allows us to perform convincing super-resolution of pCLE images.
| S-EPMC5973979 | biostudies-literature
Project description:In this paper, a novel image registration method is proposed to achieve accurate registration between images having large shape differences with the help of a set of appropriate intermediate templates. We first demonstrate that directionality is a key factor in both pairwise image registration and groupwise registration, which is defined in this paper to describe the influence of the registration direction and paths on the registration performance. In our solution, the intermediate template selection and intermediate template guided registration are two coherent steps with directionality being considered. To take advantage of the directionality, a directed graph is built based on the asymmetric distance defined on all ordered image pairs in the image population, which is fundamentally different from the undirected graph with symmetric distance metrics in all previous methods, and the shortest distance between template and subject on the directed graph is calculated. The allocated directed path can be thus utilized to better guide the registration by successively registering the subject through the intermediate templates one by one on the path towards the template. The proposed directed graph based solution can also be used in groupwise registration. Specifically, by building a minimum spanning arborescence (MSA) on the directed graph, the population center, i.e., a selected template, as well as the directed registration paths from all the rest of images to the population center, is determined simultaneously. The performance of directed graph based registration algorithm is demonstrated by the spatial normalization on both synthetic dataset and real brain MR images. It is shown that our method can achieve more accurate registration results than both the undirected graph based solution and the direct pairwise registration.
| S-EPMC3272327 | biostudies-other
Project description:Accurate and robust non-rigid registration of pre-procedure magnetic resonance (MR) imaging to intra-procedure trans-rectal ultrasound (TRUS) is critical for image-guided biopsies of prostate cancer. Prostate cancer is one of the most prevalent forms of cancer and the second leading cause of cancer-related death in men in the United States. TRUS-guided biopsy is the current clinical standard for prostate cancer diagnosis and assessment. State-of-the-art, clinical MR-TRUS image fusion relies upon semi-automated segmentations of the prostate in both the MR and the TRUS images to perform non-rigid surface-based registration of the gland. Segmentation of the prostate in TRUS imaging is itself a challenging task and prone to high variability. These segmentation errors can lead to poor registration and subsequently poor localization of biopsy targets, which may result in false-negative cancer detection. In this paper, we present a non-rigid surface registration approach to MR-TRUS fusion based on a statistical deformation model (SDM) of intra-procedural deformations derived from clinical training data. Synthetic validation experiments quantifying registration volume of interest overlaps of the PI-RADS parcellation standard and tests using clinical landmark data demonstrate that our use of an SDM for registration, with median target registration error of 2.98 mm, is significantly more accurate than the current clinical method. Furthermore, we show that the low-dimensional SDM registration results are robust to segmentation errors that are not uncommon in clinical TRUS data.
| S-EPMC5514316 | biostudies-literature