Dataset Information

Learning-based deformable image registration: effect of statistical mismatch between train and test images.

ABSTRACT: Convolutional neural networks (CNNs) offer a promising means to achieve fast deformable image registration with accuracy comparable to conventional, physics-based methods. A persistent question with CNN methods, however, is whether they will be able to generalize to data outside of the training set. We investigated this question of mismatch between train and test data with respect to first- and second-order image statistics (e.g., spatial resolution, image noise, and power spectrum). A UNet-based architecture was built and trained on simulated CT images for various conditions of image noise (dose), spatial resolution, and deformation magnitude. Target registration error was measured as a function of the difference in statistical properties between the test and training data. Generally, registration error is minimized when the training data exactly match the statistics of the test data; however, networks trained with data exhibiting a diversity in statistical characteristics generalized well across the range of statistical conditions considered. Furthermore, networks trained on simulated image content with first- and second-order statistics selected to match that of real anatomical data were shown to provide reasonable registration performance on real anatomical content, offering potential new means for data augmentation. Characterizing the behavior of a CNN in the presence of statistical mismatch is an important step in understanding how these networks behave when deployed on new, unobserved data. Such characterization can inform decisions on whether retraining is necessary and can guide the data collection and/or augmentation process for training.

SUBMITTER: Ketcha MD

PROVIDER: S-EPMC6916745 | biostudies-literature | 2019 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Learning-based deformable image registration: effect of statistical mismatch between train and test images.

Ketcha Michael D MD De Silva Tharindu T Han Runze R Uneri Ali A Vogt Sebastian S Kleinszig Gerhard G Siewerdsen Jeffrey H JH

Journal of medical imaging (Bellingham, Wash.) 20191001 4

Convolutional neural networks (CNNs) offer a promising means to achieve fast deformable image registration with accuracy comparable to conventional, physics-based methods. A persistent question with CNN methods, however, is whether they will be able to generalize to data outside of the training set. We investigated this question of mismatch between train and test data with respect to first- and second-order image statistics (e.g., spatial resolution, image noise, and power spectrum). A UNet-base ...[more]

PMID: 31853461

Similar Datasets

Project description:PURPOSE:Magnetic resonance imaging (MRI) provides noninvasive evaluation of patient's anatomy without using ionizing radiation. Due to this and the high soft-tissue contrast, MRI use has increased and has potential for use in longitudinal studies where changes in patients' anatomy or tumors at different time points are compared. Deformable image registration can be useful for these studies. Here, we describe two datasets that can be used to evaluate the registration accuracy of systems for MR images, as it cannot be assumed to be the same as that measured on CT images. ACQUISITION AND VALIDATION METHODS:Two sets of images were created to test registration accuracy. (a) A porcine phantom was created by placing ten 0.35-mm gold markers into porcine meat. The porcine phantom was immobilized in a plastic container with movable dividers. T1-weighted, T2-weighted, and CT images were acquired with the porcine phantom compressed in four different ways. The markers were not visible on the MR images, due to the selected voxel size, so they did not interfere with the measured registration accuracy, while the markers were visible on the CT images and were used to identify the known deformation between positions. (b) Synthetic images were created using 28 head and neck squamous cell carcinoma patients who had MR scans pre-, mid-, and postradiotherapy treatment. An inter- and intrapatient variation model was created using these patient scans. Four synthetic pretreatment images were created using the interpatient model, and four synthetic post-treatment images were created for each of the pretreatment images using the intrapatient model. DATA FORMAT AND USAGE NOTES:The T1-weighted, T2-weighted, and CT scans of the porcine phantom in the four positions are provided. Four T1-weighted synthetic pretreatment images each with four synthetic post-treatment images, and four T2-weighted synthetic pretreatment images each with four synthetic post-treatment images are provided. Additionally, the applied deformation vector fields to generate the synthetic post-treatment images are provided. The data are available on TCIA under the collection MRI-DIR. POTENTIAL APPLICATIONS:The proposed database provides two sets of images (one acquired and one computer generated) for use in evaluating deformable image registration accuracy. T1- and T2-weighted images are available for each technique as the different image contrast in the two types of images may impact the registration accuracy.

Project description:In adaptive radiation therapy of prostate cancer, fast and accurate registration between the planning image and treatment images of the patient is of essential importance. With the authors' recently developed deformable surface model, prostate boundaries in each treatment image can be rapidly segmented and their correspondences (or relative deformations) to the prostate boundaries in the planning image are also established automatically. However, the dense correspondences on the nonboundary regions, which are important especially for transforming the treatment plan designed in the planning image space to each treatment image space, are remained unresolved. This paper presents a novel approach to learn the statistical correlation between deformations of prostate boundary and nonboundary regions, for rapidly estimating deformations of the nonboundary regions when given the deformations of the prostate boundary at a new treatment image.The main contributions of the proposed method lie in the following aspects. First, the statistical deformation correlation will be learned from both current patient and other training patients, and further updated adaptively during the radiotherapy. Specifically, in the initial treatment stage when the number of treatment images collected from the current patient is small, the statistical deformation correlation is mainly learned from other training patients. As more treatment images are collected from the current patient, the patient-specific information will play a more important role in learning patient-specific statistical deformation correlation to effectively reflect prostate deformation of the current patient during the treatment. Eventually, only the patient-specific statistical deformation correlation is used to estimate dense correspondences when a sufficient number of treatment images have been acquired from the current patient. Second, the statistical deformation correlation will be learned by using a multiple linear regression (MLR) model, i.e., ridge regression (RR) model, which has the best prediction accuracy than other MLR models such as canonical correlation analysis (CCA) and principal component regression (PCR).To demonstrate the performance of the proposed method, we first evaluate its registration accuracy by comparing the deformation field predicted by our method with the deformation field estimated by the thin plate spline (TPS) based correspondence interpolation method on 306 serial prostate CT images of 24 patients. The average predictive error on the voxels around 5 mm of prostate boundary is 0.38 mm for our method of RR-based correlation model. Also, the corresponding maximum error is 2.89 mm. We then compare the speed for deformation interpolation by different methods. When considering the larger region of interest (ROI) with the size of 512 × 512 × 61, our method takes 24.41 seconds to interpolate the dense deformation field while TPS method needs 6.7 minutes; when considering a small ROI (surrounding prostate) with size of 112 × 110 × 93, our method takes 1.80 seconds, while TPS method needs 25 seconds.Experimental results show that the proposed method can achieve much faster registration speed yet with comparable registration accuracy, compared to the TPS-based correspondence (or deformation) interpolation approach.

Project description:Aim: To assess the ablative margin (AM) after microwave ablation (MWA) for hepatocellular carcinoma (HCC) with a deep learning-based deformable image registration (DIR) technique and analyze the relation between the AM and local tumor progression (LTP). Patients and Methods: From November 2012 to April 2019, 141 consecutive patients with single HCC (diameter ≤ 5 cm) who underwent MWA were reviewed. Baseline characteristics were collected to identify the risk factors for the determination of LTP after MWA. Contrast-enhanced magnetic resonance imaging scans were performed within 1 month before and 3 months after treatment. Complete ablation was confirmed for all lesions. The AM was measured based on the margin size between the tumor region and the deformed ablative region. To correct the misalignment, DIR between images before and after ablation was achieved by an unsupervised landmark-constrained convolutional neural network. The patients were classified into two groups according to their AMs: group A (AM ≤ 5 mm) and group B (AM > 5 mm). The cumulative LTP rates were compared between the two groups using Kaplan-Meier curves and the log-rank test. Multivariate analyses were performed on clinicopathological variables to identify factors affecting LTP. Results: After a median follow-up period of 28.9 months, LTP was found in 19 patients. The mean tumor and ablation zone sizes were 2.3 ± 0.9 cm and 3.8 ± 1.2 cm, respectively. The mean minimum ablation margin was 3.4 ± 0.7 mm (range, 0-16 mm). The DIR technique had higher AUC for 2-year LTP without a significant difference compared with the registration assessment without DL (P = 0.325). The 6-, 12-, and 24-month LTP rates were 9.9, 20.6, and 24.8%, respectively, in group A, and 4.0, 8.4, and 8.4%, respectively, in group B. There were significant differences between the two groups (P = 0.011). Multivariate analysis showed that being >65 years of age (P = 0.032, hazard ratio (HR): 2.463, 95% confidence interval (CI), 1.028-6.152) and AM ≤ 5 mm (P = 0.010, HR: 3.195, 95% CI, 1.324-7.752) were independent risk factors for LTP after MWA. Conclusion: The novel technology of unsupervised landmark-constrained convolutional neural network-based DIR is feasible and useful in evaluating the ablative effect of MWA for HCC.

Dataset Information

Learning-based deformable image registration: effect of statistical mismatch between train and test images.

Publications

Learning-based deformable image registration: effect of statistical mismatch between train and test images.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets