Distance-Dependent Multimodal Image Registration for Agriculture Tasks.
Ontology highlight
ABSTRACT:
SUBMITTER: Berenstein R
PROVIDER: S-EPMC4570450 | biostudies-literature | 2015
REPOSITORIES: biostudies-literature
Publications
Distance-Dependent Multimodal Image Registration for Agriculture Tasks.
Berenstein Ron R Hočevar Marko M Godeša Tone T Edan Yael Y Ben-Shahar Ohad O
Sensors (Basel, Switzerland) 20150821 8
Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibra ...[more]
Similar Datasets
Project description:Despite current advancement in the field of biomedical image processing, propelled by the deep learning revolution, multimodal image registration, due to its several challenges, is still often performed manually by specialists. The recent success of image-to-image (I2I) translation in computer vision applications and its growing use in biomedical areas provide a tempting possibility of transforming the multimodal registration problem into a, potentially easier, monomodal one. We conduct an empirical study of the applicability of modern I2I translation methods for the task of rigid registration of multimodal biomedical and medical 2D and 3D images. We compare the performance of four Generative Adversarial Network (GAN)-based I2I translation methods and one contrastive representation learning method, subsequently combined with two representative monomodal registration methods, to judge the effectiveness of modality translation for multimodal image registration. We evaluate these method combinations on four publicly available multimodal (2D and 3D) datasets and compare with the performance of registration achieved by several well-known approaches acting directly on multimodal image data. Our results suggest that, although I2I translation may be helpful when the modalities to register are clearly correlated, registration of modalities which express distinctly different properties of the sample are not well handled by the I2I translation approach. The evaluated representation learning method, which aims to find abstract image-like representations of the information shared between the modalities, manages better, and so does the Mutual Information maximisation approach, acting directly on the original multimodal images. We share our complete experimental setup as open-source (https://github.com/MIDA-group/MultiRegEval), including method implementations, evaluation code, and all datasets, for further reproducing and benchmarking.
| S-EPMC9704666 | biostudies-literature
Project description:With the introduction of multi-camera systems in modern plant phenotyping new opportunities for combined multimodal image analysis emerge. Visible light (VIS), fluorescence (FLU) and near-infrared images enable scientists to study different plant traits based on optical appearance, biochemical composition and nutrition status. A straightforward analysis of high-throughput image data is hampered by a number of natural and technical factors including large variability of plant appearance, inhomogeneous illumination, shadows and reflections in the background regions. Consequently, automated segmentation of plant images represents a big challenge and often requires an extensive human-machine interaction. Combined analysis of different image modalities may enable automatisation of plant segmentation in "difficult" image modalities such as VIS images by utilising the results of segmentation of image modalities that exhibit higher contrast between plant and background, i.e. FLU images. For efficient segmentation and detection of diverse plant structures (i.e. leaf tips, flowers), image registration techniques based on feature point (FP) matching are of particular interest. However, finding reliable feature points and point pairs for differently structured plant species in multimodal images can be challenging. To address this task in a general manner, different feature point detectors should be considered. Here, a comparison of seven different feature point detectors for automated registration of VIS and FLU plant images is performed. Our experimental results show that straightforward image registration using FP detectors is prone to errors due to too large structural difference between FLU and VIS modalities. We show that structural image enhancement such as background filtering and edge image transformation significantly improves performance of FP algorithms. To overcome the limitations of single FP detectors, combination of different FP methods is suggested. We demonstrate application of our enhanced FP approach for automated registration of a large amount of FLU/VIS images of developing plant species acquired from high-throughput phenotyping experiments.
| S-EPMC6768447 | biostudies-literature
Project description:PurposeSparsity of annotated data is a major limitation in medical image processing tasks such as registration. Registered multimodal image data are essential for the diagnosis of medical conditions and the success of interventional medical procedures. To overcome the shortage of data, we present a method that allows the generation of annotated multimodal 4D datasets.MethodsWe use a CycleGAN network architecture to generate multimodal synthetic data from the 4D extended cardiac-torso (XCAT) phantom and real patient data. Organ masks are provided by the XCAT phantom; therefore, the generated dataset can serve as ground truth for image segmentation and registration. Realistic simulation of respiration and heartbeat is possible within the XCAT framework. To underline the usability as a registration ground truth, a proof of principle registration is performed.ResultsCompared to real patient data, the synthetic data showed good agreement regarding the image voxel intensity distribution and the noise characteristics. The generated T1-weighted magnetic resonance imaging, computed tomography (CT), and cone beam CT images are inherently co-registered. Thus, the synthetic dataset allowed us to optimize registration parameters of a multimodal non-rigid registration, utilizing liver organ masks for evaluation.ConclusionOur proposed framework provides not only annotated but also multimodal synthetic data which can serve as a ground truth for various tasks in medical imaging processing. We demonstrated the applicability of synthetic data for the development of multimodal medical image registration algorithms.
| S-EPMC8295129 | biostudies-literature
Project description:3D histology, slice-based connectivity atlases, and diffusion MRI are common techniques to map brain wiring. While there are many modality-specific tools to process these data, there is a lack of integration across modalities. We develop an automated resource that combines histologically cleared volumes with connectivity atlases and MRI, enabling the analysis of histological features across multiple fiber tracts and networks, and their correlation with in-vivo biomarkers. We apply our pipeline in a murine stroke model, demonstrating not only strong correspondence between MRI abnormalities and CLARITY-tissue staining, but also uncovering acute cellular effects in areas connected to the ischemic core. We provide improved maps of connectivity by quantifying projection terminals from CLARITY viral injections, and integrate diffusion MRI with CLARITY viral tracing to compare connectivity maps across scales. Finally, we demonstrate tract-level histological changes of stroke through this multimodal integration. This resource can propel investigations of network alterations underlying neurological disorders.
| S-EPMC6890789 | biostudies-literature
Project description:PurposeThe integration of CT and multiparametric MRI (mpMRI) is a challenging task in high-precision radiotherapy for prostate cancer. A simple methodology for multimodal deformable image registration (DIR) of prostate cancer patients is presented.MethodsCT and mpMRI of 10 patients were considered. Organs at risk and prostate were contoured on both scans. The dominant intraprostatic lesion was additionally delineated on MRI. After a preliminary rigid image registration, the voxel intensity of all the segmented structures in both scans except the prostate was increased by a specific amount (a constant additional value, A), in order to enhance the contrast of the main organs influencing its position and shape. 70 couples of scans were obtained by varying A from 0 to 800 and they were subsequently non-rigidly registered. Quantities derived from image analysis and contour statistics were considered for the tuning of the best performing A.ResultsA = 200 resulted the minimum enhancement value required to obtain statistically significant superior registration results. Mean centre of mass distance between corresponding structures decreases from 7.4 mm in rigid registration to 5.3 mm in DIR without enhancement (DIR-0) and to 2.7 mm in DIR with A = 200 (DIR-200). Mean contour distance was 2.5, 1.9 and 0.67 mm in rigid registration, DIR-0 and DIR-200, respectively. In DIR-200 mean contours overlap increases of +13 and +24% with respect to DIR-0 and rigid registration, respectively.ConclusionContour propagation according to the vector field resulting from DIR-200 allows the delineation of dominant intraprostatic lesion on CT scan and its use for high-precision radiotherapy treatment planning. Advances in knowledge: We investigated the application of a B-spline, mutual information-based multimodal DIR coupled with a simple, patient-unspecific but efficient contrast enhancement procedure in the pelvic body area, thus obtaining a robust and accurate methodology to transfer the functional information deriving from mpMRI onto a planning CT reference volume.
| S-EPMC5963367 | biostudies-literature
Project description:This study aims at developing and testing a novel rigidity penalty suitable for the deformable registration of tightly located skeletal components in the head and neck from planning computed tomography (CT) and daily cone-beam CT (CBCT) scans of patients undergoing radiotherapy.The proposed rigidity penalty is designed to preserve intervoxel distances within each bony structure. This penalty was tested in the intensity-based B-spline deformable registration of five cervical vertebral bodies (C1-C5). The displacement vector fields (DVFs) from the registrations were compared to the DVFs generated by using rigid body motions of the cervical vertebrae, measured by the surface registration of vertebrae delineated on CT and CBCT images. Twenty five pairs of planning CT (reference) and treatment CBCTs (target) from five patients were aligned without and with the penalty. An existing penalty based on the orthonormality of the deformation gradient tensor was also tested and the effects of the penalties compared.The mean magnitude of the maximum registration error with the proposed distance-preserving penalty was (0.86, 1.12, 1.33) mm compared to (2.11, 2.49, 2.46) without penalty and (1.53, 1.64, 1.64) with the existing orthonormality-based penalty. The improvement in the accuracy of the deformable image registration was also verified by comparing the Procrustes distance between the DVFs. With the proposed penalty, the average distance was 0.11 (σ 0.03 mm) which is smaller than 0.53 (0.1 mm) without penalty and 0.28 (0.04 mm) with the orthonormality-based penalty.The accuracy of aligning multiple bony elements was improved by using the proposed distance-preserving rigidity penalty. The voxel-based statistical analysis of the registration error shows that the proposed penalty improved the integrity of the DVFs within the vertebral bodies.
| S-EPMC3843756 | biostudies-other
Project description:Image registration is a key technique in medical image analysis to estimate deformations between image pairs. A good deformation model is important for high-quality estimates. However, most existing approaches use ad-hoc deformation models chosen for mathematical convenience rather than to capture observed data variation. Recent deep learning approaches learn deformation models directly from data. However, they provide limited control over the spatial regularity of transformations. Instead of learning the entire registration approach, we learn a spatially-adaptive regularizer within a registration model. This allows controlling the desired level of regularity and preserving structural properties of a registration model. For example, diffeomorphic transformations can be attained. Our approach is a radical departure from existing deep learning approaches to image registration by embedding a deep learning model in an optimization-based registration algorithm to parameterize and data-adapt the registration model itself. Source code is publicly-available at https://github.com/uncbiag/registration.
| S-EPMC7286567 | biostudies-literature
Project description:Over 200 georeferenced registered rephotographic compilations of the Faroe Islands are provided in this dataset. The position of each compilation is georeferenced and thus locatable on a map. Each compilation consists of a historical and a corresponding contemporary image showing the same scene. With steady object features, these two images of the same geolocation are aligned pixel accurately. In the summer of 2022, all contemporary images were photographed by A. Schaffland, while historical images were retrieved from the National Museum of Denmark collections. Images show Faroese landscape and cultural heritage sites, focusing on relevant areas when the historical images were taken, e.g., Kirkjubøur, Tórshavn, and Saksun. Historic images date from the end of the 19th century to the middle of the 20th century. The historical images were taken by scientists, surveyors, archaeologists, and painters. All historical images are in the public domain, have no known rights, or are shared under a CC license. The contemporary images by A. Schaffland are released under CC BY-NC-SA 4.0. The dataset is organized as a GIS project. Historic images, not already georeferenced, were referenced with street view services. All historical images were added to the GIS database, containing camera position, viewing direction, etc. Each compilation can be displayed as an arrow from the camera position along the view direction on a map. Contemporary images were registered to historical images using a specialized tool. None or only a suboptimal rephotograph could be taken for some historical images. These historical images are still added to the database together with all other original images, providing additional data for improvements in rephotography methods in the upcoming years. The resulting image pairs can be used in image registration, landscape change, urban development, and cultural heritage research. Further, the database can be used for public engagement in heritage and as a benchmark for further rephotography and time-series projects.
| S-EPMC9969268 | biostudies-literature
Project description:BackgroundImage registration is one of the most important and universally useful computational tasks in biomedical image analysis. A recent article by Xing & Qiu (IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(10):2081-2092, 2011) is based on an inappropriately narrow conceptualization of the image registration problem as the task of making two images look alike, which disregards whether the established spatial correspondence is plausible. The authors propose a new algorithm, Nonparametric Local Smoothing (NLS) for image registration, but use image similarities alone as a measure of registration performance, although these measures do not relate reliably to the realism of the correspondence map.ResultsUsing data obtained from its authors, we show experimentally that the method proposed by Xing & Qiu is not an effective registration algorithm. While it optimizes image similarity, it does not compute accurate, interpretable transformations. Even judged by image similarity alone, the proposed method is consistently outperformed by a simple pixel permutation algorithm, which is known by design not to compute valid registrations.ConclusionsThis study has demonstrated that the NLS algorithm proposed recently for image registration, and published in one of the most respected journals in computer science, is not, in fact, an effective registration method at all. Our results also emphasize the general need to apply registration evaluation criteria that are sensitive to whether correspondences are accurate and mappings between images are physically interpretable. These goals cannot be achieved by simply reporting image similarities.
| S-EPMC3740790 | biostudies-literature
Project description:In this paper, a novel image registration method is proposed to achieve accurate registration between images having large shape differences with the help of a set of appropriate intermediate templates. We first demonstrate that directionality is a key factor in both pairwise image registration and groupwise registration, which is defined in this paper to describe the influence of the registration direction and paths on the registration performance. In our solution, the intermediate template selection and intermediate template guided registration are two coherent steps with directionality being considered. To take advantage of the directionality, a directed graph is built based on the asymmetric distance defined on all ordered image pairs in the image population, which is fundamentally different from the undirected graph with symmetric distance metrics in all previous methods, and the shortest distance between template and subject on the directed graph is calculated. The allocated directed path can be thus utilized to better guide the registration by successively registering the subject through the intermediate templates one by one on the path towards the template. The proposed directed graph based solution can also be used in groupwise registration. Specifically, by building a minimum spanning arborescence (MSA) on the directed graph, the population center, i.e., a selected template, as well as the directed registration paths from all the rest of images to the population center, is determined simultaneously. The performance of directed graph based registration algorithm is demonstrated by the spatial normalization on both synthetic dataset and real brain MR images. It is shown that our method can achieve more accurate registration results than both the undirected graph based solution and the direct pairwise registration.
| S-EPMC3272327 | biostudies-other