A Review of Multimodal Medical Image Fusion Techniques.
Ontology highlight
ABSTRACT:
SUBMITTER: Huang B
PROVIDER: S-EPMC7195632 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
A Review of Multimodal Medical Image Fusion Techniques.
Huang Bing B Yang Feng F Yin Mengxiao M Mo Xiaoying X Zhong Cheng C
Computational and mathematical methods in medicine 20200423
The medical image fusion is the process of coalescing multiple images from multiple imaging modalities to obtain a fused image with a large amount of information for increasing the clinical applicability of medical images. In this paper, we attempt to give an overview of multimodal medical image fusion methods, putting emphasis on the most recent advances in the domain based on (1) the current fusion methods, including based on deep learning, (2) imaging modalities of medical image fusion, and ( ...[more]
Similar Datasets
Project description:Alzheimer's disease (AD) is one of the most common neurodegenerative diseases. The formation of amyloid plaques by aggregated amyloid beta (A?) peptides is a primary event in AD pathology. Understanding the metabolomic features and related pathways is critical for studying plaque-related pathological events (e.g., cell death and neuron dysfunction). Mass spectrometry imaging (MSI), due to its high sensitivity and ability to obtain the spatial distribution of metabolites, has been applied to AD studies. However, limited studies of metabolites in amyloid plaques have been performed due to the drawbacks of the commonly used techniques such as matrix-assisted laser desorption/ionization MSI. In the current study, we obtained high spatial resolution (?17 ?m) MS images of the AD mouse brain using the Single-probe, a microscale sampling and ionization device, coupled to a mass spectrometer under ambient conditions. The adjacent slices were used to obtain fluorescence microscopy images to locate amyloid plaques. The MS image and the fluorescence microscopy image were fused to spatially correlate histological protein hallmarks with metabolomic features. The fused images produced significantly improved spatial resolution (?5 ?m), allowing for the determination of fine structures in MS images and metabolomic biomarkers representing amyloid plaques.
| S-EPMC6885010 | biostudies-literature
Project description:PurposeSparsity of annotated data is a major limitation in medical image processing tasks such as registration. Registered multimodal image data are essential for the diagnosis of medical conditions and the success of interventional medical procedures. To overcome the shortage of data, we present a method that allows the generation of annotated multimodal 4D datasets.MethodsWe use a CycleGAN network architecture to generate multimodal synthetic data from the 4D extended cardiac-torso (XCAT) phantom and real patient data. Organ masks are provided by the XCAT phantom; therefore, the generated dataset can serve as ground truth for image segmentation and registration. Realistic simulation of respiration and heartbeat is possible within the XCAT framework. To underline the usability as a registration ground truth, a proof of principle registration is performed.ResultsCompared to real patient data, the synthetic data showed good agreement regarding the image voxel intensity distribution and the noise characteristics. The generated T1-weighted magnetic resonance imaging, computed tomography (CT), and cone beam CT images are inherently co-registered. Thus, the synthetic dataset allowed us to optimize registration parameters of a multimodal non-rigid registration, utilizing liver organ masks for evaluation.ConclusionOur proposed framework provides not only annotated but also multimodal synthetic data which can serve as a ground truth for various tasks in medical imaging processing. We demonstrated the applicability of synthetic data for the development of multimodal medical image registration algorithms.
| S-EPMC8295129 | biostudies-literature
Project description:Medical image fusion techniques can fuse medical images from different morphologies to make the medical diagnosis more reliable and accurate, which play an increasingly important role in many clinical applications. To obtain a fused image with high visual quality and clear structure details, this paper proposes a convolutional neural network (CNN) based medical image fusion algorithm. The proposed algorithm uses the trained Siamese convolutional network to fuse the pixel activity information of source images to realize the generation of weight map. Meanwhile, a contrast pyramid is implemented to decompose the source image. According to different spatial frequency bands and a weighted fusion operator, source images are integrated. The results of comparative experiments show that the proposed fusion algorithm can effectively preserve the detailed structure information of source images and achieve good human visual effects.
| S-EPMC7218740 | biostudies-literature
Project description:Computed tomography (CT) images show structural features, while magnetic resonance imaging (MRI) images represent brain tissue anatomy but do not contain any functional information. How to effectively combine the images of the two modes has become a research challenge. In this paper, a new framework for medical image fusion is proposed which combines convolutional neural networks (CNNs) and non-subsampled shearlet transform (NSST) to simultaneously cover the advantages of them both. This method effectively retains the functional information of the CT image and reduces the loss of brain structure information and spatial distortion of the MRI image. In our fusion framework, the initial weights integrate the pixel activity information from two source images that is generated by a dual-branch convolutional network and is decomposed by NSST. Firstly, the NSST is performed on the source images and the initial weights to obtain their low-frequency and high-frequency coefficients. Then, the first component of the low-frequency coefficients is fused by a novel fusion strategy, which simultaneously copes with two key issues in the fusion processing which are named energy conservation and detail extraction. The second component of the low-frequency coefficients is fused by the strategy that is designed according to the spatial frequency of the weight map. Moreover, the high-frequency coefficients are fused by the high-frequency components of the initial weight. Finally, the final image is reconstructed by the inverse NSST. The effectiveness of the proposed method is verified using pairs of multimodality images, and the sufficient experiments indicate that our method performs well especially for medical image fusion.
| S-EPMC7178475 | biostudies-literature
Project description:PurposeTo allow for comprehensive noninvasive diagnostics of coronary artery disease (CAD) by using three-dimensional (3D) image fusion of CT coronary angiography, CT-derived fractional flow reserve (CT FFR), whole-heart dynamic 3D cardiac MRI perfusion, and 3D cardiac MRI late gadolinium enhancement (LGE).Materials and methodsSeventeen patients (54 years ± 10 [standard deviation], one female) who underwent cardiac CT and cardiac MRI were included (combined subcohort of three prospective trials). Software facilitating multimodal 3D image fusion was developed. Postprocessing of CT data included segmentation of the coronary tree and heart contours, calculation of CT FFR values, and color coding of the coronary tree according to CT FFR. Postprocessing of cardiac MRI data included segmentation of the left ventricle (LV) in cardiac MRI perfusion and cardiac MRI LGE, co-registration of cardiac MRI to CT data, and projection of cardiac MRI perfusion and LGE values onto the high spatial resolution LV from CT.ResultsImage quality was rated as good to excellent (scores: 2.5-2.6; 3 = excellent). CT coronary angiography revealed significant stenoses in seven of 17 cases (41%). CT FFR was possible in 16 of 17 cases (94%) and showed pathologic flow in seven of 17 cases (41%), six of which coincided with cases revealing significant stenoses at CT coronary angiography. Cardiac MRI perfusion identified eight of 17 patients (47%) with hypoperfusion (ischemic burden of 17% ± 5). Cardiac MRI LGE showed myocardial scar in three of 17 cases (18%, scar burden of 7% ± 4). Conventional two-dimensional readout of CT coronary angiography and cardiac MRI resulted in eight of 17 cases (47%) with uncertain findings. Most of these divergent findings could be solved when adding information from CT FFR and 3D image fusion (six of eight, 75%).ConclusionMultimodal 3D cardiac image fusion is feasible and may help with comprehensive noninvasive CAD diagnostics.Supplemental material is available for this article.© RSNA, 2020.
| S-EPMC7977970 | biostudies-literature
Project description:The combination external-beam radiotherapy and high-dose-rate brachytherapy is a standard form of treatment for patients with locally advanced uterine cervical cancer. Personalized radiotherapy in cervical cancer requires efficient and accurate dose planning and assessment across these types of treatment. To achieve radiation dose assessment, accurate mapping of the dose distribution from HDR-BT onto EBRT is extremely important. However, few systems can achieve robust dose fusion and determine the accumulated dose distribution during the entire course of treatment. We have therefore developed a toolbox (FZUImageReg), which is a user-friendly dose fusion system based on hybrid image registration for radiation dose assessment in cervical cancer radiotherapy. The main part of the software consists of a collection of medical image registration algorithms and a modular design with a user-friendly interface, which allows users to quickly configure, test, monitor, and compare different registration methods for a specific application. Owing to the large deformation, the direct application of conventional state-of-the-art image registration methods is not sufficient for the accurate alignment of EBRT and HDR-BT images. To solve this problem, a multi-phase non-rigid registration method using local landmark-based free-form deformation is proposed for locally large deformation between EBRT and HDR-BT images, followed by intensity-based free-form deformation. With the transformation, the software also provides a dose mapping function according to the deformation field. The total dose distribution during the entire course of treatment can then be presented. Experimental results clearly show that the proposed system can achieve accurate registration between EBRT and HDR-BT images and provide radiation dose warping and fusion results for dose assessment in cervical cancer radiotherapy in terms of high accuracy and efficiency.
| S-EPMC5384778 | biostudies-literature
Project description:ObjectiveThe study sought to describe the literature related to the development of methods for auditing the Unified Medical Language System (UMLS), with particular attention to identifying errors and inconsistencies of attributes of the concepts in the UMLS Metathesaurus.Materials and methodsWe applied the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) approach by searching the MEDLINE database and Google Scholar for studies referencing the UMLS and any of several terms related to auditing, error detection, and quality assurance. A qualitative analysis and summarization of articles that met inclusion criteria were performed.ResultsEighty-three studies were reviewed in detail. We first categorized techniques based on various aspects including concepts, concept names, and synonymy (n = 37), semantic type assignments (n = 36), hierarchical relationships (n = 24), lateral relationships (n = 12), ontology enrichment (n = 8), and ontology alignment (n = 18). We also categorized the methods according to their level of automation (ie, automated systematic, automated heuristic, or manual) and the type of knowledge used (ie, intrinsic or extrinsic knowledge).ConclusionsThis study is a comprehensive review of the published methods for auditing the various conceptual aspects of the UMLS. Categorizing the auditing techniques according to the various aspects will enable the curators of the UMLS as well as researchers comprehensive easy access to this wealth of knowledge (eg, for auditing lateral relationships in the UMLS). We also reviewed ontology enrichment and alignment techniques due to their critical use of and impact on the UMLS.
| S-EPMC7566540 | biostudies-literature
Project description:We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large data sets of clinical images contain a wealth of information, time constraints during acquisition result in sparse scans that fail to capture much of the anatomy. These characteristics often render computational analysis impractical as many image analysis algorithms tend to fail when applied to such images. Highly specialized algorithms that explicitly handle sparse slice spacing do not generalize well across problem domains. In contrast, we aim to enable application of existing algorithms that were originally developed for high resolution research scans to significantly undersampled scans. We introduce a generative model that captures fine-scale anatomical structure across subjects in clinical image collections and derive an algorithm for filling in the missing data in scans with large inter-slice spacing. Our experimental results demonstrate that the resulting method outperforms state-of-the-art upsampling super-resolution techniques, and promises to facilitate subsequent analysis not previously possible with scans of this quality. Our implementation is freely available at https://github.com/adalca/papago.
| S-EPMC6393212 | biostudies-literature
Project description:Posterior fusion is a powerful tool to address pathology of the cervical spine, but the decision to fuse at any level should be made with great care. Various methods and constructs for posterior cervical fusion exist, all of which aim to restore the posterior tension band's ability to resist flexion forces. We identified articles regarding posterior fusion of the subaxial cervical spine in MEDLINE, Google Scholar, and PubMed. This article is a narrative review of earlier and current concepts regarding the posterior fusion of the subaxial cervical spine, including wiring, translaminar screws, lateral mass screws, and pedicle screws, weighing the strengths and weakness of the different modalities that the surgeon should bear in mind in creating operative plans individualized to patient pathology.
| S-EPMC7253542 | biostudies-literature
Project description:Medical image registration is a vital component of many medical procedures, such as image-guided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research. The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.
| S-EPMC8611468 | biostudies-literature