A Review of Multimodal Medical Image Fusion Techniques.
Ontology highlight
ABSTRACT:
SUBMITTER: Huang B
PROVIDER: S-EPMC7195632 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
A Review of Multimodal Medical Image Fusion Techniques.
Huang Bing B Yang Feng F Yin Mengxiao M Mo Xiaoying X Zhong Cheng C
Computational and mathematical methods in medicine 20200423
The medical image fusion is the process of coalescing multiple images from multiple imaging modalities to obtain a fused image with a large amount of information for increasing the clinical applicability of medical images. In this paper, we attempt to give an overview of multimodal medical image fusion methods, putting emphasis on the most recent advances in the domain based on (1) the current fusion methods, including based on deep learning, (2) imaging modalities of medical image fusion, and ( ...[more]
Similar Datasets
Project description:Background contextFluoroscopic devices can be used to visualize subcutaneous and osseous tissue, a useful feature during pedicle screw insertion in lumbar fusion surgery. It is important that both patient and surgeon are exposed as little as possible, since these devices use potential harmful ionizing radiation.PurposeThis study aims to compare radiation exposure of different image-guided techniques in lumbar fusion surgery with pedicle screw insertion.Study designSystematic review.MethodsCochrane, Embase, PubMed and Web of Science databases were used to acquire relevant studies. Eligibility criteria were lumbar and/or sacral spine, pedicle screw, mGray and/or Sievert and/or mrem, radiation dose and/or radiation exposure. Image-guided techniques were divided in five groups: conventional C-arm, C-arm navigation, C-arm robotic, O-arm navigation and O-arm robotic. Comparisons were made based on effective dose for patients and surgeons, absorbed dose for patients and surgeons and exposure. Risk of bias was assessed using the 2017 Cochrane Risk of Bias tool on RCTs and the Cochrane ROBINS-I tool on NRCTs. Level of evidence was assessed using the guidelines of Oxford Centre for Evidence-based Medicine 2011.ResultsA total of 1423 studies were identified of which 38 were included in the analysis and assigned to one of the five groups. Results of radiation dose per procedure and per pedicle screw were described in dose ranges. Conventional C-arm appeared to result in higher effective dose for surgeons, higher absorbed dose for patients and higher exposure, compared to C-arm navigation/robotic and O-arm navigation/robotic. Level of evidence was 3 to 4 in 29 studies. Risk of bias of RCTs was intermediate, mostly due to inadequate blinding. Overall risk of bias score in NRCTs was determined as 'serious'.ConclusionsRanges of radiation doses using different modalities during pedicle screw insertion in lumbar fusion surgery are wide. Based on the highest numbers in the ranges, conventional C-arm tends to lead to a higher effective dose for surgeons, higher absorbed dose for patients and higher exposure, compared to C-arm-, and O-arm navigation/robotic. The level of evidence is low and risk of bias is fairly high. In future studies, heterogeneity should be limited by standardizing measurement methods and thoroughly describing the image-guided technique settings.
| S-EPMC9898805 | biostudies-literature
Project description:Alzheimer's disease (AD) is one of the most common neurodegenerative diseases. The formation of amyloid plaques by aggregated amyloid beta (A?) peptides is a primary event in AD pathology. Understanding the metabolomic features and related pathways is critical for studying plaque-related pathological events (e.g., cell death and neuron dysfunction). Mass spectrometry imaging (MSI), due to its high sensitivity and ability to obtain the spatial distribution of metabolites, has been applied to AD studies. However, limited studies of metabolites in amyloid plaques have been performed due to the drawbacks of the commonly used techniques such as matrix-assisted laser desorption/ionization MSI. In the current study, we obtained high spatial resolution (?17 ?m) MS images of the AD mouse brain using the Single-probe, a microscale sampling and ionization device, coupled to a mass spectrometer under ambient conditions. The adjacent slices were used to obtain fluorescence microscopy images to locate amyloid plaques. The MS image and the fluorescence microscopy image were fused to spatially correlate histological protein hallmarks with metabolomic features. The fused images produced significantly improved spatial resolution (?5 ?m), allowing for the determination of fine structures in MS images and metabolomic biomarkers representing amyloid plaques.
| S-EPMC6885010 | biostudies-literature
Project description:PurposeSparsity of annotated data is a major limitation in medical image processing tasks such as registration. Registered multimodal image data are essential for the diagnosis of medical conditions and the success of interventional medical procedures. To overcome the shortage of data, we present a method that allows the generation of annotated multimodal 4D datasets.MethodsWe use a CycleGAN network architecture to generate multimodal synthetic data from the 4D extended cardiac-torso (XCAT) phantom and real patient data. Organ masks are provided by the XCAT phantom; therefore, the generated dataset can serve as ground truth for image segmentation and registration. Realistic simulation of respiration and heartbeat is possible within the XCAT framework. To underline the usability as a registration ground truth, a proof of principle registration is performed.ResultsCompared to real patient data, the synthetic data showed good agreement regarding the image voxel intensity distribution and the noise characteristics. The generated T1-weighted magnetic resonance imaging, computed tomography (CT), and cone beam CT images are inherently co-registered. Thus, the synthetic dataset allowed us to optimize registration parameters of a multimodal non-rigid registration, utilizing liver organ masks for evaluation.ConclusionOur proposed framework provides not only annotated but also multimodal synthetic data which can serve as a ground truth for various tasks in medical imaging processing. We demonstrated the applicability of synthetic data for the development of multimodal medical image registration algorithms.
| S-EPMC8295129 | biostudies-literature
Project description:Medical image fusion techniques can fuse medical images from different morphologies to make the medical diagnosis more reliable and accurate, which play an increasingly important role in many clinical applications. To obtain a fused image with high visual quality and clear structure details, this paper proposes a convolutional neural network (CNN) based medical image fusion algorithm. The proposed algorithm uses the trained Siamese convolutional network to fuse the pixel activity information of source images to realize the generation of weight map. Meanwhile, a contrast pyramid is implemented to decompose the source image. According to different spatial frequency bands and a weighted fusion operator, source images are integrated. The results of comparative experiments show that the proposed fusion algorithm can effectively preserve the detailed structure information of source images and achieve good human visual effects.
| S-EPMC7218740 | biostudies-literature
Project description:Computed tomography (CT) images show structural features, while magnetic resonance imaging (MRI) images represent brain tissue anatomy but do not contain any functional information. How to effectively combine the images of the two modes has become a research challenge. In this paper, a new framework for medical image fusion is proposed which combines convolutional neural networks (CNNs) and non-subsampled shearlet transform (NSST) to simultaneously cover the advantages of them both. This method effectively retains the functional information of the CT image and reduces the loss of brain structure information and spatial distortion of the MRI image. In our fusion framework, the initial weights integrate the pixel activity information from two source images that is generated by a dual-branch convolutional network and is decomposed by NSST. Firstly, the NSST is performed on the source images and the initial weights to obtain their low-frequency and high-frequency coefficients. Then, the first component of the low-frequency coefficients is fused by a novel fusion strategy, which simultaneously copes with two key issues in the fusion processing which are named energy conservation and detail extraction. The second component of the low-frequency coefficients is fused by the strategy that is designed according to the spatial frequency of the weight map. Moreover, the high-frequency coefficients are fused by the high-frequency components of the initial weight. Finally, the final image is reconstructed by the inverse NSST. The effectiveness of the proposed method is verified using pairs of multimodality images, and the sufficient experiments indicate that our method performs well especially for medical image fusion.
| S-EPMC7178475 | biostudies-literature
Project description:PurposeTo allow for comprehensive noninvasive diagnostics of coronary artery disease (CAD) by using three-dimensional (3D) image fusion of CT coronary angiography, CT-derived fractional flow reserve (CT FFR), whole-heart dynamic 3D cardiac MRI perfusion, and 3D cardiac MRI late gadolinium enhancement (LGE).Materials and methodsSeventeen patients (54 years ± 10 [standard deviation], one female) who underwent cardiac CT and cardiac MRI were included (combined subcohort of three prospective trials). Software facilitating multimodal 3D image fusion was developed. Postprocessing of CT data included segmentation of the coronary tree and heart contours, calculation of CT FFR values, and color coding of the coronary tree according to CT FFR. Postprocessing of cardiac MRI data included segmentation of the left ventricle (LV) in cardiac MRI perfusion and cardiac MRI LGE, co-registration of cardiac MRI to CT data, and projection of cardiac MRI perfusion and LGE values onto the high spatial resolution LV from CT.ResultsImage quality was rated as good to excellent (scores: 2.5-2.6; 3 = excellent). CT coronary angiography revealed significant stenoses in seven of 17 cases (41%). CT FFR was possible in 16 of 17 cases (94%) and showed pathologic flow in seven of 17 cases (41%), six of which coincided with cases revealing significant stenoses at CT coronary angiography. Cardiac MRI perfusion identified eight of 17 patients (47%) with hypoperfusion (ischemic burden of 17% ± 5). Cardiac MRI LGE showed myocardial scar in three of 17 cases (18%, scar burden of 7% ± 4). Conventional two-dimensional readout of CT coronary angiography and cardiac MRI resulted in eight of 17 cases (47%) with uncertain findings. Most of these divergent findings could be solved when adding information from CT FFR and 3D image fusion (six of eight, 75%).ConclusionMultimodal 3D cardiac image fusion is feasible and may help with comprehensive noninvasive CAD diagnostics.Supplemental material is available for this article.© RSNA, 2020.
| S-EPMC7977970 | biostudies-literature
Project description:BackgroundTransfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task.Methods425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch.ResultsThe majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models.ConclusionThe investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.
| S-EPMC9007400 | biostudies-literature
Project description:The combination external-beam radiotherapy and high-dose-rate brachytherapy is a standard form of treatment for patients with locally advanced uterine cervical cancer. Personalized radiotherapy in cervical cancer requires efficient and accurate dose planning and assessment across these types of treatment. To achieve radiation dose assessment, accurate mapping of the dose distribution from HDR-BT onto EBRT is extremely important. However, few systems can achieve robust dose fusion and determine the accumulated dose distribution during the entire course of treatment. We have therefore developed a toolbox (FZUImageReg), which is a user-friendly dose fusion system based on hybrid image registration for radiation dose assessment in cervical cancer radiotherapy. The main part of the software consists of a collection of medical image registration algorithms and a modular design with a user-friendly interface, which allows users to quickly configure, test, monitor, and compare different registration methods for a specific application. Owing to the large deformation, the direct application of conventional state-of-the-art image registration methods is not sufficient for the accurate alignment of EBRT and HDR-BT images. To solve this problem, a multi-phase non-rigid registration method using local landmark-based free-form deformation is proposed for locally large deformation between EBRT and HDR-BT images, followed by intensity-based free-form deformation. With the transformation, the software also provides a dose mapping function according to the deformation field. The total dose distribution during the entire course of treatment can then be presented. Experimental results clearly show that the proposed system can achieve accurate registration between EBRT and HDR-BT images and provide radiation dose warping and fusion results for dose assessment in cervical cancer radiotherapy in terms of high accuracy and efficiency.
| S-EPMC5384778 | biostudies-literature
Project description:ObjectiveThe study sought to describe the literature related to the development of methods for auditing the Unified Medical Language System (UMLS), with particular attention to identifying errors and inconsistencies of attributes of the concepts in the UMLS Metathesaurus.Materials and methodsWe applied the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) approach by searching the MEDLINE database and Google Scholar for studies referencing the UMLS and any of several terms related to auditing, error detection, and quality assurance. A qualitative analysis and summarization of articles that met inclusion criteria were performed.ResultsEighty-three studies were reviewed in detail. We first categorized techniques based on various aspects including concepts, concept names, and synonymy (n = 37), semantic type assignments (n = 36), hierarchical relationships (n = 24), lateral relationships (n = 12), ontology enrichment (n = 8), and ontology alignment (n = 18). We also categorized the methods according to their level of automation (ie, automated systematic, automated heuristic, or manual) and the type of knowledge used (ie, intrinsic or extrinsic knowledge).ConclusionsThis study is a comprehensive review of the published methods for auditing the various conceptual aspects of the UMLS. Categorizing the auditing techniques according to the various aspects will enable the curators of the UMLS as well as researchers comprehensive easy access to this wealth of knowledge (eg, for auditing lateral relationships in the UMLS). We also reviewed ontology enrichment and alignment techniques due to their critical use of and impact on the UMLS.
| S-EPMC7566540 | biostudies-literature
Project description:We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large data sets of clinical images contain a wealth of information, time constraints during acquisition result in sparse scans that fail to capture much of the anatomy. These characteristics often render computational analysis impractical as many image analysis algorithms tend to fail when applied to such images. Highly specialized algorithms that explicitly handle sparse slice spacing do not generalize well across problem domains. In contrast, we aim to enable application of existing algorithms that were originally developed for high resolution research scans to significantly undersampled scans. We introduce a generative model that captures fine-scale anatomical structure across subjects in clinical image collections and derive an algorithm for filling in the missing data in scans with large inter-slice spacing. Our experimental results demonstrate that the resulting method outperforms state-of-the-art upsampling super-resolution techniques, and promises to facilitate subsequent analysis not previously possible with scans of this quality. Our implementation is freely available at https://github.com/adalca/papago.
| S-EPMC6393212 | biostudies-literature