Browse
Submit Data
Databases
API
Help

Dataset Information

32 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Multi-Modality Medical Image Fusion Using Convolutional Neural Network and Contrast Pyramid.

ABSTRACT: Medical image fusion techniques can fuse medical images from different morphologies to make the medical diagnosis more reliable and accurate, which play an increasingly important role in many clinical applications. To obtain a fused image with high visual quality and clear structure details, this paper proposes a convolutional neural network (CNN) based medical image fusion algorithm. The proposed algorithm uses the trained Siamese convolutional network to fuse the pixel activity information of source images to realize the generation of weight map. Meanwhile, a contrast pyramid is implemented to decompose the source image. According to different spatial frequency bands and a weighted fusion operator, source images are integrated. The results of comparative experiments show that the proposed fusion algorithm can effectively preserve the detailed structure information of source images and achieve good human visual effects.

SUBMITTER: Wang K

PROVIDER: S-EPMC7218740 | biostudies-literature | 2020 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Multi-Modality Medical Image Fusion Using Convolutional Neural Network and Contrast Pyramid.

Wang Kunpeng K Zheng Mingyao M Wei Hongyan H Qi Guanqiu G Li Yuanyuan Y

Sensors (Basel, Switzerland) 20200411 8

Medical image fusion techniques can fuse medical images from different morphologies to make the medical diagnosis more reliable and accurate, which play an increasingly important role in many clinical applications. To obtain a fused image with high visual quality and clear structure details, this paper proposes a convolutional neural network (CNN) based medical image fusion algorithm. The proposed algorithm uses the trained Siamese convolutional network to fuse the pixel activity information of ...[more]

PMID: 32290472

Similar Datasets

Automatic classification of medical image modality and anatomical location using convolutional neural network.

Project description:Modern radiologic images comply with DICOM (digital imaging and communications in medicine) standard, which, upon conversion to other image format, would lose its image detail and information such as patient demographics or type of image modality that DICOM format carries. As there is a growing interest in using large amount of image data for research purpose and acquisition of large amount of medical image is now a standard practice in the clinical setting, efficient handling and storage of large amount of image data is important in both the clinical and research setting. In this study, four classes of images were created, namely, CT (computed tomography) of abdomen, CT of brain, MRI (magnetic resonance imaging) of brain and MRI of spine. After converting these images into JPEG (Joint Photographic Experts Group) format, our proposed CNN architecture could automatically classify these 4 groups of medical images by both their image modality and anatomic location. We achieved excellent overall classification accuracy in both validation and test sets (> 99.5%), specificity and F1 score (> 99%) in each category of this dataset which contained both diseased and normal images. Our study has shown that using CNN for medical image classification is a promising methodology and could work on non-DICOM images, which could potentially save image processing time and storage space.

| S-EPMC8195382 | biostudies-literature

Diagnosis of Alzheimer's Disease via Multi-Modality 3D Convolutional Neural Network.

Project description:Alzheimer's disease (AD) is one of the most common neurodegenerative diseases. In the last decade, studies on AD diagnosis has attached great significance to artificial intelligence-based diagnostic algorithms. Among the diverse modalities of imaging data, T1-weighted MR and FDG-PET are widely used for this task. In this paper, we propose a convolutional neural network (CNN) to integrate all the multi-modality information included in both T1-MR and FDG-PET images of the hippocampal area, for the diagnosis of AD. Different from the traditional machine learning algorithms, this method does not require manually extracted features, instead, it utilizes 3D image-processing CNNs to learn features for the diagnosis or prognosis of AD. To test the performance of the proposed network, we trained the classifier with paired T1-MR and FDG-PET images in the ADNI datasets, including 731 cognitively unimpaired (labeled as CN) subjects, 647 subjects with AD, 441 subjects with stable mild cognitive impairment (sMCI) and 326 subjects with progressive mild cognitive impairment (pMCI). We obtained higher accuracies of 90.10% for CN vs. AD task, 87.46% for CN vs. pMCI task, and 76.90% for sMCI vs. pMCI task. The proposed framework yields a state-of-the-art performance. Finally, the results have demonstrated that (1) segmentation is not a prerequisite when using a CNN for the classification, (2) the combination of two modality imaging data generates better results.

| S-EPMC6555226 | biostudies-literature

Visual Interpretation of Convolutional Neural Network Predictions in Classifying Medical Image Modalities.

Project description:Deep learning (DL) methods are increasingly being applied for developing reliable computer-aided detection (CADe), diagnosis (CADx), and information retrieval algorithms. However, challenges in interpreting and explaining the learned behavior of the DL models hinders their adoption and use in real-world systems. In this study, we propose a novel method called "Class-selective Relevance Mapping" (CRM) for localizing and visualizing discriminative regions of interest (ROI) within a medical image. Such visualizations offer improved explanation of the convolutional neural network (CNN)-based DL model predictions. We demonstrate CRM effectiveness in classifying medical imaging modalities toward automatically labeling them for visual information retrieval applications. The CRM is based on linear sum of incremental mean squared errors (MSE) calculated at the output layer of the CNN model. It measures both positive and negative contributions of each spatial element in the feature maps produced from the last convolution layer leading to correct classification of an input image. A series of experiments on a "multi-modality" CNN model designed for classifying seven different types of image modalities shows that the proposed method is significantly better in detecting and localizing the discriminative ROIs than other state of the art class-activation methods. Further, to visualize its effectiveness we generate "class-specific" ROI maps by averaging the CRM scores of images in each modality class, and characterize the visual explanation through their different size, shape, and location for our multi-modality CNN model that achieved over 98% performance on a dataset constructed from publicly available images.

| S-EPMC6627892 | biostudies-literature

Thyroid ultrasound image classification using a convolutional neural network.

Project description:BackgroundUltrasound (US) is widely used in the clinical diagnosis of thyroid nodules. Artificial intelligence-powered US is becoming an important issue in the research community. This study aimed to develop an improved deep learning model-based algorithm to classify benign and malignant thyroid nodules (TNs) using thyroid US images.MethodsIn total, 592 patients with 600 TNs were included in the internal training, validation, and testing data set; 187 patients with 200 TNs were recruited for the external test data set. We developed a Visual Geometry Group (VGG)-16T model, based on the VGG-16 architecture, but with additional batch normalization (BN) and dropout layers in addition to the fully connected layers. We conducted a 10-fold cross-validation to analyze the performance of the VGG-16T model using a data set of gray-scale US images from 5 different brands of US machines.ResultsFor the internal data set, the VGG-16T model had 87.43% sensitivity, 85.43% specificity, and 86.43% accuracy. For the external data set, the VGG-16T model achieved an area under the curve (AUC) of 0.829 [95% confidence interval (CI): 0.770-0.879], a radiologist with 15 years' working experience achieved an AUC of 0.705 (95% CI: 0.659-0.801), a radiologist with 10 years' experience achieved an AUC of 0.725 (95% CI: 0.653-0.797), and a radiologist with 5 years' experience achieved an AUC of 0.660 (95% CI: 0.584-0.736).ConclusionsThe VGG-16T model had high specificity, sensitivity, and accuracy in differentiating between malignant and benign TNs. Its diagnostic performance was superior to that of experienced radiologists. Thus, the proposed improved deep-learning model can assist radiologists to diagnose thyroid cancer.

| S-EPMC8576712 | biostudies-literature

Identification of Potential Parkinson's Disease Drugs Based on Multi-Source Data Fusion and Convolutional Neural Network.

Project description:Parkinson's disease (PD) is a serious neurodegenerative disease. Most of the current treatment can only alleviate symptoms, but not stop the progress of the disease. Therefore, it is crucial to find medicines to completely cure PD. Finding new indications of existing drugs through drug repositioning can not only reduce risk and cost, but also improve research and development efficiently. A drug repurposing method was proposed to identify potential Parkinson's disease-related drugs based on multi-source data integration and convolutional neural network. Multi-source data were used to construct similarity networks, and topology information were utilized to characterize drugs and PD-associated proteins. Then, diffusion component analysis method was employed to reduce the feature dimension. Finally, a convolutional neural network model was constructed to identify potential associations between existing drugs and LProts (PD-associated proteins). Based on 10-fold cross-validation, the developed method achieved an accuracy of 91.57%, specificity of 87.24%, sensitivity of 95.27%, Matthews correlation coefficient of 0.8304, area under the receiver operating characteristic curve of 0.9731 and area under the precision-recall curve of 0.9727, respectively. Compared with the state-of-the-art approaches, the current method demonstrates superiority in some aspects, such as sensitivity, accuracy, robustness, etc. In addition, some of the predicted potential PD therapeutics through molecular docking further proved that they can exert their efficacy by acting on the known targets of PD, and may be potential PD therapeutic drugs for further experimental research. It is anticipated that the current method may be considered as a powerful tool for drug repurposing and pathological mechanism studies.

| S-EPMC9369596 | biostudies-literature

A mixed-scale dense convolutional neural network for image analysis.

Project description:Deep convolutional neural networks have been successfully applied to many image-processing problems in recent works. Popular network architectures often add additional operations and connections to the standard architecture to enable training deeper networks. To achieve accurate results in practice, a large number of trainable parameters are often required. Here, we introduce a network architecture based on using dilated convolutions to capture features at different image scales and densely connecting all feature maps with each other. The resulting architecture is able to achieve accurate results with relatively few parameters and consists of a single set of operations, making it easier to implement, train, and apply in practice, and automatically adapts to different problems. We compare results of the proposed network architecture with popular existing architectures for several segmentation problems, showing that the proposed architecture is able to achieve accurate results with fewer parameters, with a reduced risk of overfitting the training data.

| S-EPMC5777062 | biostudies-literature

A Shallow Convolutional Neural Network Predicts Prognosis of Lung Cancer Patients in Multi-Institutional CT-Image Data.

Project description:Lung cancer is the most common fatal malignancy in adults worldwide, and non-small cell lung cancer (NSCLC) accounts for 85% of lung cancer diagnoses. Computed tomography (CT) is routinely used in clinical practice to determine lung cancer treatment and assess prognosis. Here, we developed LungNet, a shallow convolutional neural network for predicting outcomes of NSCLC patients. We trained and evaluated LungNet on four independent cohorts of NSCLC patients from four medical centers: Stanford Hospital (n = 129), H. Lee Moffitt Cancer Center and Research Institute (n = 185), MAASTRO Clinic (n = 311) and Charité - Universitätsmedizin (n=84). We show that outcomes from LungNet are predictive of overall survival in all four independent survival cohorts as measured by concordance indices of 0.62, 0.62, 0.62 and 0.58 on cohorts 1, 2, 3, and 4, respectively. Further, the survival model can be used, via transfer learning, for classifying benign vs malignant nodules on the Lung Image Database Consortium (n = 1010), with improved performance (AUC=0.85) versus training from scratch (AUC=0.82). LungNet can be used as a noninvasive predictor for prognosis in NSCLC patients and can facilitate interpretation of CT images for lung cancer stratification and prognostication.

| S-EPMC8008967 | biostudies-literature

Multi-Sensor and Decision-Level Fusion-Based Structural Damage Detection Using a One-Dimensional Convolutional Neural Network.

Project description:This paper presents a novel approach to substantially improve the detection accuracy of structural damage via a one-dimensional convolutional neural network (1-D CNN) and a decision-level fusion strategy. As structural damage usually induces changes in the dynamic responses of a structure, a CNN can effectively extract structural damage information from the vibration signals and classify them into the corresponding damage categories. However, it is difficult to build a large-scale sensor system in practical engineering; the collected vibration signals are usually non-synchronous and contain incomplete structure information, resulting in some evident errors in the decision stage of the CNN. In this study, the acceleration signals of multiple acquisition points were obtained, and the signals of each acquisition point were used to train a 1-D CNN, and their performances were evaluated by using the corresponding testing samples. Subsequently, the prediction results of all CNNs were fused (decision-level fusion) to obtain the integrated detection results. This method was validated using both numerical and experimental models and compared with a control experiment (data-level fusion) in which all the acceleration signals were used to train a CNN. The results confirmed that: by fusing the prediction results of multiple CNN models, the detection accuracy was significantly improved; for the numerical and experimental models, the detection accuracy was 10% and 16-30%, respectively, higher than that of the control experiment. It was demonstrated that: training a CNN using the acceleration signals of each acquisition point and making its own decision (the CNN output) and then fusing these decisions could effectively improve the accuracy of damage detection of the CNN.

| S-EPMC8226517 | biostudies-literature

Chinese Herbal Medicine Image Recognition and Retrieval by Convolutional Neural Network.

Project description:Chinese herbal medicine image recognition and retrieval have great potential of practical applications. Several previous studies have focused on the recognition with hand-crafted image features, but there are two limitations in them. Firstly, most of these hand-crafted features are low-level image representation, which is easily affected by noise and background. Secondly, the medicine images are very clean without any backgrounds, which makes it difficult to use in practical applications. Therefore, designing high-level image representation for recognition and retrieval in real world medicine images is facing a great challenge. Inspired by the recent progress of deep learning in computer vision, we realize that deep learning methods may provide robust medicine image representation. In this paper, we propose to use the Convolutional Neural Network (CNN) for Chinese herbal medicine image recognition and retrieval. For the recognition problem, we use the softmax loss to optimize the recognition network; then for the retrieval problem, we fine-tune the recognition network by adding a triplet loss to search for the most similar medicine images. To evaluate our method, we construct a public database of herbal medicine images with cluttered backgrounds, which has in total 5523 images with 95 popular Chinese medicine categories. Experimental results show that our method can achieve the average recognition precision of 71% and the average retrieval precision of 53% over all the 95 medicine categories, which are quite promising given the fact that the real world images have multiple pieces of occluded herbal and cluttered backgrounds. Besides, our proposed method achieves the state-of-the-art performance by improving previous studies with a large margin.

| S-EPMC4892594 | biostudies-literature

Diagnostic Performance of a Convolutional Neural Network for Diminutive Colorectal Polyp Recognition

Project description:Interventions: None Primary outcome(s): The primary outcome of the study is the accuracy of the CAD-CNN system for predicting histology of diminutive colorectal polyps (1-5mm) compared with the accuracy of the prediction of the endoscopist. Both the CAD-CNN system and the endoscopist will use NBI for their predictions. Accuracy is defined as the percentage of correctly predicted optical diagnoses of the CAD-CNN system and/or endoscopist compared to the gold standard pathology. For the calculation of the accuracy, adenomas and SSLs will be dichotomised as neoplastic polyps, while HPs and other non-neoplastic histology are considered non-neoplastic. Study Design: N/A: single arm study, Open (masking not used), N/A , unknown, Other

| 2443187 | ecrin-mdr-crc

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data