Natural Image Reconstruction From fMRI Using Deep Learning: A Survey.
Ontology highlight
ABSTRACT:
SUBMITTER: Rakhimberdina Z
PROVIDER: S-EPMC8722107 | biostudies-literature | 2021
REPOSITORIES: biostudies-literature
Publications
Natural Image Reconstruction From fMRI Using Deep Learning: A Survey.
Rakhimberdina Zarina Z Jodelet Quentin Q Liu Xin X Murata Tsuyoshi T
Frontiers in neuroscience 20211220
With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We ...[more]
Similar Datasets
Project description:We investigated the effect of deep learning-based image reconstruction (DLIR) compared to iterative reconstruction on image quality in CT pulmonary angiography (CTPA) for suspected pulmonary embolism (PE). For 220 patients with suspected PE, CTPA studies were reconstructed using filtered back projection (FBP), adaptive statistical iterative reconstruction (ASiR-V 30%, 60% and 90%) and DLIR (low, medium and high strength). Contrast-to-noise ratio (CNR) served as the primary parameter of objective image quality. Subgroup analyses were performed for normal weight, overweight and obese individuals. For patients with confirmed PE (n = 40), we further measured PE-specific CNR. Subjective image quality was assessed independently by two experienced radiologists. CNR was lowest for FBP and enhanced with increasing levels of ASiR-V and, even more with increasing strength of DLIR. High strength DLIR resulted in an additional improvement in CNR by 29-67% compared to ASiR-V 90% (p < 0.05). PE-specific CNR increased by 75% compared to ASiR-V 90% (p < 0.05). Subjective image quality was significantly higher for medium and high strength DLIR compared to all other image reconstructions (p < 0.05). In CT pulmonary angiography, DLIR significantly outperforms iterative reconstruction for increasing objective and subjective image quality. This may allow for further reductions in radiation exposure in suspected PE.
| S-EPMC10827738 | biostudies-literature
Project description:Recently, deep neural networks have been widely and successfully applied in computer vision tasks and have attracted growing interest in medical imaging. One barrier for the application of deep neural networks to medical imaging is the need for large amounts of prior training pairs, which is not always feasible in clinical practice. This is especially true for medical image reconstruction problems, where raw data are needed. Inspired by the deep image prior framework, in this paper, we proposed a personalized network training method where no prior training pairs are needed, but only the patient's own prior information. The network is updated during the iterative reconstruction process using the patient-specific prior information and measured data. We formulated the maximum-likelihood estimation as a constrained optimization problem and solved it using the alternating direction method of multipliers algorithm. Magnetic resonance imaging guided positron emission tomography reconstruction was employed as an example to demonstrate the effectiveness of the proposed framework. Quantification results based on simulation and real data show that the proposed reconstruction framework can outperform Gaussian post-smoothing and anatomically guided reconstructions using the kernel method or the neural-network penalty.
| S-EPMC6584077 | biostudies-literature
Project description:Phase recovery from intensity-only measurements forms the heart of coherent imaging techniques and holography. In this study, we demonstrate that a neural network can learn to perform phase recovery and holographic image reconstruction after appropriate training. This deep learning-based approach provides an entirely new framework to conduct holographic imaging by rapidly eliminating twin-image and self-interference-related spatial artifacts. This neural network-based method is fast to compute and reconstructs phase and amplitude images of the objects using only one hologram, requiring fewer measurements in addition to being computationally faster. We validated this method by reconstructing the phase and amplitude images of various samples, including blood and Pap smears and tissue sections. These results highlight that challenging problems in imaging science can be overcome through machine learning, providing new avenues to design powerful computational imaging systems.
| S-EPMC6060068 | biostudies-literature
Project description:ObjectiveTo evaluate the image quality and lesion detectability of lower-dose CT (LDCT) of the abdomen and pelvis obtained using a deep learning image reconstruction (DLIR) algorithm compared with those of standard-dose CT (SDCT) images.Materials and methodsThis retrospective study included 123 patients (mean age ± standard deviation, 63 ± 11 years; male:female, 70:53) who underwent contrast-enhanced abdominopelvic LDCT between May and August 2020 and had prior SDCT obtained using the same CT scanner within a year. LDCT images were reconstructed with hybrid iterative reconstruction (h-IR) and DLIR at medium and high strengths (DLIR-M and DLIR-H), while SDCT images were reconstructed with h-IR. For quantitative image quality analysis, image noise, signal-to-noise ratio, and contrast-to-noise ratio were measured in the liver, muscle, and aorta. Among the three different LDCT reconstruction algorithms, the one showing the smallest difference in quantitative parameters from those of SDCT images was selected for qualitative image quality analysis and lesion detectability evaluation. For qualitative analysis, overall image quality, image noise, image sharpness, image texture, and lesion conspicuity were graded using a 5-point scale by two radiologists. Observer performance in focal liver lesion detection was evaluated by comparing the jackknife free-response receiver operating characteristic figures-of-merit (FOM).ResultsLDCT (35.1% dose reduction compared with SDCT) images obtained using DLIR-M showed similar quantitative measures to those of SDCT with h-IR images. All qualitative parameters of LDCT with DLIR-M images but image texture were similar to or significantly better than those of SDCT with h-IR images. The lesion detectability on LDCT with DLIR-M images was not significantly different from that of SDCT with h-IR images (reader-averaged FOM, 0.887 vs. 0.874, respectively; p = 0.581).ConclusionOverall image quality and detectability of focal liver lesions is preserved in contrast-enhanced abdominopelvic LDCT obtained with DLIR-M relative to those in SDCT with h-IR.
| S-EPMC8961013 | biostudies-literature
Project description:Background This study aims to evaluate the impact of a novel deep learning-based image reconstruction (DLIR) algorithm on the image quality in computed tomographic angiography (CTA) for pre-interventional planning of transcatheter aortic valve implantation (TAVI). Methods We analyzed 50 consecutive patients (median age 80 years, 25 men) who underwent TAVI planning CT on a 256-dectector-row CT. Images were reconstructed with adaptive statistical iterative reconstruction V (ASIR-V) and DLIR. Intravascular image noise, edge sharpness, signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) were quantified for ascending aorta, descending aorta, abdominal aorta and iliac arteries. Two readers (one radiologist and one interventional cardiologist) scored task-specific subjective image quality on a five-point scale. Results DLIR significantly reduced median image noise by 29–57% at all anatomical locations (all P<0.001). Accordingly, median SNR improved by 44–133% (all P<0.001) and median CNR improved by 44–125% (all P<0.001). DLIR significantly improved subjective image quality for all four pre-specified TAVI-specific tasks (measuring the annulus, assessing valve morphology and calcifications, the coronary ostia, and the suitability of the aorto-iliac access route) for both the radiologist and the interventional cardiologist (P≤0.001). Measurements of the aortic annulus circumference, area and diameter did not differ between ASIR-V and DLIR reconstructions (all P>0.05). Conclusions DLIR significantly improves objective and subjective image quality in TAVI planning CT compared to a state-of-the-art iterative reconstruction without affecting measurements of the aortic annulus. This may provide an opportunity for further reductions in contrast medium volume in this population.
| S-EPMC9929406 | biostudies-literature
Project description:Fast and accurate reconstruction of magnetic resonance (MR) images from under-sampled data is important in many clinical applications. In recent years, deep learning-based methods have been shown to produce superior performance on MR image reconstruction. However, these methods require large amounts of data which is difficult to collect and share due to the high cost of acquisition and medical data privacy regulations. In order to overcome this challenge, we propose a federated learning (FL) based solution in which we take advantage of the MR data available at different institutions while preserving patients' privacy. However, the generalizability of models trained with the FL setting can still be suboptimal due to domain shift, which results from the data collected at multiple institutions with different sensors, disease types, and acquisition protocols, etc. With the motivation of circumventing this challenge, we propose a cross-site modeling for MR image reconstruction in which the learned intermediate latent features among different source sites are aligned with the distribution of the latent features at the target site. Extensive experiments are conducted to provide various insights about FL for MR image reconstruction. Experimental results demonstrate that the proposed framework is a promising direction to utilize multi-institutional data without compromising patients' privacy for achieving improved MR image reconstruction. Our code is available at https://github.com/guopengf/FL-MRCM.
| S-EPMC9017654 | biostudies-literature
Project description:Conventional reconstruction methods for photoacoustic images are not suitable for the scenario of sparse sensing and geometrical limitation. To overcome these challenges and enhance the quality of reconstruction, several learning-based methods have recently been introduced for photoacoustic tomography reconstruction. The goal of this study is to compare and systematically evaluate the recently proposed learning-based methods and modified networks for photoacoustic image reconstruction. Specifically, learning-based post-processing methods and model-based learned iterative reconstruction methods are investigated. In addition to comparing the differences inherently brought by the models, we also study the impact of different inputs on the reconstruction effect. Our results demonstrate that the reconstruction performance mainly stems from the effective amount of information carried by the input. The inherent difference of the models based on the learning-based post-processing method does not provide a significant difference in photoacoustic image reconstruction. Furthermore, the results indicate that the model-based learned iterative reconstruction method outperforms all other learning-based post-processing methods in terms of generalizability and robustness.
| S-EPMC8165448 | biostudies-literature
Project description:Deep learning has been used to improve photoacoustic (PA) image reconstruction. One major challenge is that errors cannot be quantified to validate predictions when ground truth is unknown. Validation is key to quantitative applications, especially using limited-bandwidth ultrasonic linear detector arrays. Here, we propose a hybrid Bayesian convolutional neural network (Hybrid-BCNN) to jointly predict PA image and segmentation with error (uncertainty) predictions. Each output pixel represents a probability distribution where error can be quantified. The Hybrid-BCNN was trained with simulated PA data and applied to both simulations and experiments. Due to the sparsity of PA images, segmentation focuses Hybrid-BCNN on minimizing the loss function in regions with PA signals for better predictions. The results show that accurate PA segmentations and images are obtained, and error predictions are highly statistically correlated to actual errors. To leverage error predictions, confidence processing created PA images above a specific confidence level.
| S-EPMC11424948 | biostudies-literature
Project description:BackgroundSingle photon emission computed tomography (SPECT) is an important functional tool for clinical diagnosis and scientific research of brain disorders, but suffers from limited spatial resolution and high noise due to hardware design and imaging physics. The present study is to develop a deep learning technique for SPECT image reconstruction that directly converts raw projection data to image with high resolution and low noise, while an efficient training method specifically applicable to medical image reconstruction is presented.MethodsA custom software was developed to generate 20,000 2-D brain phantoms, of which 16,000 were used to train the neural network, 2,000 for validation, and the final 2,000 for testing. To reduce development difficulty, a two-step training strategy for network design was adopted. We first compressed full-size activity image (128×128 pixels) to a one-D vector consisting of 256×1 pixels, accomplished by an autoencoder (AE) consisting of an encoder and a decoder. The vector is a good representation of the full-size image in a lower-dimensional space and was used as a compact label to develop the second network that maps between the projection-data domain and the vector domain. Since the label had 256 pixels only, the second network was compact and easy to converge. The second network, when successfully developed, was connected to the decoder (a portion of AE) to decompress the vector to a regular 128×128 image. Therefore, a complex network was essentially divided into two compact neural networks trained separately in sequence but eventually connectable.ResultsA total of 2,000 test examples, a synthetic brain phantom, and de-identified patient data were used to validate SPECTnet. Results obtained from SPECTnet were compared with those obtained from our clinic OS-EM method. Images with lower noise and more accurate information in the uptake areas were obtained by SPECTnet.ConclusionsThe challenge of developing a complex deep neural network is reduced by training two separate compact connectable networks. The combination of the two networks forms the full version of SPECTnet. Results show that the developed neural network can produce more accurate SPECT images.
| S-EPMC8246183 | biostudies-literature
Project description:Background and Objectives: Although reducing the radiation dose level is important during diagnostic computed tomography (CT) applications, effective image quality enhancement strategies are crucial to compensate for the degradation that is caused by a dose reduction. We performed this prospective study to quantify emphysema on ultra-low-dose CT images that were reconstructed using deep learning-based image reconstruction (DLIR) algorithms, and compared and evaluated the accuracies of DLIR algorithms versus standard-dose CT. Materials and Methods: A total of 32 patients were prospectively enrolled, and all underwent standard-dose and ultra-low-dose (120 kVp; CTDIvol < 0.7 mGy) chest CT scans at the same time in a single examination. A total of six image datasets (filtered back projection (FBP) for standard-dose CT, and FBP, adaptive statistical iterative reconstruction (ASIR-V) 50%, DLIR-low, DLIR-medium, DLIR-high for ultra-low-dose CT) were reconstructed for each patient. Image noise values, emphysema indices, total lung volumes, and mean lung attenuations were measured in the six image datasets and compared (one-way repeated measures ANOVA). Results: The mean effective doses for standard-dose and ultra-low-dose CT scans were 3.43 ± 0.57 mSv and 0.39 ± 0.03 mSv, respectively (p < 0.001). The total lung volume and mean lung attenuation of five image datasets of ultra-low-dose CT scans, emphysema indices of ultra-low-dose CT scans reconstructed using ASIR-V 50 or DLIR-low, and the image noise of ultra-low-dose CT scans that were reconstructed using DLIR-low were not different from those of standard-dose CT scans. Conclusions: Ultra-low-dose CT images that were reconstructed using DLIR-low were found to be useful for emphysema quantification at a radiation dose of only 11% of that required for standard-dose CT.
| S-EPMC9317892 | biostudies-literature