Single image super-resolution via Image Quality Assessment-Guided Deep Learning Network.
Ontology highlight
ABSTRACT:
SUBMITTER: Xiong Z
PROVIDER: S-EPMC7595386 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
Single image super-resolution via Image Quality Assessment-Guided Deep Learning Network.
Xiong Zhengqiang Z Lin Manhui M Lin Zhen Z Sun Tao T Yang Guangyi G Wang Zhengxing Z
PloS one 20201029 10
In recent years, deep learning (DL) networks have been widely used in super-resolution (SR) and exhibit improved performance. In this paper, an image quality assessment (IQA)-guided single image super-resolution (SISR) method is proposed in DL architecture, in order to achieve a nice tradeoff between perceptual quality and distortion measure of the SR result. Unlike existing DL-based SR algorithms, an IQA net is introduced to extract perception features from SR results, calculate corresponding l ...[more]
Similar Datasets
Project description:Image super-resolution (SR) is one of the vital image processing methods that improve the resolution of an image in the field of computer vision. In the last two decades, significant progress has been made in the field of super-resolution, especially by utilizing deep learning methods. This survey is an effort to provide a detailed survey of recent progress in single-image super-resolution in the perspective of deep learning while also informing about the initial classical methods used for image super-resolution. The survey classifies the image SR methods into four categories, i.e., classical methods, supervised learning-based methods, unsupervised learning-based methods, and domain-specific SR methods. We also introduce the problem of SR to provide intuition about image quality metrics, available reference datasets, and SR challenges. Deep learning-based approaches of SR are evaluated using a reference dataset. Some of the reviewed state-of-the-art image SR methods include the enhanced deep SR network (EDSR), cycle-in-cycle GAN (CinCGAN), multiscale residual network (MSRN), meta residual dense network (Meta-RDN), recurrent back-projection network (RBPN), second-order attention network (SAN), SR feedback network (SRFBN) and the wavelet-based residual attention network (WRAN). Finally, this survey is concluded with future directions and trends in SR and open problems in SR to be addressed by the researchers.
| S-EPMC8293932 | biostudies-literature
Project description:PURPOSE:Probe-based confocal laser endomicroscopy (pCLE) is a recent imaging modality that allows performing in vivo optical biopsies. The design of pCLE hardware, and its reliance on an optical fibre bundle, fundamentally limits the image quality with a few tens of thousands fibres, each acting as the equivalent of a single-pixel detector, assembled into a single fibre bundle. Video registration techniques can be used to estimate high-resolution (HR) images by exploiting the temporal information contained in a sequence of low-resolution (LR) images. However, the alignment of LR frames, required for the fusion, is computationally demanding and prone to artefacts. METHODS:In this work, we propose a novel synthetic data generation approach to train exemplar-based Deep Neural Networks (DNNs). HR pCLE images with enhanced quality are recovered by the models trained on pairs of estimated HR images (generated by the video registration algorithm) and realistic synthetic LR images. Performance of three different state-of-the-art DNNs techniques were analysed on a Smart Atlas database of 8806 images from 238 pCLE video sequences. The results were validated through an extensive image quality assessment that takes into account different quality scores, including a Mean Opinion Score (MOS). RESULTS:Results indicate that the proposed solution produces an effective improvement in the quality of the obtained reconstructed image. CONCLUSION:The proposed training strategy and associated DNNs allows us to perform convincing super-resolution of pCLE images.
| S-EPMC5973979 | biostudies-literature
Project description:Single-image super resolution is a process of obtaining a high-resolution image from a set of low-resolution observations by signal processing. While super resolution has been demonstrated to improve image quality in scaled down images in the image domain, its effects on the Fourier-based image acquisition technique, such as MRI, remains unknown.We performed high-resolution ex vivo late gadolinium enhancement (LGE) magnetic resonance imaging (0.4 × 0.4 × 0.4 mm3) in postinfarction swine hearts (n = 24). The swine hearts were divided into the training set (n = 14) and the test set (n = 10), and in all hearts, low-resolution images were simulated from the high-resolution images. In the training set, super-resolution dictionaries with pairs of small matching patches of the high- and low-resolution images were created. In the test set, super resolution recovered high-resolution images from low-resolution images using the dictionaries. The same algorithm was also applied to patient LGE (n = 4) to assess its effects. Compared with interpolated images, super resolution significantly improved basic image quality indices (P < 0.001). Super resolution using Fourier-based zero padding achieved the best image quality. However, the magnitude of improvement was small in images with zero padding. Super resolution substantially improved the spatial resolution of the patient LGE images by sharpening the edges of the heart and the scar. In conclusion, single-image super resolution significantly improves image errors. However, the magnitude of improvement was relatively small in images with Fourier-based zero padding. These findings provide evidence to support its potential use in myocardial scar imaging.
| S-EPMC4331080 | biostudies-literature
Project description:Single-particle cryo-electron microscopy (cryo-EM) is a powerful imaging modality capable of visualizing proteins and macromolecular complexes at near-atomic resolution. The low electron-doses used to prevent radiation damage to the biological samples, however, result in images where the power of the noise is 100 times greater than the power of the signal. To overcome these low signal-to-noise ratios (SNRs), hundreds of thousands of particle projections are averaged to determine the three-dimensional structure of the molecule of interest. The sampling requirements of high-resolution imaging impose limitations on the pixel sizes that can be used for acquisition, limiting the size of the field of view and requiring data collection sessions of several days to accumulate sufficient numbers of particles. Meanwhile, recent image super-resolution (SR) techniques based on neural networks have shown state-of-the-art performance on natural images. Building on these advances, here, we present a multiple-image SR algorithm based on deep internal learning designed specifically to work under low-SNR conditions. Our approach leverages the internal image statistics of cryo-EM movies and does not require training on ground-truth data. When applied to single-particle datasets of apoferritin and T20S proteasome, we show that the resolution of the 3D structure obtained from SR micrographs can surpass the limits imposed by the imaging system. Our results indicate that the combination of low magnification imaging with in silico image SR has the potential to accelerate cryo-EM data collection by virtue of including more particles in each exposure and doing so without sacrificing resolution.
| S-EPMC10951919 | biostudies-literature
Project description:Background Cardiac MRI is limited by long acquisition times, yet faster acquisition of smaller-matrix images reduces spatial detail. Deep learning (DL) might enable both faster acquisition and higher spatial detail via super-resolution. Purpose To explore the feasibility of using DL to enhance spatial detail from small-matrix MRI acquisitions and evaluate its performance against that of conventional image upscaling methods. Materials and Methods Short-axis cine cardiac MRI examinations performed between January 2012 and December 2018 at one institution were retrospectively collected for algorithm development and testing. Convolutional neural networks (CNNs), a form of DL, were trained to perform super resolution in image space by using synthetically generated low-resolution data. There were 70%, 20%, and 10% of examinations allocated to training, validation, and test sets, respectively. CNNs were compared against bicubic interpolation and Fourier-based zero padding by calculating the structural similarity index (SSIM) between high-resolution ground truth and each upscaling method. Means and standard deviations of the SSIM were reported, and statistical significance was determined by using the Wilcoxon signed-rank test. For evaluation of clinical performance, left ventricular volumes were measured, and statistical significance was determined by using the paired Student t test. Results For CNN training and retrospective analysis, 400 MRI scans from 367 patients (mean age, 48 years ± 18; 214 men) were included. All CNNs outperformed zero padding and bicubic interpolation at upsampling factors from two to 64 (P < .001). CNNs outperformed zero padding on more than 99.2% of slices (9828 of 9907). In addition, 10 patients (mean age, 51 years ± 22; seven men) were prospectively recruited for super-resolution MRI. Super-resolved low-resolution images yielded left ventricular volumes comparable to those from full-resolution images (P > .05), and super-resolved full-resolution images appeared to further enhance anatomic detail. Conclusion Deep learning outperformed conventional upscaling methods and recovered high-frequency spatial information. Although training was performed only on short-axis cardiac MRI examinations, the proposed strategy appeared to improve quality in other imaging planes. © RSNA, 2020 Online supplemental material is available for this article.
| S-EPMC7263289 | biostudies-literature
Project description:Single-molecule localization microscopy (SMLM) can be used to resolve subcellular structures and achieve a tenfold improvement in spatial resolution compared to that obtained by conventional fluorescence microscopy. However, the separation of single-molecule fluorescence events that requires thousands of frames dramatically increases the image acquisition time and phototoxicity, impeding the observation of instantaneous intracellular dynamics. Here we develop a deep-learning based single-frame super-resolution microscopy (SFSRM) method which utilizes a subpixel edge map and a multicomponent optimization strategy to guide the neural network to reconstruct a super-resolution image from a single frame of a diffraction-limited image. Under a tolerable signal density and an affordable signal-to-noise ratio, SFSRM enables high-fidelity live-cell imaging with spatiotemporal resolutions of 30 nm and 10 ms, allowing for prolonged monitoring of subcellular dynamics such as interplays between mitochondria and endoplasmic reticulum, the vesicle transport along microtubules, and the endosome fusion and fission. Moreover, its adaptability to different microscopes and spectra makes it a useful tool for various imaging systems.
| S-EPMC10195829 | biostudies-literature
Project description:The disease spots on the grape leaves can be detected by using the image processing and deep learning methods. However, the accuracy and efficiency of the detection are still the challenges. The convolutional substrate information is fuzzy, and the detection results are not satisfactory if the disease spot is relatively small. In particular, the detection will be difficult if the number of pixels of the spot is <32 × 32 in the image. In order to effectively address this problem, we present a super-resolution image enhancement and convolutional neural network-based algorithm for the detection of black rot on grape leaves. First, the original image is up-sampled and enhanced with local details using the bilinear interpolation. As a result, the number of pixels in the image increase. Then, the enhanced images are fed into the proposed YOLOv3-SPP network for detection. In the proposed network, the IOU (Intersection Over Union, IOU) in the original YOLOv3 network is replaced with GIOU (Generalized Intersection Over Union, GIOU). In addition, we also add the SPP (Spatial Pyramid Pooling, SPP) module to improve the detection performance of the network. Finally, the official pre-trained weights of YOLOv3 are used for fast convergence. The test set test_pv from the Plant Village and the test set test_orchard from the orchard field were used to evaluate the network performance. The results of test_pv show that the grape leaf black rot is detected by the YOLOv3-SPP with 95.79% detection accuracy and 94.52% detector recall, which is a 5.94% greater in terms of accuracy and 10.67% greater in terms of recall as compared to the original YOLOv3. The results of test_orchard show that the method proposed in this paper can be applied in field environment with 86.69% detection precision and 82.27% detector recall, and the accuracy and recall were improved to 94.05 and 93.26% if the images with the simple background. Therefore, the detection method proposed in this work effectively solves the detection task of small targets and improves the detection effectiveness of the grape leaf black rot.
| S-EPMC8277438 | biostudies-literature
Project description:BackgroundA variety of external factors might seriously degrade PET image quality and lead to inconsistent results. The aim of this study is to explore a potential PET image quality assessment (QA) method with deep learning (DL).MethodsA total of 89 PET images were acquired from Peking Union Medical College Hospital (PUMCH) in China in this study. Ground-truth quality for images was assessed by two senior radiologists and classified into five grades (grade 1, grade 2, grade 3, grade 4, and grade 5). Grade 5 is the best image quality. After preprocessing, the Dense Convolutional Network (DenseNet) was trained to automatically recognize optimal- and poor-quality PET images. Accuracy (ACC), sensitivity, specificity, receiver operating characteristic curve (ROC), and area under the ROC Curve (AUC) were used to evaluate the diagnostic properties of all models. All indicators of models were assessed using fivefold cross-validation. An image quality QA tool was developed based on our deep learning model. A PET QA report can be automatically obtained after inputting PET images.ResultsFour tasks were generated. Task2 showed worst performance in AUC,ACC, specificity and sensitivity among 4 tasks, and task1 showed unstable performance between training and testing and task3 showed low specificity in both training and testing. Task 4 showed the best diagnostic properties and discriminative performance between poor image quality (grade 1, grade 2) and good quality (grade 3, grade 4, grade 5) images. The automated quality assessment of task 4 showed ACC = 0.77, specificity = 0.71, and sensitivity = 0.83, in the train set; ACC = 0.85, specificity = 0.79, and sensitivity = 0.91, in the test set, respectively. The ROC measuring performance of task 4 had an AUC of 0.86 in the train set and 0.91 in the test set. The image QA tool could output basic information of images, scan and reconstruction parameters, typical instances of PET images, and deep learning score.ConclusionsThis study highlights the feasibility of the assessment of image quality in PET images using a deep learning model, which may assist with accelerating clinical research by reliably assessing image quality.
| S-EPMC10243071 | biostudies-literature
Project description:MotivationSuper-resolution fluorescence microscopy with a resolution beyond the diffraction limit of light, has become an indispensable tool to directly visualize biological structures in living cells at a nanometer-scale resolution. Despite advances in high-density super-resolution fluorescent techniques, existing methods still have bottlenecks, including extremely long execution time, artificial thinning and thickening of structures, and lack of ability to capture latent structures.ResultsHere, we propose a novel deep learning guided Bayesian inference (DLBI) approach, for the time-series analysis of high-density fluorescent images. Our method combines the strength of deep learning and statistical inference, where deep learning captures the underlying distribution of the fluorophores that are consistent with the observed time-series fluorescent images by exploring local features and correlation along time-axis, and statistical inference further refines the ultrastructure extracted by deep learning and endues physical meaning to the final image. In particular, our method contains three main components. The first one is a simulator that takes a high-resolution image as the input, and simulates time-series low-resolution fluorescent images based on experimentally calibrated parameters, which provides supervised training data to the deep learning model. The second one is a multi-scale deep learning module to capture both spatial information in each input low-resolution image as well as temporal information among the time-series images. And the third one is a Bayesian inference module that takes the image from the deep learning module as the initial localization of fluorophores and removes artifacts by statistical inference. Comprehensive experimental results on both real and simulated datasets demonstrate that our method provides more accurate and realistic local patch and large-field reconstruction than the state-of-the-art method, the 3B analysis, while our method is more than two orders of magnitude faster.Availability and implementationThe main program is available at https://github.com/lykaust15/DLBI.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC6022599 | biostudies-literature
Project description:BACKGROUND AND PURPOSE:The superior soft-tissue contrast of 4D-T2w MRI motivates its use for delineation in radiotherapy treatment planning. We address current limitations of slice-selective implementations, including thick slices and artefacts originating from data incompleteness and variable breathing. MATERIALS AND METHODS:A method was developed to calculate midposition and 4D-T2w images of the whole thorax from continuously acquired axial and sagittal 2D-T2w MRI (1.5?×?1.5?×?5.0?mm3). The method employed image-derived respiratory surrogates, deformable image registration and super-resolution reconstruction. Volunteer imaging and a respiratory motion phantom were used for validation. The minimum number of dynamic acquisitions needed to calculate a representative midposition image was investigated by retrospectively subsampling the data (10-30 dynamic acquisitions). RESULTS:Super-resolution 4D-T2w MRI (1.0?×?1.0?×?1.0?mm3, 8 respiratory phases) did not suffer from data incompleteness and exhibited reduced stitching artefacts compared to sorted multi-slice MRI. Experiments using a respiratory motion phantom and colour-intensity projection images demonstrated a minor underestimation of the motion range. Midposition diaphragm differences in retrospectively subsampled acquisitions were <1.1?mm compared to the full dataset. 10 dynamic acquisitions were found sufficient to generate midposition MRI. CONCLUSIONS:A motion-modelling and super-resolution method was developed to calculate high quality 4D/midposition T2w MRI from orthogonal 2D-T2w MRI.
| S-EPMC6294732 | biostudies-literature