Image reconstruction through a multimode fiber with a simple neural network architecture.
Ontology highlight
ABSTRACT:
SUBMITTER: Zhu C
PROVIDER: S-EPMC7806887 | biostudies-literature | 2021 Jan
REPOSITORIES: biostudies-literature
Publications
Image reconstruction through a multimode fiber with a simple neural network architecture.
Zhu Changyan C Chan Eng Aik EA Wang You Y Peng Weina W Guo Ruixiang R Zhang Baile B Soci Cesare C Chong Yidong Y
Scientific reports 20210113 1
Multimode fibers (MMFs) have the potential to carry complex images for endoscopy and related applications, but decoding the complex speckle patterns produced by mode-mixing and modal dispersion in MMFs is a serious challenge. Several groups have recently shown that convolutional neural networks (CNNs) can be trained to perform high-fidelity MMF image reconstruction. We find that a considerably simpler neural network architecture, the single hidden layer dense neural network, performs at least as ...[more]
Similar Datasets
Project description:In positron emission tomography (PET) image reconstruction, the Bayesian framework with various regularization terms has been implemented to constrain the radio tracer distribution. Varying the regularizing weight of a maximum a posteriori (MAP) algorithm specifies a lower bound of the tradeoff between variance and spatial resolution measured from the reconstructed images. The purpose of this paper is to build a patch-based image enhancement scheme to reduce the size of the unachievable region below the bound and thus to quantitatively improve the Bayesian PET imaging. We cast the proposed enhancement as a regression problem which models a highly nonlinear and spatial-varying mapping between the reconstructed image patches and an enhanced image patch. An artificial neural network model named multilayer perceptron (MLP) with backpropagation was used to solve this regression problem through learning from examples. Using the BrainWeb phantoms, we simulated brain PET data at different count levels of different subjects with and without lesions. The MLP was trained using the image patches reconstructed with a MAP algorithm of different regularization parameters for one normal subject at a certain count level. To evaluate the performance of the trained MLP, reconstructed images from other simulations and two patient brain PET imaging data sets were processed. In every testing cases, we demonstrate that the MLP enhancement technique improves the noise and bias tradeoff compared with the MAP reconstruction using different regularizing weights thus decreasing the size of the unachievable region defined by the MAP algorithm in the variance/resolution plane.
| S-EPMC6132251 | biostudies-literature
Project description:Multimode fibers (MMFs) are an example of a highly scattering medium, which scramble the coherent light propagating within them to produce seemingly random patterns. Thus, for applications such as imaging and image projection through an MMF, careful measurements of the relationship between the inputs and outputs of the fiber are required. We show, as a proof of concept, that a deep neural network can learn the input-output relationship in a 0.75?m long MMF. Specifically, we demonstrate that a deep convolutional neural network (CNN) can learn the nonlinear relationships between the amplitude of the speckle pattern (phase information lost) obtained at the output of the fiber and the phase or the amplitude at the input of the fiber. Effectively, the network performs a nonlinear inversion task. We obtained image fidelities (correlations) as high as ~98% for reconstruction and ~94% for image projection in the MMF compared with the image recovered using the full knowledge of the system transmission characterized with the complex measured matrix. We further show that the network can be trained for transfer learning, i.e., it can transmit images through the MMF, which belongs to another class not used for training/testing.
| S-EPMC6168552 | biostudies-literature
Project description:PURPOSE:To develop and evaluate a novel deep learning-based reconstruction framework called SANTIS (Sampling-Augmented Neural neTwork with Incoherent Structure) for efficient MR image reconstruction with improved robustness against sampling pattern discrepancy. METHODS:With a combination of data cycle-consistent adversarial network, end-to-end convolutional neural network mapping, and data fidelity enforcement for reconstructing undersampled MR data, SANTIS additionally utilizes a sampling-augmented training strategy by extensively varying undersampling patterns during training, so that the network is capable of learning various aliasing structures and thereby removing undersampling artifacts more effectively and robustly. The performance of SANTIS was demonstrated for accelerated knee imaging and liver imaging using a Cartesian trajectory and a golden-angle radial trajectory, respectively. Quantitative metrics were used to assess its performance against different references. The feasibility of SANTIS in reconstructing dynamic contrast-enhanced images was also demonstrated using transfer learning. RESULTS:Compared to conventional reconstruction that exploits image sparsity, SANTIS achieved consistently improved reconstruction performance (lower errors and greater image sharpness). Compared to standard learning-based methods without sampling augmentation (e.g., training with a fixed undersampling pattern), SANTIS provides comparable reconstruction performance, but significantly improved robustness, against sampling pattern discrepancy. SANTIS also achieved encouraging results for reconstructing liver images acquired at different contrast phases. CONCLUSION:By extensively varying undersampling patterns, the sampling-augmented training strategy in SANTIS can remove undersampling artifacts more robustly. The novel concept behind SANTIS can particularly be useful for improving the robustness of deep learning-based image reconstruction against discrepancy between training and inference, an important, but currently less explored, topic.
| S-EPMC6660404 | biostudies-literature
Project description:Multimode fibers can guide thousands of modes capable of delivering spatial information. Unfortunately, mode dispersion and coupling have so far prevented their use in endoscopic applications. To address this long-lasting challenge, we present a robust scanning fluorescence endoscope. A spatial light modulator shapes the input excitation wavefront to focus light on the distal tip of the fiber and to rapidly scan the focus over the region of interest. A detector array collects the fluorescence emission propagated back from the sample to the proximal tip of the fiber. We demonstrate that proper selection of the multimode fiber is critical for a robust calibration and for high signal-to-background ratio performance. We compare different types of multimode fibers and experimentally show that a focus created through a graded-index fiber can withstand a few millimeters of fiber distal tip translation. The resulting scanning endoscopic microscope images fluorescent samples over a field of view of 80µm with a resolution of 2µm.
| S-EPMC5772399 | biostudies-literature
Project description:It is critical, but difficult, to catch the small variation in genomic or other kinds of data that differentiates phenotypes or categories. A plethora of data is available, but the information from its genes or elements is spread over arbitrarily, making it challenging to extract relevant details for identification. However, an arrangement of similar genes into clusters makes these differences more accessible and allows for robust identification of hidden mechanisms (e.g. pathways) than dealing with elements individually. Here we propose, DeepInsight, which converts non-image samples into a well-organized image-form. Thereby, the power of convolution neural network (CNN), including GPU utilization, can be realized for non-image samples. Furthermore, DeepInsight enables feature extraction through the application of CNN for non-image samples to seize imperative information and shown promising results. To our knowledge, this is the first work to apply CNN simultaneously on different kinds of non-image datasets: RNA-seq, vowels, text, and artificial.
| S-EPMC6684600 | biostudies-literature
Project description:Optical coherence tomography (OCT) is a widely used non-invasive biomedical imaging modality that can rapidly provide volumetric images of samples. Here, we present a deep learning-based image reconstruction framework that can generate swept-source OCT (SS-OCT) images using undersampled spectral data, without any spatial aliasing artifacts. This neural network-based image reconstruction does not require any hardware changes to the optical setup and can be easily integrated with existing swept-source or spectral-domain OCT systems to reduce the amount of raw spectral data to be acquired. To show the efficacy of this framework, we trained and blindly tested a deep neural network using mouse embryo samples imaged by an SS-OCT system. Using 2-fold undersampled spectral data (i.e., 640 spectral points per A-line), the trained neural network can blindly reconstruct 512 A-lines in 0.59 ms using multiple graphics-processing units (GPUs), removing spatial aliasing artifacts due to spectral undersampling, also presenting a very good match to the images of the same samples, reconstructed using the full spectral OCT data (i.e., 1280 spectral points per A-line). We also successfully demonstrate that this framework can be further extended to process 3× undersampled spectral data per A-line, with some performance degradation in the reconstructed image quality compared to 2× spectral undersampling. Furthermore, an A-line-optimized undersampling method is presented by jointly optimizing the spectral sampling locations and the corresponding image reconstruction network, which improved the overall imaging performance using less spectral data points per A-line compared to 2× or 3× spectral undersampling results. This deep learning-enabled image reconstruction approach can be broadly used in various forms of spectral-domain OCT systems, helping to increase their imaging speed without sacrificing image resolution and signal-to-noise ratio.
| S-EPMC8322159 | biostudies-literature
Project description:In recent years, the transformative potential of deep neural networks (DNNs) for analysing and interpreting NMR data has clearly been recognised. However, most applications of DNNs in NMR to date either struggle to outperform existing methodologies or are limited in scope to a narrow range of data that closely resemble the data that the network was trained on. These limitations have prevented a widescale uptake of DNNs in NMR. Addressing this, we introduce FID-Net, a deep neural network architecture inspired by WaveNet, for performing analyses on time domain NMR data. We first demonstrate the effectiveness of this architecture in reconstructing non-uniformly sampled (NUS) biomolecular NMR spectra. It is shown that a single network is able to reconstruct a diverse range of 2D NUS spectra that have been obtained with arbitrary sampling schedules, with a range of sweep widths, and a variety of other acquisition parameters. The performance of the trained FID-Net in this case exceeds or matches existing methods currently used for the reconstruction of NUS NMR spectra. Secondly, we present a network based on the FID-Net architecture that can efficiently virtually decouple 13Cα-13Cβ couplings in HNCA protein NMR spectra in a single shot analysis, while at the same time leaving glycine residues unmodulated. The ability for these DNNs to work effectively in a wide range of scenarios, without retraining, paves the way for their widespread usage in analysing NMR data.
| S-EPMC8131344 | biostudies-literature
Project description:The energy resolution in hyperspectral imaging techniques has always been an important matter in data interpretation. In many cases, spectral information is distorted by elements such as instruments' broad optical transfer function, and electronic high frequency noises. In the past decades, advances in artificial intelligence methods have provided robust tools to better study sophisticated system artifacts in spectral data and take steps towards removing these artifacts from the experimentally obtained data. This study evaluates the capability of a recently developed deep convolutional neural network script, EELSpecNet, in restoring the reality of a spectral data. The particular strength of the deep neural networks is to remove multiple instrumental artifacts such as random energy jitters of the source, signal convolution by the optical transfer function and high frequency noise at once using a single training data set. Here, EELSpecNet performance in reducing noise, and restoring the original reality of the spectra is evaluated for near zero-loss electron energy loss spectroscopy signals in Scanning Transmission Electron Microscopy. EELSpecNet demonstrates to be more efficient and more robust than the currently widely used Bayesian statistical method, even in harsh conditions (e.g. high signal broadening, intense high frequency noise).
| S-EPMC9581942 | biostudies-literature
Project description:SIGNIFICANCE:Noninvasive in vivo fast pulsatile blood flow measurement in deep tissue is important because the blood flow waveform is correlated with physiological parameters, such as blood pressure and elasticity of blood vessels. Compromised blood flow may cause diseases, such as stroke, foot ulcer, and myocardial ischemia. There is great clinical demand for a portable and cost-effective device for noninvasive pulsatile blood flow measurement. AIM:A diffuse-optics-based method, diffuse speckle pulsatile flowmetry (DSPF), was developed for fast measurement (?300??Hz) of deep tissue blood flow noninvasively. To validate its performance, both a phantom experiment and in vivo demonstration were conducted. APPROACH:Over the past two decades, single-mode fibers have been used as detection fibers in most diffuse-optics-based deep tissue blood flow measurement modalities. We used a multimode (MM) detection fiber with a core size of 200???m for diffused speckle pattern detection. A background intensity correction algorithm was implemented for speckle contrast calculation. The MM detection fiber helped to achieve a level of deep tissue blood flow measurement similar to that of conventional modalities, such as diffuse correlation spectroscopy and diffuse speckle contrast analysis, but it increases the measurement rate of blood flow to 300 Hz. RESULTS:The design and implementation of the DSPF system were introduced. The theory of the background intensity correction for the diffused speckle pattern detected by the MM fiber was explained. A flow phantom was built for validation of the performance of the DSPF system. An in vivo cuff-induced occlusion experiment was performed to demonstrate the capability of the proposed DSPF system. CONCLUSIONS:An MM detection fiber can help to achieve fast (?300??Hz) pulsatile blood flow measurement in the proposed DSPF method. The cost-effective device and the fiber-based flexible probe increase the usability of the DSPF system significantly.
| S-EPMC7219964 | biostudies-literature
Project description:Neuromorphic photonics has recently emerged as a promising hardware accelerator, with significant potential speed and energy advantages over digital electronics for machine learning algorithms, such as neural networks of various types. Integrated photonic networks are particularly powerful in performing analog computing of matrix-vector multiplication (MVM) as they afford unparalleled speed and bandwidth density for data transmission. Incorporating nonvolatile phase-change materials in integrated photonic devices enables indispensable programming and in-memory computing capabilities for on-chip optical computing. Here, we demonstrate a multimode photonic computing core consisting of an array of programable mode converters based on on-waveguide metasurfaces made of phase-change materials. The programmable converters utilize the refractive index change of the phase-change material Ge2Sb2Te5 during phase transition to control the waveguide spatial modes with a very high precision of up to 64 levels in modal contrast. This contrast is used to represent the matrix elements, with 6-bit resolution and both positive and negative values, to perform MVM computation in neural network algorithms. We demonstrate a prototypical optical convolutional neural network that can perform image processing and recognition tasks with high accuracy. With a broad operation bandwidth and a compact device footprint, the demonstrated multimode photonic core is promising toward large-scale photonic neural networks with ultrahigh computation throughputs.
| S-EPMC7782756 | biostudies-literature