Dataset Information

High-resolution single-photon imaging with physics-informed deep learning.

ABSTRACT: High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depths, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this physical noise model, we synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depths, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique in a series of experiments including microfluidic inspection, Fourier ptychography, and high-speed imaging. The experiments validate the technique's state-of-the-art super-resolution SPAD imaging performance.

SUBMITTER: Bian L

PROVIDER: S-EPMC10516985 | biostudies-literature | 2023 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

High-resolution single-photon imaging with physics-informed deep learning.

Bian Liheng L Song Haoze H Peng Lintao L Chang Xuyang X Yang Xi X Horstmeyer Roarke R Ye Lin L Zhu Chunli C Qin Tong T Zheng Dezhi D Zhang Jun J

Nature communications 20230922 1

High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depths, 3 d ...[more]

PMID: 37737270

Similar Datasets

Project description:PurposeCurrent methods for patient-specific voxel-level dosimetry in radionuclide therapy suffer from a trade-off between accuracy and computational efficiency. Monte Carlo (MC) radiation transport algorithms are considered the gold standard for voxel-level dosimetry but can be computationally expensive, whereas faster dose voxel kernel (DVK) convolution can be suboptimal in the presence of tissue heterogeneities. Furthermore, the accuracies of both these methods are limited by the spatial resolution of the reconstructed emission image. To overcome these limitations, this paper considers a single deep convolutional neural network (CNN) with residual learning (named DblurDoseNet) that learns to produce dose-rate maps while compensating for the limited resolution of SPECT images.MethodsWe trained our CNN using MC-generated dose-rate maps that directly corresponded to the true activity maps in virtual patient phantoms. Residual learning was applied such that our CNN learned only the difference between the true dose-rate map and DVK dose-rate map with density scaling. Our CNN consists of a 3D depth feature extractor followed by a 2D U-Net, where the input was 11 slices (3.3 cm) of a given Lu-177 SPECT/CT image and density map, and the output was the dose-rate map corresponding to the center slice. The CNN was trained with nine virtual patient phantoms and tested on five different phantoms plus 42 SPECT/CT scans of patients who underwent Lu-177 DOTATATE therapy.ResultsWhen testing on virtual patient phantoms, the lesion/organ mean dose-rate error and the normalized root mean square error (NRMSE) relative to the ground truth of the CNN method was consistently lower than DVK and MC, when applied to SPECT images. Compared to DVK/MC, the average improvement for the CNN in mean dose-rate error was 55%/53% and 66%/56%; and in NRMSE was 18%/17% and 10%/11% for lesion and kidney regions, respectively. Line profiles and dose-volume histograms demonstrated compensation for SPECT resolution effects in the CNN-generated dose-rate maps. The ensemble noise standard deviation, determined from multiple Poisson realizations, was improved by 21%/27% compared to DVK/MC. In patients, potential improvements from CNN dose-rate maps compared to DVK/MC were illustrated qualitatively, due to the absence of ground truth. The trained residual CNN took about 30 s on a single GPU (Tesla V100) to generate a 512 × $\; \times \;$ 512 × $\; \times \;$ 130 dose-rate map for a patient.ConclusionThe proposed residual CNN, trained using phantoms generated from patient images, has potential for real-time patient-specific dosimetry in clinical treatment planning due to its demonstrated improvement in accuracy, resolution, noise, and speed over the DVK/MC approaches.

Dataset Information

High-resolution single-photon imaging with physics-informed deep learning.

Publications

High-resolution single-photon imaging with physics-informed deep learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets