Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning.
Ontology highlight
ABSTRACT:
SUBMITTER: Wang Z
PROVIDER: S-EPMC8107123 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Here, we present a step-by-step protocol for the implementation of deep-learning-enhanced light-field microscopy enabling 3D imaging of instantaneous biological processes. We first provide the instructions to build a light-field microscope (LFM) capable of capturing optically encoded dynamic signals. Then, we detail the data processing and model training of a view-channel-depth (VCD) neural network, which enables instant 3D image reconstruction from a single 2D light-field snapshot. Finally, we describe VCD-LFM imaging of several model organisms and demonstrate image-based quantitative studies on neural activities and cardio-hemodynamics. For complete details on the use and execution of this protocol, please refer to Wang et al. (2021).1.
| S-EPMC9898296 | biostudies-literature
Project description:The generation of tailored complex light fields with multi-core fiber (MCF) lensless microendoscopes is widely used in biomedicine. However, the computer-generated holograms (CGHs) used for such applications are typically generated by iterative algorithms, which demand high computation effort, limiting advanced applications like fiber-optic cell manipulation. The random and discrete distribution of the fiber cores in an MCF induces strong spatial aliasing to the CGHs, hence, an approach that can rapidly generate tailored CGHs for MCFs is highly demanded. We demonstrate a novel deep neural network-CoreNet, providing accurate tailored CGHs generation for MCFs at a near video rate. The CoreNet is trained by unsupervised learning and speeds up the computation time by two magnitudes with high fidelity light field generation compared to the previously reported CGH algorithms for MCFs. Real-time generated tailored CGHs are on-the-fly loaded to the phase-only spatial light modulator (SLM) for near video-rate complex light fields generation through the MCF microendoscope. This paves the avenue for real-time cell rotation and several further applications that require real-time high-fidelity light delivery in biomedicine.
| S-EPMC9095618 | biostudies-literature
Project description:Fast signal processing and real-time displays are essential for practical imaging modality in various fields of applications. However, the imaging speed in optical-resolution photoacoustic microscopy (OR-PAM), in particular, depends on factors such as the pulse repetition rate of the laser, scanning method, field of view (FOV), and signal processing time. In the past, efforts to increase acquisition speed either focused on developing new scanning methods or using lasers with higher pulse repetition rates. However, high-speed signal processing is also important for real-time volumetric display in OR-PAM. In this study, we carried out parallel signal processing using a graphics processing unit (GPU) to enable fast signal processing and wide-field real-time displays in laser-scanning OR-PAM. The average total GPU processing time for a B-mode PAM image was approximately 1.35 ms at a display speed of 480 fps when the data samples were acquired with 736 (axial) × 500 (lateral) points/B-mode-frame at a pulse repetition rate of 300 kHz. In addition, we successfully displayed maximum amplitude projection images of a mouse's ear as volumetric images with an FOV of 3 mm × 3 mm (500 × 500 pixels) at 1.02 s, corresponding to 0.98 fps.
| S-EPMC4679244 | biostudies-other
Project description:High-speed three-dimensional (3D) intravital imaging in animals is useful for studying transient subcellular interactions and functions in health and disease. Light-field microscopy (LFM) provides a computational solution for snapshot 3D imaging with low phototoxicity but is restricted by low resolution and reconstruction artifacts induced by optical aberrations, motion and noise. Here, we propose virtual-scanning LFM (VsLFM), a physics-based deep learning framework to increase the resolution of LFM up to the diffraction limit within a snapshot. By constructing a 40 GB high-resolution scanning LFM dataset across different species, we exploit physical priors between phase-correlated angular views to address the frequency aliasing problem. This enables us to bypass hardware scanning and associated motion artifacts. Here, we show that VsLFM achieves ultrafast 3D imaging of diverse processes such as the beating heart in embryonic zebrafish, voltage activity in Drosophila brains and neutrophil migration in the mouse liver at up to 500 volumes per second.
| S-EPMC10172145 | biostudies-literature
Project description:The forces exerted by single cells in the three-dimensional (3D) environments play a crucial role in modulating cellular functions and behaviors closely related to physiological and pathological processes. Cellular force microscopy (CFM) provides a feasible solution for quantifying mechanical interactions, which usually regains cellular forces from deformation information of extracellular matrices embedded with fluorescent beads. Owing to computational complexity, traditional 3D-CFM is usually extremely time consuming, which makes it challenging for efficient force recovery and large-scale sample analysis. With the aid of deep neural networks, this study puts forward a novel, data-driven 3D-CFM to reconstruct 3D cellular force fields directly from volumetric images with random fluorescence patterns. The deep-learning-based network is established through stacking deep convolutional neural networks (DCNN) and specific function layers. Some necessary physical information associated with constitutive relation of extracellular matrix material is coupled to the data-driven network. The mini-batch stochastic-gradient-descent and back-propagation algorithms are introduced to ensure its convergence and training efficiency. The networks not only have good generalization ability and robustness but also can recover 3D cellular forces directly from the input fluorescence image pairs. Particularly, the computational efficiency of the deep-learning-based network is at least one to two orders of magnitude higher than that of traditional 3D-CFM. This study provides a novel scheme for developing high-performance 3D-CFM to quantitatively characterize mechanical interactions between single cells and surrounding extracellular matrices, which is of vital importance for quantitative investigations in biomechanics and mechanobiology.
| S-EPMC9247343 | biostudies-literature
Project description:We explain how to concentrate light simultaneously at multiple selected volumetric positions by means of a 4D illumination light field. First, to select target objects, a 4D imaging light field is captured. A light field mask is then computed automatically for this selection to avoid illumination of the remaining areas. With one-photon illumination, simultaneous generation of complex volumetric light patterns becomes possible. As a full light-field can be captured and projected simultaneously at the desired exposure and excitation times, short readout and lighting durations are supported.
| S-EPMC4929678 | biostudies-literature
Project description:Volumetric interrogation of the organization and processes of intracellular organelles and molecules in cellular systems with a high spatiotemporal resolution is essential for understanding cell physiology, development, and pathology. Here, we report high-resolution Fourier light-field microscopy (HR-FLFM) for fast and volumetric live-cell imaging. HR-FLFM transforms conventional cell microscopy and enables exploration of less accessible spatiotemporal-limiting regimes for single-cell studies. The results present a near-diffraction-limited resolution in all three dimensions, a five-fold extended focal depth to several micrometers, and a scanning-free volume acquisition time up to milliseconds. The system demonstrates instrumentation accessibility, low photo damage for continuous observation, and high compatibility with general cell assays. We anticipate HR-FLFM to offer a promising methodological pathway for investigating a wide range of intracellular processes and functions with exquisite spatiotemporal contextual details.
| S-EPMC8318351 | biostudies-literature
Project description:Visualizing diverse anatomical and functional traits that span many spatial scales with high spatio-temporal resolution provides insights into the fundamentals of living organisms. Light-field microscopy (LFM) has recently emerged as a scanning-free, scalable method that allows for high-speed, volumetric functional brain imaging. Given those promising applications at the tissue level, at its other extreme, this highly-scalable approach holds great potential for observing structures and dynamics in single-cell specimens. However, the challenge remains for current LFM to achieve a subcellular level, near-diffraction-limited 3D spatial resolution. Here, we report high-resolution LFM (HR-LFM) for live-cell imaging with a resolution of 300-700 nm in all three dimensions, an imaging depth of several micrometers, and a volume acquisition time of milliseconds. We demonstrate the technique by imaging various cellular dynamics and structures and tracking single particles. The method may advance LFM as a particularly useful tool for understanding biological systems at multiple spatio-temporal levels.
| S-EPMC6363205 | biostudies-other
Project description:Synthetic phased array (SPA) beamforming with Hadamard coding and aperture weighting is an optimal option for real-time volumetric imaging with a ring array, a particularly attractive geometry in intracardiac and intravascular applications. However, the imaging frame rate of this method is limited by the immense computational load required in synthetic beamforming. For fast imaging with a ring array, we developed graphics processing unit (GPU)-based, real-time image reconstruction software that exploits massive data-level parallelism in beamforming operations. The GPU-based software reconstructs and displays three cross-sectional images at 45 frames per second (fps). This frame rate is 4.5 times higher than that for our previously-developed multi-core CPU-based software. In an alternative imaging mode, it shows one B-mode image rotating about the axis and its maximum intensity projection, processed at a rate of 104 fps . This paper describes the image reconstruction procedure on the GPU platform and presents the experimental images obtained using this software.
| S-EPMC4019434 | biostudies-literature
Project description:Volumetric imaging by fluorescence microscopy is often limited by anisotropic spatial resolution, in which the axial resolution is inferior to the lateral resolution. To address this problem, we present a deep-learning-enabled unsupervised super-resolution technique that enhances anisotropic images in volumetric fluorescence microscopy. In contrast to the existing deep learning approaches that require matched high-resolution target images, our method greatly reduces the effort to be put into practice as the training of a network requires only a single 3D image stack, without a priori knowledge of the image formation process, registration of training data, or separate acquisition of target data. This is achieved based on the optimal transport-driven cycle-consistent generative adversarial network that learns from an unpaired matching between high-resolution 2D images in the lateral image plane and low-resolution 2D images in other planes. Using fluorescence confocal microscopy and light-sheet microscopy, we demonstrate that the trained network not only enhances axial resolution but also restores suppressed visual details between the imaging planes and removes imaging artifacts.
| S-EPMC9178036 | biostudies-literature