Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning.
Ontology highlight
ABSTRACT:
SUBMITTER: Wang Z
PROVIDER: S-EPMC8107123 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Fast signal processing and real-time displays are essential for practical imaging modality in various fields of applications. However, the imaging speed in optical-resolution photoacoustic microscopy (OR-PAM), in particular, depends on factors such as the pulse repetition rate of the laser, scanning method, field of view (FOV), and signal processing time. In the past, efforts to increase acquisition speed either focused on developing new scanning methods or using lasers with higher pulse repetition rates. However, high-speed signal processing is also important for real-time volumetric display in OR-PAM. In this study, we carried out parallel signal processing using a graphics processing unit (GPU) to enable fast signal processing and wide-field real-time displays in laser-scanning OR-PAM. The average total GPU processing time for a B-mode PAM image was approximately 1.35 ms at a display speed of 480 fps when the data samples were acquired with 736 (axial) × 500 (lateral) points/B-mode-frame at a pulse repetition rate of 300 kHz. In addition, we successfully displayed maximum amplitude projection images of a mouse's ear as volumetric images with an FOV of 3 mm × 3 mm (500 × 500 pixels) at 1.02 s, corresponding to 0.98 fps.
| S-EPMC4679244 | biostudies-other
Project description:We explain how to concentrate light simultaneously at multiple selected volumetric positions by means of a 4D illumination light field. First, to select target objects, a 4D imaging light field is captured. A light field mask is then computed automatically for this selection to avoid illumination of the remaining areas. With one-photon illumination, simultaneous generation of complex volumetric light patterns becomes possible. As a full light-field can be captured and projected simultaneously at the desired exposure and excitation times, short readout and lighting durations are supported.
| S-EPMC4929678 | biostudies-literature
Project description:Visualizing diverse anatomical and functional traits that span many spatial scales with high spatio-temporal resolution provides insights into the fundamentals of living organisms. Light-field microscopy (LFM) has recently emerged as a scanning-free, scalable method that allows for high-speed, volumetric functional brain imaging. Given those promising applications at the tissue level, at its other extreme, this highly-scalable approach holds great potential for observing structures and dynamics in single-cell specimens. However, the challenge remains for current LFM to achieve a subcellular level, near-diffraction-limited 3D spatial resolution. Here, we report high-resolution LFM (HR-LFM) for live-cell imaging with a resolution of 300-700 nm in all three dimensions, an imaging depth of several micrometers, and a volume acquisition time of milliseconds. We demonstrate the technique by imaging various cellular dynamics and structures and tracking single particles. The method may advance LFM as a particularly useful tool for understanding biological systems at multiple spatio-temporal levels.
| S-EPMC6363205 | biostudies-other
Project description:Volumetric interrogation of the organization and processes of intracellular organelles and molecules in cellular systems with a high spatiotemporal resolution is essential for understanding cell physiology, development, and pathology. Here, we report high-resolution Fourier light-field microscopy (HR-FLFM) for fast and volumetric live-cell imaging. HR-FLFM transforms conventional cell microscopy and enables exploration of less accessible spatiotemporal-limiting regimes for single-cell studies. The results present a near-diffraction-limited resolution in all three dimensions, a five-fold extended focal depth to several micrometers, and a scanning-free volume acquisition time up to milliseconds. The system demonstrates instrumentation accessibility, low photo damage for continuous observation, and high compatibility with general cell assays. We anticipate HR-FLFM to offer a promising methodological pathway for investigating a wide range of intracellular processes and functions with exquisite spatiotemporal contextual details.
| S-EPMC8318351 | biostudies-literature
Project description:Synthetic phased array (SPA) beamforming with Hadamard coding and aperture weighting is an optimal option for real-time volumetric imaging with a ring array, a particularly attractive geometry in intracardiac and intravascular applications. However, the imaging frame rate of this method is limited by the immense computational load required in synthetic beamforming. For fast imaging with a ring array, we developed graphics processing unit (GPU)-based, real-time image reconstruction software that exploits massive data-level parallelism in beamforming operations. The GPU-based software reconstructs and displays three cross-sectional images at 45 frames per second (fps). This frame rate is 4.5 times higher than that for our previously-developed multi-core CPU-based software. In an alternative imaging mode, it shows one B-mode image rotating about the axis and its maximum intensity projection, processed at a rate of 104 fps . This paper describes the image reconstruction procedure on the GPU platform and presents the experimental images obtained using this software.
| S-EPMC4019434 | biostudies-literature
Project description:We apply wide-field fluorescence microscopy to measure real-time attachment of photosynthetic proteins to plasmonically active silver nanowires. The observation of this effect is enabled, on the one hand, by sensitive detection of fluorescence and, on the other hand, by plasmonic enhancement of protein fluorescence. We examined two sample configurations with substrates being a bare glass coverslip and a coverslip functionalized with a monolayer of streptavidin. The different preparation of the substrate changes the observed behavior as far as attachment of the protein is concerned as well as its subsequent photobleaching. For the latter substrate the conjugation process is measurably slower. The described method can be universally applied in studying protein-nanostructure interactions for real-time fluorescence-based sensing.
| S-EPMC5796375 | biostudies-literature
Project description:We explain how volumetric light-field excitation can be converted to a process that entirely avoids 3D reconstruction, deconvolution, and calibration of optical elements while taking scattering in the probe better into account. For spatially static probes, this is achieved by an efficient (one-time) light-transport sampling and light-field factorization. Individual probe particles (and arbitrary combinations thereof) can subsequently be excited in a dynamically controlled way while still supporting volumetric reconstruction of the entire probe in real-time based on a single light-field recording.
| S-EPMC5656577 | biostudies-literature
Project description:Single-pixel cameras capture images without the requirement for a multi-pixel sensor, enabling the use of state-of-the-art detector technologies and providing a potentially low-cost solution for sensing beyond the visible spectrum. One limitation of single-pixel cameras is the inherent trade-off between image resolution and frame rate, with current compressive (compressed) sensing techniques being unable to support real-time video. In this work we demonstrate the application of deep learning with convolutional auto-encoder networks to recover real-time 128?×?128 pixel video at 30 frames-per-second from a single-pixel camera sampling at a compression ratio of 2%. In addition, by training the network on a large database of images we are able to optimise the first layer of the convolutional network, equivalent to optimising the basis used for scanning the image intensities. This work develops and implements a novel approach to solving the inverse problem for single-pixel cameras efficiently and represents a significant step towards real-time operation of computational imagers. By learning from examples in a particular context, our approach opens up the possibility of high resolution for task-specific adaptation, with importance for applications in gas sensing, 3D imaging and metrology.
| S-EPMC5799195 | biostudies-literature
Project description:Spatial light interference microscopy (SLIM) is a highly sensitive quantitative phase imaging method, which is capable of unprecedented structure studies in biology and beyond. In addition to the ?/2 shift introduced in phase contrast between the scattered and unscattered light from the sample, 4 phase shifts are generated in SLIM, by increments of ?/2 using a reflective liquid crystal phase modulator (LCPM). As 4 phase shifted images are required to produce a quantitative phase image, the switching speed of the LCPM and the acquisition rate of the camera limit the acquisition rate and, thus, SLIM's applicability to highly dynamic samples. In this paper we present a fast SLIM setup which can image at a maximum rate of 50 frames per second and provide in real-time quantitative phase images at 50/4?=?12.5 frames per second. We use a fast LCPM for phase shifting and a fast scientific-grade complementary metal oxide semiconductor (sCMOS) camera (Andor) for imaging. We present the dispersion relation, i.e. decay rate vs. spatial mode, associated with dynamic beating cardiomyocyte cells from the quantitative phase images obtained with the real-time SLIM system.
| S-EPMC3574023 | biostudies-literature
Project description:Super-resolved structured illumination microscopy (SR-SIM) is among the fastest fluorescence microscopy techniques capable of surpassing the optical diffraction limit. Current custom-build instruments are able to deliver two-fold resolution enhancement with high acquisition speed. SR-SIM is usually a two-step process, with raw-data acquisition and subsequent, time-consuming post-processing for image reconstruction. In contrast, wide-field and (multi-spot) confocal techniques produce high-resolution images instantly. Such immediacy is also possible with SR-SIM, by tight integration of a video-rate capable SIM with fast reconstruction software. Here we present instant SR-SIM by VIGOR (Video-rate Immediate GPU-accelerated Open-Source Reconstruction). We demonstrate multi-color SR-SIM at video frame-rates, with less than 250 ms delay between measurement and reconstructed image display. This is achieved by modifying and extending high-speed SR-SIM image acquisition with a new, GPU-enhanced, network-enabled image-reconstruction software. We demonstrate high-speed surveying of biological samples in multiple colors and live imaging of moving mitochondria as an example of intracellular dynamics.
| S-EPMC6754501 | biostudies-literature