Gap Reconstruction in Optical Motion Capture Sequences Using Neural Networks.
Ontology highlight
ABSTRACT:
SUBMITTER: Skurowski P
PROVIDER: S-EPMC8472986 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The current study aimed to assess the repeatability and validity of cervical range of motion (CROM) measurements using an optical motion capture system (OMCS), compared with a CROM device. A total of 20 healthy volunteers were selected and enrolled in the current study after informed consent was received. The motion of the cervical spine in all directions was measured using the OMCS and CROM devices. Reproducibility of data was assessed using the intra-group correlation coefficient (ICC), standard error of measurement (SEM) and minimum detectable change (MDC). Validity was assessed using the coefficient of determination (R2) in combination with Pearson's correlation coefficient. Bland-Altman plot were presented for the two measurement methods. The range of motion (ROM) was measured by using the OMCS and the CROM device during the same session. Both procedures evidenced high ICCs [OMCS: ICC (1,2) =0.802-0.981; CROM device: ICC (1,2) =0.768-0.948], low SEM values (OMCS: 0.98°-1.38°; CROM device: 1.04°-2.45°) and low MDC values (OMCS: 2.72°-3.81°; CROM device: 2.89°-6.78°). A high R2 (0.568-0.882) and Pearson's correlation coefficient (0.753-0.939) were determined. The Bland-Altman plots demonstrated that most of the data were within the 95% consistency limit. In summary, the OMCS has good repeatability and validity when measuring CROM and is an effective way to evaluate cervical vertebral range of motion.
| S-EPMC6862302 | biostudies-literature
Project description:Marker-based Optical Motion Capture (OMC) systems and associated musculoskeletal (MSK) modelling predictions offer non-invasively obtainable insights into muscle and joint loading at an in vivo level, aiding clinical decision-making. However, an OMC system is lab-based, expensive, and requires a line of sight. Inertial Motion Capture (IMC) techniques are widely-used alternatives, which are portable, user-friendly, and relatively low-cost, although with lesser accuracy. Irrespective of the choice of motion capture technique, one typically uses an MSK model to obtain the kinematic and kinetic outputs, which is a computationally expensive tool increasingly well approximated by machine learning (ML) methods. Here, an ML approach is presented that maps experimentally recorded IMC input data to the human upper-extremity MSK model outputs computed from ('gold standard') OMC input data. Essentially, this proof-of-concept study aims to predict higher-quality MSK outputs from the much easier-to-obtain IMC data. We use OMC and IMC data simultaneously collected for the same subjects to train different ML architectures that predict OMC-driven MSK outputs from IMC measurements. In particular, we employed various neural network (NN) architectures, such as Feed-Forward Neural Networks (FFNNs) and Recurrent Neural Networks (RNNs) (vanilla, Long Short-Term Memory, and Gated Recurrent Unit) and a comprehensive search for the best-fit model in the hyperparameters space in both subject-exposed (SE) as well as subject-naive (SN) settings. We observed a comparable performance for both FFNN and RNN models, which have a high degree of agreement (ravg,SE,FFNN=0.90±0.19, ravg,SE,RNN=0.89±0.17, ravg,SN,FFNN=0.84±0.23, and ravg,SN,RNN=0.78±0.23) with the desired OMC-driven MSK estimates for held-out test data. The findings demonstrate that mapping IMC inputs to OMC-driven MSK outputs using ML models could be instrumental in transitioning MSK modelling from 'lab to field'.
| S-EPMC10215337 | biostudies-literature
Project description:In this paper, a marker-based, single-person optical motion capture method (DeepMoCap) is proposed using multiple spatio-temporally aligned infrared-depth sensors and retro-reflective straps and patches (reflectors). DeepMoCap explores motion capture by automatically localizing and labeling reflectors on depth images and, subsequently, on 3D space. Introducing a non-parametric representation to encode the temporal correlation among pairs of colorized depthmaps and 3D optical flow frames, a multi-stage Fully Convolutional Network (FCN) architecture is proposed to jointly learn reflector locations and their temporal dependency among sequential frames. The extracted reflector 2D locations are spatially mapped in 3D space, resulting in robust 3D optical data extraction. The subject's motion is efficiently captured by applying a template-based fitting technique on the extracted optical data. Two datasets have been created and made publicly available for evaluation purposes; one comprising multi-view depth and 3D optical flow annotated images (DMC2.5D), and a second, consisting of spatio-temporally aligned multi-view depth images along with skeleton, inertial and ground truth MoCap data (DMC3D). The FCN model outperforms its competitors on the DMC2.5D dataset using 2D Percentage of Correct Keypoints (PCK) metric, while the motion capture outcome is evaluated against RGB-D and inertial data fusion approaches on DMC3D, outperforming the next best method by 4 . 5 % in total 3D PCK accuracy.
| S-EPMC6359336 | biostudies-literature
Project description:Optical motion capture (OMC) systems are commonly used to capture in-vivo three-dimensional joint kinematics. However, the skin-based markers may not reflect the underlying bone movement, a source of error known as soft tissue artifact (STA). This study examined STA during wrist motion by evaluating the agreement between OMC and biplanar videoradiography (BVR). Nine subjects completed 7 different wrist motion tasks: doorknob rotation to capture supination and pronation, radial-ulnar deviation, flexion-extension, circumduction, hammering, and pitcher pouring. BVR and OMC captured the motion simultaneously. Wrist kinematics were quantified using helical motion parameters of rotation and translation, and Bland-Altman analysis quantified the mean difference (bias) and 95% limit of agreement (LOA). The rotational bias of doorknob pronation, a median bias of -4.9°, was significantly larger than the flexion-extension (0.7°, p < 0.05) and radial-ulnar deviation (1.8°, p < 0.01) tasks. The rotational LOA range was significantly smaller in the flexion-extension task (5.9°) compared to pitcher (11.6°, p < 0.05) and doorknob pronation (17.9°, p < 0.05) tasks. The translation bias did not differ between tasks. The translation LOA range was significantly larger in circumduction (9.8°) compared to the radial-ulnar deviation (6.3°, p < 0.05) and pitcher (3.4°, p < 0.05) tasks. While OMC technology has a wide-range of successful applications, we demonstrated it has relatively poor agreement with BVR in tracking wrist motion, and that the agreement depends on the nature and direction of wrist motion.
| S-EPMC8089049 | biostudies-literature
Project description:Motion capture of unrestrained moving animals is a major analytic tool in neuroethology and behavioral physiology. At present, several motion capture methodologies have been developed, all of which have particular limitations regarding experimental application. Whereas marker-based motion capture systems are very robust and easily adjusted to suit different setups, tracked species, or body parts, they cannot be applied in experimental situations where markers obstruct the natural behavior (e.g., when tracking delicate, elastic, and/or sensitive body structures). On the other hand, marker-less motion capture systems typically require setup- and animal-specific adjustments, for example by means of tailored image processing, decision heuristics, and/or machine learning of specific sample data. Among the latter, deep-learning approaches have become very popular because of their applicability to virtually any sample of video data. Nevertheless, concise evaluation of their training requirements has rarely been done, particularly with regard to the transfer of trained networks from one application to another. To address this issue, the present study uses insect locomotion as a showcase example for systematic evaluation of variation and augmentation of the training data. For that, we use artificially generated video sequences with known combinations of observed, real animal postures and randomized body position, orientation, and size. Moreover, we evaluate the generalization ability of networks that have been pre-trained on synthetic videos to video recordings of real walking insects, and estimate the benefit in terms of reduced requirement for manual annotation. We show that tracking performance is affected only little by scaling factors ranging from 0.5 to 1.5. As expected from convolutional networks, the translation of the animal has no effect. On the other hand, we show that sufficient variation of rotation in the training data is essential for performance, and make concise suggestions about how much variation is required. Our results on transfer from synthetic to real videos show that pre-training reduces the amount of necessary manual annotation by about 50%.
| S-EPMC8100444 | biostudies-literature
Project description:Cryo-electron microscopy (cryo-EM) single-particle analysis has proven powerful in determining the structures of rigid macromolecules. However, many imaged protein complexes exhibit conformational and compositional heterogeneity that poses a major challenge to existing three-dimensional reconstruction methods. Here, we present cryoDRGN, an algorithm that leverages the representation power of deep neural networks to directly reconstruct continuous distributions of 3D density maps and map per-particle heterogeneity of single-particle cryo-EM datasets. Using cryoDRGN, we uncovered residual heterogeneity in high-resolution datasets of the 80S ribosome and the RAG complex, revealed a new structural state of the assembling 50S ribosome, and visualized large-scale continuous motions of a spliceosome complex. CryoDRGN contains interactive tools to visualize a dataset's distribution of per-particle variability, generate density maps for exploratory analysis, extract particle subsets for use with other tools and generate trajectories to visualize molecular motions. CryoDRGN is open-source software freely available at http://cryodrgn.csail.mit.edu .
| S-EPMC8183613 | biostudies-literature
Project description:Diffractive optical networks unify wave optics and deep learning to all-optically compute a given machine learning or computational imaging task as the light propagates from the input to the output plane. Here, we report the design of diffractive optical networks for the classification and reconstruction of spatially overlapping, phase-encoded objects. When two different phase-only objects spatially overlap, the individual object functions are perturbed since their phase patterns are summed up. The retrieval of the underlying phase images from solely the overlapping phase distribution presents a challenging problem, the solution of which is generally not unique. We show that through a task-specific training process, passive diffractive optical networks composed of successive transmissive layers can all-optically and simultaneously classify two different randomly-selected, spatially overlapping phase images at the input. After trained with ~ 550 million unique combinations of phase-encoded handwritten digits from the MNIST dataset, our blind testing results reveal that the diffractive optical network achieves an accuracy of > 85.8% for all-optical classification of two overlapping phase images of new handwritten digits. In addition to all-optical classification of overlapping phase objects, we also demonstrate the reconstruction of these phase images based on a shallow electronic neural network that uses the highly compressed output of the diffractive optical network as its input (with e.g., ~ 20-65 times less number of pixels) to rapidly reconstruct both of the phase images, despite their spatial overlap and related phase ambiguity. The presented phase image classification and reconstruction framework might find applications in e.g., computational imaging, microscopy and quantitative phase imaging fields.
| S-EPMC9120207 | biostudies-literature
Project description: Not available
| S-EPMC10601046 | biostudies-literature
Project description:Optical motion capture systems are state-of-the-art in motion acquisition; however, like any measurement system they are not error-free: noise is their intrinsic feature. The works so far mostly employ a simple noise model, expressing the uncertainty as a simple variance. In the work, we demonstrate that it might be not sufficient and we prove the existence of several types of noise and demonstrate how to quantify them using Allan variance. Such a knowledge is especially important for using optical motion capture to calibrate other techniques, and for applications requiring very fine quality of recording. For the automated readout of the noise coefficients, we solve the multidimensional regression problem using sophisticated metaheuristics in the exploration-exploitation scheme. We identified in the laboratory the notable contribution to the overall noise from white noise and random walk, and a minor contribution from blue noise and flicker, whereas the violet noise is absent. Besides classic types of noise we identified the presence of the correlated noises and periodic distortion. We analyzed also how the noise types scale with an increasing number of cameras. We had also the opportunity to observe the influence of camera failure on the overall performance.
| S-EPMC6832304 | biostudies-literature
Project description:Deep learning has been transformative in many fields, motivating the emergence of various optical computing architectures. Diffractive optical network is a recently introduced optical computing framework that merges wave optics with deep-learning methods to design optical neural networks. Diffraction-based all-optical object recognition systems, designed through this framework and fabricated by 3D printing, have been reported to recognize hand-written digits and fashion products, demonstrating all-optical inference and generalization to sub-classes of data. These previous diffractive approaches employed monochromatic coherent light as the illumination source. Here, we report a broadband diffractive optical neural network design that simultaneously processes a continuum of wavelengths generated by a temporally incoherent broadband source to all-optically perform a specific task learned using deep learning. We experimentally validated the success of this broadband diffractive neural network architecture by designing, fabricating and testing seven different multi-layer, diffractive optical systems that transform the optical wavefront generated by a broadband THz pulse to realize (1) a series of tuneable, single-passband and dual-passband spectral filters and (2) spatially controlled wavelength de-multiplexing. Merging the native or engineered dispersion of various material systems with a deep-learning-based design strategy, broadband diffractive neural networks help us engineer the light-matter interaction in 3D, diverging from intuitive and analytical design methods to create task-specific optical components that can all-optically perform deterministic tasks or statistical inference for optical machine learning.
| S-EPMC6885516 | biostudies-literature