Dataset Information

Robust and automatic motion-capture data recovery using soft skeleton constraints and model averaging.

ABSTRACT: Motion capture allows accurate recording of human motion, with applications in many fields, including entertainment, medicine, sports science and human computer interaction. A common difficulty with this technology is the occurrence of missing data, due to occlusions, or recording conditions. Various models have been proposed to estimate missing data. Some are based on interpolation, low-rank properties or inter-correlations. Others involve dataset matching or skeleton constraints. While the latter have the advantage of promoting a realistic motion estimation, they require prior knowledge of skeleton constraints, or the availability of a prerecorded dataset. In this article, we propose a probabilistic averaging method of several recovery models (referred to as Probabilistic Model Averaging (PMA) in this paper), based on the likelihoods of the distances between body points. This method has the advantage of being automatic, while allowing an efficient gap data recovery. To support and validate the proposed method, we use a set of four individual recovery models, based on linear/nonlinear regression in local coordinate systems. Finally, we propose two heuristic algorithms to enforce skeleton constraints in the reconstructed motion, which can be used on any individual recovery model. For validation purposes, random gaps were introduced into motion-capture sequences, and the effects of factors such as the number of simultaneous gaps, gap length and sequence duration were analyzed. Results show that the proposed probabilistic averaging method yields better recovery than (i) each of the four individual models and (ii) two recent state-of-the-art models, regardless of gap length, sequence duration and number of simultaneous gaps. Moreover, both of our heuristic skeleton-constraint algorithms significantly improve the recovery for 7 out of 8 tested motion-capture sequences (p < 0.05), for 10 simultaneous gaps of 5 seconds. The code is available for free download at: https://github.com/numediart/MocapRecovery.

SUBMITTER: Tits M

PROVIDER: S-EPMC6039011 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Robust and automatic motion-capture data recovery using soft skeleton constraints and model averaging.

Tits Mickaël M Tilmanne Joëlle J Dutoit Thierry T

PloS one 20180710 7

Motion capture allows accurate recording of human motion, with applications in many fields, including entertainment, medicine, sports science and human computer interaction. A common difficulty with this technology is the occurrence of missing data, due to occlusions, or recording conditions. Various models have been proposed to estimate missing data. Some are based on interpolation, low-rank properties or inter-correlations. Others involve dataset matching or skeleton constraints. While the lat ...[more]

PMID: 29990367

Similar Datasets

Project description:Agent-based models of 'flocking' and 'schooling' have shown that a weighted average of neighbor velocities, with weights that decay gradually with distance, yields emergent collective motion. Weighted averaging thus offers a potential mechanism of self-organization that recruits an increasing, but self-limiting, number of individuals into collective motion. Previously, we identified and modeled such a 'soft metric' neighborhood of interaction in human crowds that decays exponentially to zero at a distance of 4-5m. Here we investigate the limits of weighted averaging in humans and find that it is surprisingly robust: pedestrians align with the mean heading direction in their neighborhood, despite high levels of noise and diverging motions in the crowd, as predicted by the model. In three Virtual Reality experiments, participants were immersed in a crowd of virtual humans in a mobile head-mounted display and were instructed to walk with the crowd. By perturbing the heading (walking direction) of virtual neighbors and measuring the participant's trajectory, we probed the limits of weighted averaging. (1) In the 'Noisy Neighbors' experiment, the neighbor headings were randomized (range 0-90°) about the crowd's mean direction (±10° or ±20°, left or right); (2) in the 'Splitting Crowd' experiment, the crowd split into two groups (heading difference = 10-40°) and the proportion of the crowd in one group was varied (50-84%); (3) in the 'Coherent Subgroup' experiment, a perturbed subgroup varied in its coherence (heading SD = 0-2°) about a mean direction (±10° or ±20°) within a noisy crowd (heading range = 180°), and the proportion of the crowd in the subgroup was varied. In each scenario, the results were predicted by the weighted averaging model, and attraction strength (turning rate) increased with the participant's deviation from the mean heading direction, not with group coherence. However, the results indicate that humans ignore highly discrepant headings (45-90°). These findings reveal that weighted averaging in humans is highly robust and generates a common heading direction that acts as a positive feedback to recruit more individuals into collective motion, in a self-reinforcing cascade. Therefore, this 'soft' metric neighborhood serves as a mechanism of self-organization in human crowds.

Project description:In this study, we propose a novel markerless motion capture system (MCS) for monkeys, in which 3D surface images of monkeys were reconstructed by integrating data from four depth cameras, and a skeleton model of the monkey was fitted onto 3D images of monkeys in each frame of the video. To validate the MCS, first, estimated 3D positions of body parts were compared between the 3D MCS-assisted estimation and manual estimation based on visual inspection when a monkey performed a shuttling behavior in which it had to avoid obstacles in various positions. The mean estimation error of the positions of body parts (3-14 cm) and of head rotation (35-43°) between the 3D MCS-assisted and manual estimation were comparable to the errors between two different experimenters performing manual estimation. Furthermore, the MCS could identify specific monkey actions, and there was no false positive nor false negative detection of actions compared with those in manual estimation. Second, to check the reproducibility of MCS-assisted estimation, the same analyses of the above experiments were repeated by a different user. The estimation errors of positions of most body parts between the two experimenters were significantly smaller in the MCS-assisted estimation than in the manual estimation. Third, effects of methamphetamine (MAP) administration on the spontaneous behaviors of four monkeys were analyzed using the MCS. MAP significantly increased head movements, tended to decrease locomotion speed, and had no significant effect on total path length. The results were comparable to previous human clinical data. Furthermore, estimated data following MAP injection (total path length, walking speed, and speed of head rotation) correlated significantly between the two experimenters in the MCS-assisted estimation (r = 0.863 to 0.999). The results suggest that the presented MCS in monkeys is useful in investigating neural mechanisms underlying various psychiatric disorders and developing pharmacological interventions.

Project description:Marker-based Optical Motion Capture (OMC) systems and associated musculoskeletal (MSK) modelling predictions offer non-invasively obtainable insights into muscle and joint loading at an in vivo level, aiding clinical decision-making. However, an OMC system is lab-based, expensive, and requires a line of sight. Inertial Motion Capture (IMC) techniques are widely-used alternatives, which are portable, user-friendly, and relatively low-cost, although with lesser accuracy. Irrespective of the choice of motion capture technique, one typically uses an MSK model to obtain the kinematic and kinetic outputs, which is a computationally expensive tool increasingly well approximated by machine learning (ML) methods. Here, an ML approach is presented that maps experimentally recorded IMC input data to the human upper-extremity MSK model outputs computed from ('gold standard') OMC input data. Essentially, this proof-of-concept study aims to predict higher-quality MSK outputs from the much easier-to-obtain IMC data. We use OMC and IMC data simultaneously collected for the same subjects to train different ML architectures that predict OMC-driven MSK outputs from IMC measurements. In particular, we employed various neural network (NN) architectures, such as Feed-Forward Neural Networks (FFNNs) and Recurrent Neural Networks (RNNs) (vanilla, Long Short-Term Memory, and Gated Recurrent Unit) and a comprehensive search for the best-fit model in the hyperparameters space in both subject-exposed (SE) as well as subject-naive (SN) settings. We observed a comparable performance for both FFNN and RNN models, which have a high degree of agreement (ravg,SE,FFNN=0.90±0.19, ravg,SE,RNN=0.89±0.17, ravg,SN,FFNN=0.84±0.23, and ravg,SN,RNN=0.78±0.23) with the desired OMC-driven MSK estimates for held-out test data. The findings demonstrate that mapping IMC inputs to OMC-driven MSK outputs using ML models could be instrumental in transitioning MSK modelling from 'lab to field'.

Project description:OBJECTIVES:In this study, we present a fully automated and robust self-navigated approach to obtain 4-dimensional (4-D) motion-resolved images during free breathing. MATERIALS AND METHODS:The proposed method, Consistently Acquired Projections for Tuned and Robust Estimation (CAPTURE), is a variant of the stack-of-stars gradient-echo sequence. A 1-D navigator was consistently acquired at a fixed azimuthal angle for all stacks of spokes to reduce nonphysiological signal contamination due to system imperfections. The resulting projections were then "tuned" using complex phase rotation to adapt to scan-to-scan variations, followed by the detection of the respiratory curve. Four-dimensional motion-corrected and uncorrected images were then reconstructed via respiratory and temporal binning, respectively.This Health Insurance Portability and Accountability Act-compliant study was performed with Institutional Review Board approval. A phantom experiment was performed using a custom-made deformable motion phantom with an adjustable frequency and amplitude. For in vivo experiments, 10 healthy participants and 12 liver tumor patients provided informed consent and were imaged with the CAPTURE sequence.Two radiologists, blinded to which images were motion-corrected and which were not, independently reviewed the images and scored the image quality using a 5-point Likert scale. RESULTS:In the respiratory motion phantom experiment, CAPTURE reversed the effects of motion blurring and restored edge sharpness from 36% to 78% of that observed in the images from the static scan.Despite large intra- and intersubject variability in respiration patterns, CAPTURE successfully detected the respiratory motion signal in all participants and significantly improved the image quality according to the subjective radiological assessments of 2 raters (P < 0.05 for both raters) with a 1 to 2-point improvement in the median Likert scores across the whole set of participants. Small lesions (<1 cm in size) which might otherwise be missed on uncorrected images because of motion blurring were more clearly depicted on the CAPTURE images. CONCLUSIONS:CAPTURE provides a robust and fully automated solution for obtaining 4-D motion-resolved images in a free-breathing setting. With its unique tuning feature, CAPTURE can adapt to large intersubject and interscan variations. CAPTURE also enables better lesion delineation because of improved image sharpness, thereby increasing the visibility of small lesions.

Dataset Information

Robust and automatic motion-capture data recovery using soft skeleton constraints and model averaging.

Publications

Robust and automatic motion-capture data recovery using soft skeleton constraints and model averaging.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets