Dataset Information

Learning dictionaries of sparse codes of 3D movements of body joints for real-time human activity understanding.

ABSTRACT: Real-time human activity recognition is essential for human-robot interactions for assisted healthy independent living. Most previous work in this area is performed on traditional two-dimensional (2D) videos and both global and local methods have been used. Since 2D videos are sensitive to changes of lighting condition, view angle, and scale, researchers begun to explore applications of 3D information in human activity understanding in recently years. Unfortunately, features that work well on 2D videos usually don't perform well on 3D videos and there is no consensus on what 3D features should be used. Here we propose a model of human activity recognition based on 3D movements of body joints. Our method has three steps, learning dictionaries of sparse codes of 3D movements of joints, sparse coding, and classification. In the first step, space-time volumes of 3D movements of body joints are obtained via dense sampling and independent component analysis is then performed to construct a dictionary of sparse codes for each activity. In the second step, the space-time volumes are projected to the dictionaries and a set of sparse histograms of the projection coefficients are constructed as feature representations of the activities. Finally, the sparse histograms are used as inputs to a support vector machine to recognize human activities. We tested this model on three databases of human activities and found that it outperforms the state-of-the-art algorithms. Thus, this model can be used for real-time human activity recognition in many applications.

SUBMITTER: Qi J

PROVIDER: S-EPMC4256388 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Learning dictionaries of sparse codes of 3D movements of body joints for real-time human activity understanding.

Qi Jin J Yang Zhiyong Z

PloS one 20141204 12

Real-time human activity recognition is essential for human-robot interactions for assisted healthy independent living. Most previous work in this area is performed on traditional two-dimensional (2D) videos and both global and local methods have been used. Since 2D videos are sensitive to changes of lighting condition, view angle, and scale, researchers begun to explore applications of 3D information in human activity understanding in recently years. Unfortunately, features that work well on 2D ...[more]

PMID: 25473850

Similar Datasets

Project description:The visual cortex's hierarchical, multi-level organization is captured in many biologically inspired computational vision models, the general idea being that progressively larger scale (spatially/temporally) and more complex visual features are represented in progressively higher areas. However, most earlier models use localist representations (codes) in each representational field (which we equate with the cortical macrocolumn, "mac"), at each level. In localism, each represented feature/concept/event (hereinafter "item") is coded by a single unit. The model we describe, Sparsey, is hierarchical as well but crucially, it uses sparse distributed coding (SDC) in every mac in all levels. In SDC, each represented item is coded by a small subset of the mac's units. The SDCs of different items can overlap and the size of overlap between items can be used to represent their similarity. The difference between localism and SDC is crucial because SDC allows the two essential operations of associative memory, storing a new item and retrieving the best-matching stored item, to be done in fixed time for the life of the model. Since the model's core algorithm, which does both storage and retrieval (inference), makes a single pass over all macs on each time step, the overall model's storage/retrieval operation is also fixed-time, a criterion we consider essential for scalability to the huge ("Big Data") problems. A 2010 paper described a nonhierarchical version of this model in the context of purely spatial pattern processing. Here, we elaborate a fully hierarchical model (arbitrary numbers of levels and macs per level), describing novel model principles like progressive critical periods, dynamic modulation of principal cells' activation functions based on a mac-level familiarity measure, representation of multiple simultaneously active hypotheses, a novel method of time warp invariant recognition, and we report results showing learning/recognition of spatiotemporal patterns.

Project description:Total and regional body composition are strongly correlated with metabolic syndrome and have been estimated non-invasively from 3D optical scans using linear parameterizations of body shape and linear regression models. Prior works produced accurate and precise predictions on many, but not all, body composition targets relative to the reference dual X-Ray absorptiometry (DXA) measurement. Here, we report the effects of replacing linear models with nonlinear parameterization and regression models on the precision and accuracy of body composition estimation in a novel application of deep 3D convolutional graph networks to human body composition modeling. We assembled an ensemble dataset of 4286 topologically standardized 3D optical scans from four different human body shape databases, DFAUST, CAESAR, Shape Up! Adults, and Shape Up! Kids and trained a parameterized shape model using a graph convolutional 3D autoencoder (3DAE) in lieu of linear PCA. We trained a nonlinear Gaussian process regression (GPR) on the 3DAE parameter space to predict body composition via correlations to paired DXA reference measurements from the Shape Up! scan subset. We tested our model on a set of 424 randomly withheld test meshes and compared the effects of nonlinear computation against prior linear models. Nonlinear GPR produced up to 20% reduction in prediction error and up to 30% increase in precision over linear regression for both sexes in 10 tested body composition variables. Deep shape features produced 6-8% reduction in prediction error over linear PCA features for males only and a 4-14% reduction in precision error for both sexes. Our best performing nonlinear model predicting body composition from deep features outperformed prior work using linear methods on all tested body composition prediction metrics in both precision and accuracy. All coefficients of determination (R2) for all predicted variables were above 0.86. We show that GPR is a more precise and accurate method for modeling body composition mappings from body shape features than linear regression. Deep 3D features learned by a graph convolutional autoencoder only improved male body composition accuracy but improved precision in both sexes. Our work achieved lower estimation RMSEs than all previous work on 10 metrics of body composition.

Dataset Information

Learning dictionaries of sparse codes of 3D movements of body joints for real-time human activity understanding.

Publications

Learning dictionaries of sparse codes of 3D movements of body joints for real-time human activity understanding.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets