Dataset Information

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning.

ABSTRACT: A lack of automated, quantitative, and accurate assessment of social behaviors in mammalian animal models has limited progress toward understanding mechanisms underlying social interactions and their disorders such as autism. Here we present a new integrated hardware and software system that combines video tracking, depth sensing, and machine learning for automatic detection and quantification of social behaviors involving close and dynamic interactions between two mice of different coat colors in their home cage. We designed a hardware setup that integrates traditional video cameras with a depth camera, developed computer vision tools to extract the body "pose" of individual animals in a social context, and used a supervised learning algorithm to classify several well-described social behaviors. We validated the robustness of the automated classifiers in various experimental settings and used them to examine how genetic background, such as that of Black and Tan Brachyury (BTBR) mice (a previously reported autism model), influences social behavior. Our integrated approach allows for rapid, automated measurement of social behaviors across diverse experimental designs and also affords the ability to develop new, objective behavioral metrics.

SUBMITTER: Hong W

PROVIDER: S-EPMC4586844 | biostudies-literature | 2015 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning.

Hong Weizhe W Kennedy Ann A Burgos-Artizzu Xavier P XP Zelikowsky Moriel M Navonne Santiago G SG Perona Pietro P Anderson David J DJ

Proceedings of the National Academy of Sciences of the United States of America 20150909 38

A lack of automated, quantitative, and accurate assessment of social behaviors in mammalian animal models has limited progress toward understanding mechanisms underlying social interactions and their disorders such as autism. Here we present a new integrated hardware and software system that combines video tracking, depth sensing, and machine learning for automatic detection and quantification of social behaviors involving close and dynamic interactions between two mice of different coat colors ...[more]

PMID: 26354123

Similar Datasets

Project description:Introduction: Video-based clinical rating plays an important role in assessing dystonia and monitoring the effect of treatment in dyskinetic cerebral palsy (CP). However, evaluation by clinicians is time-consuming, and the quality of rating is dependent on experience. The aim of the current study is to provide a proof-of-concept for a machine learning approach to automatically assess scoring of dystonia using 2D stick figures extracted from videos. Model performance was compared to human performance. Methods: A total of 187 video sequences of 34 individuals with dyskinetic CP (8-23 years, all non-ambulatory) were filmed at rest during lying and supported sitting. Videos were scored by three raters according to the Dyskinesia Impairment Scale (DIS) for arm and leg dystonia (normalized scores ranging from 0-1). Coordinates in pixels of the left and right wrist, elbow, shoulder, hip, knee and ankle were extracted using DeepLabCut, an open source toolbox that builds on a pose estimation algorithm. Within a subset, tracking accuracy was assessed for a pretrained human model and for models trained with an increasing number of manually labeled frames. The mean absolute error (MAE) between DeepLabCut's prediction of the position of body points and manual labels was calculated. Subsequently, movement and position features were calculated from extracted body point coordinates. These features were fed into a Random Forest Regressor to train a model to predict the clinical scores. The model performance trained with data from one rater evaluated by MAEs (model-rater) was compared to inter-rater accuracy. Results: A tracking accuracy of 4.5 pixels (approximately 1.5 cm) could be achieved by adding 15-20 manually labeled frames per video. The MAEs for the trained models ranged from 0.21 ± 0.15 for arm dystonia to 0.14 ± 0.10 for leg dystonia (normalized DIS scores). The inter-rater MAEs were 0.21 ± 0.22 and 0.16 ± 0.20, respectively. Conclusion: This proof-of-concept study shows the potential of using stick figures extracted from common videos in a machine learning approach to automatically assess dystonia. Sufficient tracking accuracy can be reached by manually adding labels within 15-20 frames per video. With a relatively small data set, it is possible to train a model that can automatically assess dystonia with a performance comparable to human scoring.

Project description:Quantifying and understanding movement is critical for a wide range of questions in basic and applied ecology. Movement ecology is also fostered by technological advances that allow automated tracking for a wide range of animal species. However, for aquatic macroinvertebrates, such detailed methods do not yet exist. We developed a video tracking method for two different species of benthic macroinvertebrates, the crawling isopod Asellus aquaticus and the swimming fresh water amphipod Gammarus pulex. We tested the effects of different light sources and marking techniques on their movement behavior to establish the possibilities and limitations of the experimental protocol and to ensure that the basic handling of test specimens would not bias conclusions drawn from movement path analyses. To demonstrate the versatility of our method, we studied the influence of varying population densities on different movement parameters related to resting behavior, directionality, and step lengths. We found that our method allows studying species with different modes of dispersal and under different conditions. For example, we found that gammarids spend more time moving at higher population densities, while asellids rest more under similar conditions. At the same time, in response to higher densities, gammarids mostly decreased average step lengths, whereas asellids did not. Gammarids, however, were also more sensitive to general handling and marking than asellids. Our protocol for marking and video tracking can be easily adopted for other species of aquatic macroinvertebrates or testing conditions, for example, presence or absence of food sources, shelter, or predator cues. Nevertheless, limitations with regard to the marking protocol, material, and a species' physical build need to be considered and tested before a wider application, particularly for swimming species. Data obtained with this approach can deepen the understanding of population dynamics on larger spatial scales and of the effects of different management strategies on a species' dispersal potential.

Dataset Information

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning.

Publications

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets