Dataset Information

A Differentiable Physics Engine for Deep Learning in Robotics.

ABSTRACT: An important field in robotics is the optimization of controllers. Currently, robots are often treated as a black box in this optimization process, which is the reason why derivative-free optimization methods such as evolutionary algorithms or reinforcement learning are omnipresent. When gradient-based methods are used, models are kept small or rely on finite difference approximations for the Jacobian. This method quickly grows expensive with increasing numbers of parameters, such as found in deep learning. We propose the implementation of a modern physics engine, which can differentiate control parameters. This engine is implemented for both CPU and GPU. Firstly, this paper shows how such an engine speeds up the optimization process, even for small problems. Furthermore, it explains why this is an alternative approach to deep Q-learning, for using deep learning in robotics. Finally, we argue that this is a big step for deep learning in robotics, as it opens up new possibilities to optimize robots, both in hardware and software.

SUBMITTER: Degrave J

PROVIDER: S-EPMC6416213 | biostudies-other | 2019

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

A Differentiable Physics Engine for Deep Learning in Robotics.

Degrave Jonas J Hermans Michiel M Dambre Joni J Wyffels Francis F

Frontiers in neurorobotics 20190307

An important field in robotics is the optimization of controllers. Currently, robots are often treated as a black box in this optimization process, which is the reason why derivative-free optimization methods such as evolutionary algorithms or reinforcement learning are omnipresent. When gradient-based methods are used, models are kept small or rely on finite difference approximations for the Jacobian. This method quickly grows expensive with increasing numbers of parameters, such as found in de ...[more]

PMID: 30899218

Similar Datasets

Project description:PurposeEarlier work showed that IVIM-NETorig , an unsupervised physics-informed deep neural network, was faster and more accurate than other state-of-the-art intravoxel-incoherent motion (IVIM) fitting approaches to diffusion-weighted imaging (DWI). This study presents a substantially improved version, IVIM-NEToptim , and characterizes its superior performance in pancreatic cancer patients.MethodIn simulations (signal-to-noise ratio [SNR] = 20), the accuracy, independence, and consistency of IVIM-NET were evaluated for combinations of hyperparameters (fit S0, constraints, network architecture, number of hidden layers, dropout, batch normalization, learning rate), by calculating the normalized root-mean-square error (NRMSE), Spearman's ρ, and the coefficient of variation (CVNET ), respectively. The best performing network, IVIM-NEToptim was compared to least squares (LS) and a Bayesian approach at different SNRs. IVIM-NEToptim 's performance was evaluated in an independent dataset of 23 patients with pancreatic ductal adenocarcinoma. Fourteen of the patients received no treatment between two repeated scan sessions and nine received chemoradiotherapy between the repeated sessions. Intersession within-subject standard deviations (wSD) and treatment-induced changes were assessed.ResultsIn simulations (SNR = 20), IVIM-NEToptim outperformed IVIM-NETorig in accuracy (NRMSE(D) = 0.177 vs 0.196; NMRSE(f) = 0.220 vs 0.267; NMRSE(D*) = 0.386 vs 0.393), independence (ρ(D*, f) = 0.22 vs 0.74), and consistency (CVNET (D) = 0.013 vs 0.104; CVNET (f) = 0.020 vs 0.054; CVNET (D*) = 0.036 vs 0.110). IVIM-NEToptim showed superior performance to the LS and Bayesian approaches at SNRs < 50. In vivo, IVIM-NEToptim showed significantly less noisy parameter maps with lower wSD for D and f than the alternatives. In the treated cohort, IVIM-NEToptim detected the most individual patients with significant parameter changes compared to day-to-day variations.ConclusionIVIM-NEToptim is recommended for accurate, informative, and consistent IVIM fitting to DWI data.

Project description:Social robotics represents a branch of human-robot interaction dedicated to developing systems to control the robots to operate in unstructured environments with the presence of human beings. Social robots must interact with human beings by understanding social signals and responding appropriately to them. Most social robots are still pre-programmed, not having great ability to learn and respond with actions adequate during an interaction with humans. Recently more elaborate methods use body movements, gaze direction, and body language. However, these methods generally neglect vital signs present during an interaction, such as the human emotional state. In this article, we address the problem of developing a system to turn a robot able to decide, autonomously, what behaviors to emit in the function of the human emotional state. From one side, the use of Reinforcement Learning (RL) represents a way for social robots to learn advanced models of social cognition, following a self-learning paradigm, using characteristics automatically extracted from high-dimensional sensory information. On the other side, Deep Learning (DL) models can help the robots to capture information from the environment, abstracting complex patterns from the visual information. The combination of these two techniques is known as Deep Reinforcement Learning (DRL). The purpose of this work is the development of a DRL system to promote a natural and socially acceptable interaction among humans and robots. For this, we propose an architecture, Social Robotics Deep Q-Network (SocialDQN), for teaching social robots to behave and interact appropriately with humans based on social signals, especially on human emotional states. This constitutes a relevant contribution for the area since the social signals must not only be recognized by the robot but help him to take action appropriated according to the situation presented. Characteristics extracted from people’s faces are considered for extracting the human emotional state aiming to improve the robot perception. The development and validation of the system are carried out with the support of SimDRLSR simulator. Results obtained through several tests demonstrate that the system learned satisfactorily to maximize the rewards, and consequently, the robot behaves in a socially acceptable way.

Dataset Information

A Differentiable Physics Engine for Deep Learning in Robotics.

Publications

A Differentiable Physics Engine for Deep Learning in Robotics.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets