An Adaptive Imitation Learning Framework for Robotic Complex Contact-Rich Insertion Tasks.
Ontology highlight
ABSTRACT:
SUBMITTER: Wang Y
PROVIDER: S-EPMC8787218 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Autonomous navigation to a specified waypoint is traditionally accomplished with a layered stack of global path planning and local motion planning modules that generate feasible and obstacle-free trajectories. While these modules can be modified to meet task-specific constraints and user preferences, current modification procedures require substantial effort on the part of an expert roboticist with a great deal of technical training. In this paper, we simplify this process by inserting a Machine Learning module between the global path planning and local motion planning modules of an off-the shelf navigation stack. This model can be trained with human demonstrations of the preferred navigation behavior, using a training procedure based on Behavioral Cloning, allowing for an intuitive modification of the navigation policy by non-technical users to suit task-specific constraints. We find that our approach can successfully adapt a robot's navigation behavior to become more like that of a demonstrator. Moreover, for a fixed amount of demonstration data, we find that the proposed technique compares favorably to recent baselines with respect to both navigation success rate and trajectory similarity to the demonstrator.
| S-EPMC8204187 | biostudies-literature
Project description:Humans often learn new things via imitation. Here we draw on studies of imitation in children to characterise the brain system(s) involved in the imitation of different sequence types using functional magnetic resonance imaging. On each trial, healthy adult participants learned one of two rule types governing the sequencing of three pictures: a motor-spatial rule (in the spatial task) or an object-based rule (in the cognitive task). Sequences were learned via one of three demonstration types: a video of a hand selecting items in the sequence using a joystick (Hand condition), a computer display highlighting each item in order (Ghost condition), or a text-based demonstration of the sequence (Text condition). Participants then used a joystick to execute the learned sequence. Patterns of activation during demonstration observation suggest specialisation for object-based imitation in inferior frontal gyrus, specialisation for spatial sequences in anterior intraparietal sulcus (IPS), and a general preference for imitation in middle IPS. Adult behavioural performance contrasted with that of children in previous studies-indicating that they experienced more difficulty with the cognitive task-while neuroimaging results support the engagement of different neural regions when solving these tasks. Further study is needed on whether children's differential performance is related to delayed IPS maturation.
| S-EPMC6075756 | biostudies-literature
Project description:Despite recent advances in robot-assisted training, the benefits of haptic guidance on motor (re)learning are still limited. While haptic guidance may increase task performance during training, it may also decrease participants' effort and interfere with the perception of the environment dynamics, hindering somatosensory information crucial for motor learning. Importantly, haptic guidance limits motor variability, a factor considered essential for learning. We propose that Model Predictive Controllers (MPC) might be good alternatives to haptic guidance since they minimize the assisting forces and promote motor variability during training. We conducted a study with 40 healthy participants to investigate the effectiveness of MPCs on learning a dynamic task. The task consisted of swinging a virtual pendulum to hit incoming targets with the pendulum ball. The environment was haptically rendered using a Delta robot. We designed two MPCs: the first MPC-end-effector MPC-applied the optimal assisting forces on the end-effector. A second MPC-ball MPC-applied its forces on the virtual pendulum ball to further reduce the assisting forces. The participants' performance during training and learning at short- and long-term retention tests were compared to a control group who trained without assistance, and a group that trained with conventional haptic guidance. We hypothesized that the end-effector MPC would promote motor variability and minimize the assisting forces during training, and thus, promote learning. Moreover, we hypothesized that the ball MPC would enhance the performance and motivation during training but limit the motor variability and sense of agency (i.e., the feeling of having control over their movements), and therefore, limit learning. We found that the MPCs reduce the assisting forces compared to haptic guidance. Training with the end-effector MPC increases the movement variability and does not hinder the pendulum swing variability during training, ultimately enhancing the learning of the task dynamics compared to the other groups. Finally, we observed that increases in the sense of agency seemed to be associated with learning when training with the end-effector MPC. In conclusion, training with MPCs enhances motor learning of tasks with complex dynamics and are promising strategies to improve robotic training outcomes in neurological patients.
| S-EPMC7884323 | biostudies-literature
Project description:In this paper, we propose a deep reinforcement learning-based framework that enables adaptive and continuous control of a robot to push unseen objects from random positions to the target position. Our approach takes into account contact information in the design of the reward function, resulting in improved success rates, generalization for unseen objects, and task efficiency compared to policies that do not consider contact information. Through reinforcement learning using only one object in simulation, we obtain a learned policy for manipulating a single object, which demonstrates good generalization when applied to the task of pushing unseen objects. Finally, we validate the effectiveness of our approach in real-world scenarios.
| S-EPMC10536165 | biostudies-literature
Project description:Epidemics may pose a significant dilemma for governments and individuals. The personal or public health consequences of inaction may be catastrophic; but the economic consequences of drastic response may likewise be catastrophic. In the face of these trade-offs, governments and individuals must therefore strike a balance between the economic and personal health costs of reducing social contacts and the public health costs of neglecting to do so. As risk of infection increases, potentially infectious contact between people is deliberately reduced either individually or by decree. This must be balanced against the social and economic costs of having fewer people in contact, and therefore active in the labor force or enrolled in school. Although the importance of adaptive social contact on epidemic outcomes has become increasingly recognized, the most important properties of coupled human-natural epidemic systems are still not well understood. We develop a theoretical model for adaptive, optimal control of the effective social contact rate using traditional epidemic modeling tools and a utility function with delayed information. This utility function trades off the population-wide contact rate with the expected cost and risk of increasing infections. Our analytical and computational analysis of this simple discrete-time deterministic strategic model reveals the existence of an endemic equilibrium, oscillatory dynamics around this equilibrium under some parametric conditions, and complex dynamic regimes that shift under small parameter perturbations. These results support the supposition that infectious disease dynamics under adaptive behavior change may have an indifference point, may produce oscillatory dynamics without other forcing, and constitute complex adaptive systems with associated dynamics. Implications for any epidemic in which adaptive behavior influences infectious disease dynamics include an expectation of fluctuations, for a considerable time, around a quasi-equilibrium that balances public health and economic priorities, that shows multiple peaks and surges in some scenarios, and that implies a high degree of uncertainty in mathematical projections.
| S-EPMC7875423 | biostudies-literature
Project description:To coordinate actions with an interaction partner requires a constant exchange of sensorimotor signals. Humans acquire these skills in infancy and early childhood mostly by imitation learning and active engagement with a skilled partner. They require the ability to predict and adapt to one's partner during an interaction. In this work we want to explore these ideas in a human-robot interaction setting in which a robot is required to learn interactive tasks from a combination of observational and kinesthetic learning. To this end, we propose a deep learning framework consisting of a number of components for (1) human and robot motion embedding, (2) motion prediction of the human partner, and (3) generation of robot joint trajectories matching the human motion. As long-term motion prediction methods often suffer from the problem of regression to the mean, our technical contribution here is a novel probabilistic latent variable model which does not predict in joint space but in latent space. To test the proposed method, we collect human-human interaction data and human-robot interaction data of four interactive tasks "hand-shake," "hand-wave," "parachute fist-bump," and "rocket fist-bump." We demonstrate experimentally the importance of predictive and adaptive components as well as low-level abstractions to successfully learn to imitate human behavior in interactive social tasks.
| S-EPMC7806025 | biostudies-literature
Project description:IntroductionThe learning curve associated with robotic pancreatoduodenectomy (RPD) is a hurdle for new programs to achieve optimal results. Since early analysis, robotic training has recently expanded, and the RPD approach has been refined. The purpose of this study is to examine RPD outcomes for surgeons who implemented a new program after receiving formal RPD training to determine if such training reduces the learning curve.MethodsOutcomes for consecutive patients undergoing RPD at a single tertiary institution were compared to optimal RPD benchmarks from a previously reported learning curve analysis. Two surgical oncologists with formal RPD training performed all operations with one surgeon as bedside assistant and the other at the console.ResultsForty consecutive RPD operations were evaluated. Mean operative time was 354 ± 54 min, and blood loss was 300 ml. Length of stay was 7 days. Three patients (7.5%) underwent conversion to open. Pancreatic fistula affected five patients (12.5%). Operative time was stable over the study and lower than the reported benchmark. These RPD operative outcomes were similar to reported surgeon outcomes after the learning curve.ConclusionThis study suggests formal robotic training facilitates safe and efficient adoption of RPD for new programs, reducing or eliminating the learning curve.
| S-EPMC7902319 | biostudies-literature
Project description:Automating retrosynthesis with artificial intelligence expedites organic chemistry research in digital laboratories. However, most existing deep-learning approaches are hard to explain, like a "black box" with few insights. Here, we propose RetroExplainer, formulizing the retrosynthesis task into a molecular assembly process, containing several retrosynthetic actions guided by deep learning. To guarantee a robust performance of our model, we propose three units: a multi-sense and multi-scale Graph Transformer, structure-aware contrastive learning, and dynamic adaptive multi-task learning. The results on 12 large-scale benchmark datasets demonstrate the effectiveness of RetroExplainer, which outperforms the state-of-the-art single-step retrosynthesis approaches. In addition, the molecular assembly process renders our model with good interpretability, allowing for transparent decision-making and quantitative attribution. When extended to multi-step retrosynthesis planning, RetroExplainer has identified 101 pathways, in which 86.9% of the single reactions correspond to those already reported in the literature. As a result, RetroExplainer is expected to offer valuable insights for reliable, high-throughput, and high-quality organic synthesis in drug development.
| S-EPMC10547708 | biostudies-literature
Project description:Neuroprosthetic brain-computer interfaces function via an algorithm which decodes neural activity of the user into movements of an end effector, such as a cursor or robotic arm. In practice, the decoder is often learned by updating its parameters while the user performs a task. When the user's intention is not directly observable, recent methods have demonstrated value in training the decoder against a surrogate for the user's intended movement. Here we show that training a decoder in this way is a novel variant of an imitation learning problem, where an oracle or expert is employed for supervised training in lieu of direct observations, which are not available. Specifically, we describe how a generic imitation learning meta-algorithm, dataset aggregation (DAgger), can be adapted to train a generic brain-computer interface. By deriving existing learning algorithms for brain-computer interfaces in this framework, we provide a novel analysis of regret (an important metric of learning efficacy) for brain-computer interfaces. This analysis allows us to characterize the space of algorithmic variants and bounds on their regret rates. Existing approaches for decoder learning have been performed in the cursor control setting, but the available design principles for these decoders are such that it has been impossible to scale them to naturalistic settings. Leveraging our findings, we then offer an algorithm that combines imitation learning with optimal control, which should allow for training of arbitrary effectors for which optimal control can generate goal-oriented control. We demonstrate this novel and general BCI algorithm with simulated neuroprosthetic control of a 26 degree-of-freedom model of an arm, a sophisticated and realistic end effector.
| S-EPMC4871564 | biostudies-literature
Project description:It has been well established that people respond faster when they perform an action that is congruent with an observed action than when they respond with an incongruent action. Here we propose a new method of using interactive Virtual Characters (VCs) to test if social congruency effects can be obtained in a richer social context with sequential hand-arm actions. Two separate experiments were conducted, exploring if it is feasible to measure spatial congruency (Experiment 1) and anatomical congruency (Experiment 2) in response to a VC, compared to the same action sequence indicated by three virtual balls. In Experiment 1, we found a robust spatial congruency effect for both VC and virtual balls, modulated by a social facilitation effect for participants who felt the VC was human. In Experiment 2 which allowed for anatomical congruency, a form by congruency interaction provided evidence that participants automatically imitate the actions of the VC but do not imitate the balls. Our method and results build a bridge between studies using minimal stimuli in automatic interaction and studies of mimicry in a rich social interaction, and open new research venue for future research in the area of automatic imitation with a more ecologically valid social interaction.
| S-EPMC4460321 | biostudies-literature