A reinforcement learning enhanced pseudo-inverse approach to self-collision avoidance of redundant robots
Ontology highlight
ABSTRACT:
SUBMITTER:
PROVIDER: S-EPMC11006967 | biostudies-literature | 2024 Jan
REPOSITORIES: biostudies-literature
Similar Datasets
Project description: Not available
| S-EPMC11607400 | biostudies-literature
Project description:Although avoidance is a prevalent feature of anxiety-related psychopathology, differences in the measurement of avoidance between humans and non-human animals hinder our progress in its theoretical understanding and treatment. To address this, we developed a novel translational measure of anxiety-related avoidance in the form of an approach-avoidance reinforcement learning task, by adapting a paradigm from the non-human animal literature to study the same cognitive processes in human participants. We used computational modelling to probe the putative cognitive mechanisms underlying approach-avoidance behaviour in this task and investigated how they relate to subjective task-induced anxiety. In a large online study (n = 372), participants who experienced greater task-induced anxiety avoided choices associated with punishment, even when this resulted in lower overall reward. Computational modelling revealed that this effect was explained by greater individual sensitivities to punishment relative to rewards. We replicated these findings in an independent sample (n = 627) and we also found fair-to-excellent reliability of measures of task performance in a sub-sample retested 1 week later (n = 57). Our findings demonstrate the potential of approach-avoidance reinforcement learning tasks as translational and computational models of anxiety-related avoidance. Future studies should assess the predictive validity of this approach in clinical samples and experimental manipulations of anxiety.
| S-EPMC10645421 | biostudies-literature
Project description:This paper investigates the potential of the intrinsically motivated reinforcement learning (IMRL) approach for robotic drumming. For this purpose, we implemented an IMRL-based algorithm for a drumming robot called ZRob, an underactuated two-DoF robotic arm with flexible grippers. Two ZRob robots were instructed to play rhythmic patterns derived from MIDI files. The RL algorithm is based on the deep deterministic policy gradient (DDPG) method, but instead of relying solely on extrinsic rewards, the robots are trained using a combination of both extrinsic and intrinsic reward signals. The results of the training experiments show that the utilization of intrinsic reward can lead to meaningful novel rhythmic patterns, while using only extrinsic reward would lead to predictable patterns identical to the MIDI inputs. Additionally, the observed drumming patterns are influenced not only by the learning algorithm but also by the robots' physical dynamics and the drum's constraints. This work suggests new insights into the potential of embodied intelligence for musical performance.
| S-EPMC11609846 | biostudies-literature
Project description:Medical robots should not collide with close by obstacles during medical procedures, such as lamps, screens, or medical personnel. Redundant robots have more degrees of freedom than needed for moving endoscopic tools during surgery and can be reshaped to avoid obstacles by moving purely in the space of these additional degrees of freedom (null space). Although state-of-the-art robots allow surgeons to hand-guide endoscopic tools, reshaping the robot in null space is not intuitive for surgeons. Here we propose a learned task space control that allows surgeons to intuitively teach preferred robot configurations (shapes) that avoid obstacles using a VR-based planner in simulation. Later during surgery, surgeons control both the endoscopic tool and robot configuration (shape) with one hand. In a user study, we found that learned task space control outperformed state-of-the-art naive task space control in all the measured performance metrics (time, effort, and user-perceived effort). Our solution allowed users to intuitively interact with robots in VR and reshape robots while moving tools in medical and industrial applications.
| S-EPMC11825656 | biostudies-literature
Project description:Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user's error expectation of the robot's current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user's preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user's preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.
| S-EPMC8677775 | biostudies-literature
Project description:As robots continue to acquire useful skills, their ability to teach their expertise will provide humans the two-fold benefit of learning from robots and collaborating fluently with them. For example, robot tutors could teach handwriting to individual students and delivery robots could convey their navigation conventions to better coordinate with nearby human workers. Because humans naturally communicate their behaviors through selective demonstrations, and comprehend others' through reasoning that resembles inverse reinforcement learning (IRL), we propose a method of teaching humans based on demonstrations that are informative for IRL. But unlike prior work that optimizes solely for IRL, this paper incorporates various human teaching strategies (e.g. scaffolding, simplicity, pattern discovery, and testing) to better accommodate human learners. We assess our method with user studies and find that our measure of test difficulty corresponds well with human performance and confidence, and also find that favoring simplicity and pattern discovery increases human performance on difficult tests. However, we did not find a strong effect for our method of scaffolding, revealing shortcomings that indicate clear directions for future work.
| S-EPMC8278287 | biostudies-literature
Project description:Collision avoidance between multiple walkers, such as pedestrians in a crowd, is based on a reciprocal coupling between the walkers with a continuous loop between perception and action. Such interpersonal coordination has previously been studied in the case of dyadic locomotor interactions. However, when walking through a crowd of people, collision avoidance is not restricted to dyadic interactions. We examined how dyadic avoidance (1 vs. 1) compared to triadic avoidance (1 vs. 2). Additionally, we examined how the dynamics of a passable gap between two walkers affected locomotor interactions. To this end, we manipulated the starting formation of two walkers that formed a potentially pass-able gap for the other walker. We analyzed the interactions in terms of the evolution over time of the Minimal Predicted Distance and the Dynamics of the Gap, which both provide information about what action is afforded (i.e., passing in front/behind and the pass-ability of the gap). Results showed that some triadic interactions invited for sequential interactions, resulting in avoidance strategies comparable with dyadic interactions. However, some formations resulted in simultaneous interactions where the dynamics of the pass-ability of the gap revealed that the coordination strategy emerged over time through the bi-directional interactions between all walkers. Future work should address which circumstances invite for simultaneous and which for sequential interactions between multiple walkers. This study contributed toward understanding how collision is avoided between multiple walkers at the level of the local interactions.
| S-EPMC6284014 | biostudies-literature
Project description:Visual motion provides rich geometrical cues about the three-dimensional configuration of the world. However, how brains decode the spatial information carried by motion signals remains poorly understood. Here, we study a collision-avoidance behavior in Drosophila as a simple model of motion-based spatial vision. With simulations and psychophysics, we demonstrate that walking Drosophila exhibit a pattern of slowing to avoid collisions by exploiting the geometry of positional changes of objects on near-collision courses. This behavior requires the visual neuron LPLC1, whose tuning mirrors the behavior and whose activity drives slowing. LPLC1 pools inputs from object and motion detectors, and spatially biased inhibition tunes it to the geometry of collisions. Connectomic analyses identified circuitry downstream of LPLC1 that faithfully inherits its response properties. Overall, our results reveal how a small neural circuit solves a specific spatial vision task by combining distinct visual features to exploit universal geometrical constraints of the visual world.
| S-EPMC9177691 | biostudies-literature
Project description:Soft continuum manipulators have the potential to replace traditional surgical catheters; offering greater dexterity with access to previously unfeasible locations for a wide range of interventions including neurological and cardiovascular. Magnetically actuated catheters are of particular interest due to their potential for miniaturization and remote control. Challenges around the operation of these catheters exist however, and one of these occurs when the angle between the actuating field and the local magnetization vector of the catheter exceeds 90°. In this arrangement, deformation generated by the resultant magnetic moment acts to increase magnetic torque, leading to potential instability. This phenomenon can cause unpredictable responses to actuation, particularly for soft, flexible materials. When coupled with the inherent challenges of sensing and localization inside living tissue, this behavior represents a barrier to progress. In this feasibility study we propose and investigate the use of helical fiber reinforcement within magnetically actuated soft continuum manipulators. Using numerical simulation to explore the design space, we optimize fiber parameters to enhance the ratio of torsional to bending stiffness. Through bespoke fabrication of an optimized helix design we validate a single, prototypical two-segment, 40 mm × 6 mm continuum manipulator demonstrating a reduction of 67% in unwanted twisting under actuation.
| S-EPMC8297468 | biostudies-literature
Project description:We have investigated how birds avoid mid-air collisions during head-on encounters. Trajectories of birds flying towards each other in a tunnel were recorded using high speed video cameras. Analysis and modelling of the data suggest two simple strategies for collision avoidance: (a) each bird veers to its right and (b) each bird changes its altitude relative to the other bird according to a preset preference. Both strategies suggest simple rules by which collisions can be avoided in head-on encounters by two agents, be they animals or machines. The findings are potentially applicable to the design of guidance algorithms for automated collision avoidance on aircraft.
| S-EPMC5040264 | biostudies-literature