Implementation of Q learning and deep Q network for controlling a self balancing robot model.
Ontology highlight
ABSTRACT:
SUBMITTER: Rahman MM
PROVIDER: S-EPMC6302870 | biostudies-other | 2018
REPOSITORIES: biostudies-other
Publications
Implementation of Q learning and deep Q network for controlling a self balancing robot model.
Rahman Md Muhaimin MM Rashid S M Hasanur SMH Hossain M M MM
Robotics and biomimetics 20181221 1
In this paper, the implementations of two reinforcement learnings namely, Q learning and deep Q network (DQN) on the Gazebo model of a self balancing robot have been discussed. The goal of the experiments is to make the robot model learn the best actions for staying balanced in an environment. The more time it can remain within a specified limit, the more reward it accumulates and hence more balanced it is. We did various tests with many hyperparameters and demonstrated the performance curves. ...[more]
Similar Datasets
Project description:Quadruped robots require robust and general locomotion skills to exploit their mobility potential in complex and challenging environments. In this work, we present an implementation of a robust end-to-end learning-based controller on the Solo12 quadruped. Our method is based on deep reinforcement learning of joint impedance references. The resulting control policies follow a commanded velocity reference while being efficient in its energy consumption and easy to deploy. We detail the learning procedure and method for transfer on the real robot. We show elaborate experiments. Finally, we present experimental results of the learned locomotion on various grounds indoors and outdoors. These results show that the Solo12 robot is a suitable open-source platform for research combining learning and control because of the easiness in transferring and deploying learned controllers.
| S-EPMC10366154 | biostudies-literature
Project description:Development of spiking neural networks (SNNs) controlling mobile robots is one of the modern challenges in computational neuroscience and artificial intelligence. Such networks, being replicas of biological ones, are expected to have a higher computational potential than traditional artificial neural networks (ANNs). The critical problem is in the design of robust learning algorithms aimed at building a "living computer" based on SNNs. Here, we propose a simple SNN equipped with a Hebbian rule in the form of spike-timing-dependent plasticity (STDP). The SNN implements associative learning by exploiting the spatial properties of STDP. We show that a LEGO robot controlled by the SNN can exhibit classical and operant conditioning. Competition of spike-conducting pathways in the SNN plays a fundamental role in establishing associations of neural connections. It replaces the irrelevant associations by new ones in response to a change in stimuli. Thus, the robot gets the ability to relearn when the environment changes. The proposed SNN and the stimulation protocol can be further enhanced and tested in developing neuronal cultures, and also admit the use of memristive devices for hardware implementation.
| S-EPMC7054464 | biostudies-literature
Project description:We present a novel photonic chip design for high bandwidth four-degree optical switches that support high-dimensional switching mechanisms with low insertion loss and low crosstalk in a low power consumption level and a short switching time. Such four-degree photonic chips can be used to build an integrated full-grid Photonic-on-Chip Network (PCN). With four distinct input/output directions, the proposed photonic chips are superior compared to the current bidirectional photonic switches, where a conventionally sizable PCN can only be constructed as a linear chain of bidirectional chips. Our four-directional photonic chips are more flexible and scalable for the design of modern optical switches, enabling the construction of multi-dimensional photonic chip networks that are widely applied for intra-chip communication networks and photonic data centers. More noticeably, our photonic networks can be self-controlling with our proposed Multi-Sample Discovery model, a deep reinforcement learning model based on Proximal Policy Optimization. On a PCN, we can optimize many criteria such as transmission loss, power consumption, and routing time, while preserving performance and scaling up the network with dynamic changes. Experiments on simulated data demonstrate the effectiveness and scalability of the proposed architectural design and optimization algorithm. Perceivable insights make the constructed architecture become the self-controlling photonic-on-chip networks.
| S-EPMC8632908 | biostudies-literature
Project description:The proposed architecture applies the principle of predictive coding and deep learning in a brain-inspired approach to robotic sensorimotor control. It is composed of many layers each of which is a recurrent network. The component networks can be spontaneously active due to the homeokinetic learning rule, a principle that has been studied previously for the purpose of self-organized generation of behavior. We present robotic simulations that illustrate the function of the network and show evidence that deeper networks enable more complex exploratory behavior.
| S-EPMC7523181 | biostudies-literature
Project description:In silico analysis of biological activity data has become an essential technique in pharmaceutical development. Specifically, the so-called proteochemometric models aim to share information between targets in machine learning ligand-target activity prediction models. However, bioactivity data sets used in proteochemometric modeling are usually imbalanced, which could potentially affect the performance of the models. In this work, we explored the effect of different balancing strategies in deep learning proteochemometric target-compound activity classification models while controlling for the compound series bias through clustering. These strategies were (1) no_resampling, (2) resampling_after_clustering, (3) resampling_before_clustering, and (4) semi_resampling. These schemas were evaluated in kinases, GPCRs, nuclear receptors, and proteases from BindingDB. We observed that the predicted proportion of positives was driven by the actual data balance in the test set. Additionally, it was confirmed that data balance had an impact on the performance estimates of the proteochemometric model. We recommend a combination of data augmentation and clustering in the training set (semi_resampling) to mitigate the data imbalance effect in a realistic scenario. The code of this analysis is publicly available at https://github.com/b2slab/imbalance_pcm_benchmark.
| S-EPMC8594867 | biostudies-literature
Project description:While the influence of alkyl chain length and headgroup size on self-assembly behaviour has been well-established for simple surfactants, the rational control over the pH- and concentration-dependent self-assembly behaviour in stimuli responsive peptides remains an elusive goal. Here, we show that different amphiphilic peptides can have similar self-assembly phase diagrams, providing the relative strengths of the attractive and repulsive forces are balanced. Using palmitoyl-YYAAEEEEK(DO3A:Gd)-NH2 and palmitoyl-YAAEEEEK(DO3A:Gd)-NH2 as controls, we show that reducing hydrophobic attractive forces through fewer methylene groups in the alkyl chain will lead to a similar self-assembly phase diagram as increasing the electrostatic repulsive forces via the addition of a glutamic acid residue. These changes allow creation of self-assembled MRI vehicles with slightly different micelle and nanofiber diameters but with minimal changes in the spin-lattice T1 relaxivity. These findings reveal a powerful strategy to design self-assembled vehicles with different sizes but with similar self-assembly profiles.
| S-EPMC6432923 | biostudies-other
Project description:PurposeTo develop a model-guided self-supervised deep learning MRI reconstruction framework called reference-free latent map extraction (RELAX) for rapid quantitative MR parameter mapping.MethodsTwo physical models are incorporated for network training in RELAX, including the inherent MR imaging model and a quantitative model that is used to fit parameters in quantitative MRI. By enforcing these physical model constraints, RELAX eliminates the need for full sampled reference data sets that are required in standard supervised learning. Meanwhile, RELAX also enables direct reconstruction of corresponding MR parameter maps from undersampled k-space. Generic sparsity constraints used in conventional iterative reconstruction, such as the total variation constraint, can be additionally included in the RELAX framework to improve reconstruction quality. The performance of RELAX was tested for accelerated T1 and T2 mapping in both simulated and actually acquired MRI data sets and was compared with supervised learning and conventional constrained reconstruction for suppressing noise and/or undersampling-induced artifacts.ResultsIn the simulated data sets, RELAX generated good T1 /T2 maps in the presence of noise and/or undersampling artifacts, comparable to artifact/noise-free ground truth. The inclusion of a spatial total variation constraint helps improve image quality. For the in vivo T1 /T2 mapping data sets, RELAX achieved superior reconstruction quality compared with conventional iterative reconstruction, and similar reconstruction performance to supervised deep learning reconstruction.ConclusionThis work has demonstrated the initial feasibility of rapid quantitative MR parameter mapping based on self-supervised deep learning. The RELAX framework may also be further extended to other quantitative MRI applications by incorporating corresponding quantitative imaging models.
| S-EPMC9185837 | biostudies-literature
Project description:The discovery of chemical reactions is an inherently unpredictable and time-consuming process1. An attractive alternative is to predict reactivity, although relevant approaches, such as computer-aided reaction design, are still in their infancy2. Reaction prediction based on high-level quantum chemical methods is complex3, even for simple molecules. Although machine learning is powerful for data analysis4,5, its applications in chemistry are still being developed6. Inspired by strategies based on chemists' intuition7, we propose that a reaction system controlled by a machine learning algorithm may be able to explore the space of chemical reactions quickly, especially if trained by an expert8. Here we present an organic synthesis robot that can perform chemical reactions and analysis faster than they can be performed manually, as well as predict the reactivity of possible reagent combinations after conducting a small number of experiments, thus effectively navigating chemical reaction space. By using machine learning for decision making, enabled by binary encoding of the chemical inputs, the reactions can be assessed in real time using nuclear magnetic resonance and infrared spectroscopy. The machine learning system was able to predict the reactivity of about 1,000 reaction combinations with accuracy greater than 80 per cent after considering the outcomes of slightly over 10 per cent of the dataset. This approach was also used to calculate the reactivity of published datasets. Further, by using real-time data from our robot, these predictions were followed up manually by a chemist, leading to the discovery of four reactions.
| S-EPMC6223543 | biostudies-literature
Project description:A control system for bipedal walking in the sagittal plane was developed in simulation. The biped model was built based on anthropometric data for a 1.8 m tall male of average build. At the core of the controller is a deep deterministic policy gradient (DDPG) neural network that was trained in GAZEBO, a physics simulator, to predict the ideal foot placement to maintain stable walking despite external disturbances. The complexity of the DDPG network was decreased through carefully selected state variables and a distributed control system. Additional controllers for the hip joints during their stance phases and the ankle joint during toe-off phase help to stabilize the biped during walking. The simulated biped can walk at a steady pace of approximately 1 m/s, and during locomotion it can maintain stability with a 30 kg·m/s impulse applied forward on the torso or a 40 kg·m/s impulse applied rearward. It also maintains stable walking with a 10 kg backpack or a 25 kg front pack. The controller was trained on a 1.8 m tall model, but also stabilizes models 1.4-2.3 m tall with no changes.
| S-EPMC6477666 | biostudies-literature
Project description:ObjectiveTo develop an end-to-end deep learning framework based on a protein-protein interaction (PPI) network to make synergistic anticancer drug combination predictions.Materials and methodsWe propose a deep learning framework named Graph Convolutional Network for Drug Synergy (GraphSynergy). GraphSynergy adapts a spatial-based Graph Convolutional Network component to encode the high-order topological relationships in the PPI network of protein modules targeted by a pair of drugs, as well as the protein modules associated with a specific cancer cell line. The pharmacological effects of drug combinations are explicitly evaluated by their therapy and toxicity scores. An attention component is also introduced in GraphSynergy, which aims to capture the pivotal proteins that play a part in both PPI network and biomolecular interactions between drug combinations and cancer cell lines.ResultsGraphSynergy outperforms the classic and state-of-the-art models in predicting synergistic drug combinations on the 2 latest drug combination datasets. Specifically, GraphSynergy achieves accuracy values of 0.7553 (11.94% improvement compared to DeepSynergy, the latest published drug combination prediction algorithm) and 0.7557 (10.95% improvement compared to DeepSynergy) on DrugCombDB and Oncology-Screen datasets, respectively. Furthermore, the proteins allocated with high contribution weights during the training of GraphSynergy are proved to play a role in view of molecular functions and biological processes, such as transcription and transcription regulation.ConclusionThe introduction of topological relations between drug combination and cell line within the PPI network can significantly improve the capability of synergistic drug combination identification.
| S-EPMC8510276 | biostudies-literature