Dataset Information

Data Efficient Reinforcement Learning for Integrated Lateral Planning and Control in Automated Parking System.

ABSTRACT: Reinforcement learning (RL) is a promising direction in automated parking systems (APSs), as integrating planning and tracking control using RL can potentially maximize the overall performance. However, commonly used model-free RL requires many interactions to achieve acceptable performance, and model-based RL in APS cannot continuously learn. In this paper, a data-efficient RL method is constructed to learn from data by use of a model-based method. The proposed method uses a truncated Monte Carlo tree search to evaluate parking states and select moves. Two artificial neural networks are trained to provide the search probability of each tree branch and the final reward for each state using self-trained data. The data efficiency is enhanced by weighting exploration with parking trajectory returns, an adaptive exploration scheme, and experience augmentation with imaginary rollouts. Without human demonstrations, a novel training pipeline is also used to train the initial action guidance network and the state value network. Compared with path planning and path-following methods, the proposed integrated method can flexibly co-ordinate the longitudinal and lateral motion to park a smaller parking space in one maneuver. Its adaptability to changes in the vehicle model is verified by joint Carsim and MATLAB simulation, demonstrating that the algorithm converges within a few iterations. Finally, experiments using a real vehicle platform are used to further verify the effectiveness of the proposed method. Compared with obtaining rewards using simulation, the proposed method achieves a better final parking attitude and success rate.

SUBMITTER: Song S

PROVIDER: S-EPMC7766926 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Data Efficient Reinforcement Learning for Integrated Lateral Planning and Control in Automated Parking System.

Song Shaoyu S Chen Hui H Sun Hongwei H Liu Meicen M

Sensors (Basel, Switzerland) 20201218 24

Reinforcement learning (RL) is a promising direction in automated parking systems (APSs), as integrating planning and tracking control using RL can potentially maximize the overall performance. However, commonly used model-free RL requires many interactions to achieve acceptable performance, and model-based RL in APS cannot continuously learn. In this paper, a data-efficient RL method is constructed to learn from data by use of a model-based method. The proposed method uses a truncated Monte Car ...[more]

PMID: 33353153

Similar Datasets

Project description:Purpose To develop and evaluate an integrated solution for automatic intensity-modulated radiation therapy (IMRT) planning in non-small-cell lung cancer (NSCLC) cases. Methods A novel algorithm named as multi-objectives adjustment policy network (MOAPN) was proposed and trained to learn how to adjust multiple optimization objectives in commercial Eclipse treatment planning system (TPS), based on the multi-agent deep reinforcement learning (DRL) scheme. Furthermore, a three-dimensional (3D) dose prediction module was developed to generate the patient-specific initial optimization objectives to reduce the overall exploration space during MOAPN training. 114 previously treated NSCLC cases suitable for stereotactic body radiotherapy (SBRT) were selected from the clinical database. 87 cases were used for the model training, and the remaining 27 cases for evaluating the feasibility and effectiveness of MOAPN in automatic treatment planning. Results For all tested cases, the average number of adjustment steps was 21 ± 5.9 (mean ± 1 standard deviation). Compared with the MOAPN initial plans, the actual dose of chest wall, spinal cord, heart, lung (affected side), esophagus and bronchus in the MOAPN final plans reduced by 14.5%, 11.6%, 4.7%, 16.7%, 1.6% and 7.7%, respectively. The dose result of OARs in the MOAPN final plans was similar to those in the clinical plans. The complete automatic treatment plan for a new case was generated based on the integrated solution, with about 5-6 min. Conclusion We successfully developed an integrated solution for automatic treatment planning. Using the 3D dose prediction module to obtain the patient-specific optimization objectives, MOAPN formed action-value policy can simultaneously adjust multiple objectives to obtain a high-quality plan in a shorter time. This integrated solution contributes to improving the efficiency of the overall planning workflow and reducing the variation of plan quality in different regions and treatment centers. Although improvement is warranted, this proof-of-concept study has demonstrated the feasibility of this integrated solution in automatic treatment planning based on the Eclipse TPS.

Project description:BackgroundThe identification of the electrical stimulation parameters for neuromodulation is a subject-specific and time-consuming procedure that presently mostly relies on the expertise of the user (e.g., clinician, experimenter, bioengineer). Since the parameters of stimulation change over time (due to displacement of electrodes, skin status, etc.), patients undergo recurrent, long calibration sessions, along with visits to the clinics, which are inefficient and expensive. To address this issue, we developed an automatized calibration system based on reinforcement learning (RL) allowing for accurate and efficient identification of the peripheral nerve stimulation parameters for somatosensory neuroprostheses.MethodsWe developed an RL algorithm to automatically select neurostimulation parameters for restoring sensory feedback with transcutaneous electrical nerve stimulation (TENS). First, the algorithm was trained offline on a dataset comprising 49 subjects. Then, the neurostimulation was then integrated with a graphical user interface (GUI) to create an intuitive AI-based mapping platform enabling the user to autonomously perform the sensation characterization procedure. We assessed the algorithm against the performance of both experienced and naïve and of a brute force algorithm (BFA), on 15 nerves from five subjects. Then, we validated the AI-based platform on six neuropathic nerves affected by distal sensory loss.ResultsOur automatized approach demonstrated the ability to find the optimal values of neurostimulation achieving reliable and comfortable elicited sensations. When compared to alternatives, RL outperformed the naïve and BFA, significantly decreasing the time for mapping and the number of delivered stimulation trains, while improving the overall quality. Furthermore, the RL algorithm showed performance comparable to trained experimenters. Finally, we exploited it successfully for eliciting sensory feedback in neuropathic patients.ConclusionsOur findings demonstrated that the AI-based platform based on a RL algorithm can automatically and efficiently calibrate parameters for somatosensory nerve stimulation. This holds promise to avoid experts' employment in similar scenarios, thanks to the merging between AI and neurotech. Our RL algorithm has the potential to be used in other neuromodulation fields requiring a mapping process of the stimulation parameters.Trial registrationClinicalTrial.gov (Identifier: NCT04217005).

Project description:Reinforcement learning (RL) is a powerful and flexible paradigm for searching for solutions in high-dimensional action spaces. However, bridging the gap between playing computer games with thousands of simulated episodes and solving real scientific problems with complex and involved environments (up to actual laboratory experiments) requires improvements in terms of sample efficiency to make the most of expensive information. The discovery of new drugs is a major commercial application of RL, motivated by the very large nature of the chemical space and the need to perform multiparameter optimization (MPO) across different properties. In silico methods, such as virtual library screening (VS) and de novo molecular generation with RL, show great promise in accelerating this search. However, incorporation of increasingly complex computational models in these workflows requires increasing sample efficiency. Here, we introduce an active learning system linked with an RL model (RL-AL) for molecular design, which aims to improve the sample-efficiency of the optimization process. We identity and characterize unique challenges combining RL and AL, investigate the interplay between the systems, and develop a novel AL approach to solve the MPO problem. Our approach greatly expedites the search for novel solutions relative to baseline-RL for simple ligand- and structure-based oracle functions, with a 5-66-fold increase in hits generated for a fixed oracle budget and a 4-64-fold reduction in computational time to find a specific number of hits. Furthermore, compounds discovered through RL-AL display substantial enrichment of a multi-parameter scoring objective, indicating superior efficacy in curating high-scoring compounds, without a reduction in output diversity. This significant acceleration improves the feasibility of oracle functions that have largely been overlooked in RL due to high computational costs, for example free energy perturbation methods, and in principle is applicable to any RL domain.

Dataset Information

Data Efficient Reinforcement Learning for Integrated Lateral Planning and Control in Automated Parking System.

Publications

Data Efficient Reinforcement Learning for Integrated Lateral Planning and Control in Automated Parking System.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets