Dataset Information

Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures.

ABSTRACT: Predictions and predictive knowledge have seen recent success in improving not only robot control but also other applications ranging from industrial process control to rehabilitation. A property that makes these predictive approaches well-suited for robotics is that they can be learned online and incrementally through interaction with the environment. However, a remaining challenge for many prediction-learning approaches is an appropriate choice of prediction-learning parameters, especially parameters that control the magnitude of a learning machine's updates to its predictions (the learning rates or step sizes). Typically, these parameters are chosen based on an extensive parameter search-an approach that neither scales well nor is well-suited for tasks that require changing step sizes due to non-stationarity. To begin to address this challenge, we examine the use of online step-size adaptation using the Modular Prosthetic Limb: a sensor-rich robotic arm intended for use by persons with amputations. Our method of choice, Temporal-Difference Incremental Delta-Bar-Delta (TIDBD), learns and adapts step sizes on a feature level; importantly, TIDBD allows step-size tuning and representation learning to occur at the same time. As a first contribution, we show that TIDBD is a practical alternative for classic Temporal-Difference (TD) learning via an extensive parameter search. Both approaches perform comparably in terms of predicting future aspects of a robotic data stream, but TD only achieves comparable performance with a carefully hand-tuned learning rate, while TIDBD uses a robust meta-parameter and tunes its own learning rates. Secondly, our results show that for this particular application TIDBD allows the system to automatically detect patterns characteristic of sensor failures common to a number of robotic applications. As a third contribution, we investigate the sensitivity of classic TD and TIDBD with respect to the initial step-size values on our robotic data set, reaffirming the robustness of TIDBD as shown in previous papers. Together, these results promise to improve the ability of robotic devices to learn from interactions with their environments in a robust way, providing key capabilities for autonomous agents and robots.

SUBMITTER: Gunther J

PROVIDER: S-EPMC7805647 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures.

Günther Johannes J Ady Nadia M NM Kearney Alex A Dawson Michael R MR Pilarski Patrick M PM

Frontiers in robotics and AI 20200313

Predictions and predictive knowledge have seen recent success in improving not only robot control but also other applications ranging from industrial process control to rehabilitation. A property that makes these predictive approaches well-suited for robotics is that they can be learned online and incrementally through interaction with the environment. However, a remaining challenge for many prediction-learning approaches is an appropriate choice of prediction-learning parameters, especially par ...[more]

PMID: 33501202

Similar Datasets

Project description:BackgroundIncremental hemodialysis, a strategy to individualize dialysis prescription based on residual kidney function, may be associated with enhanced quality of life and decreased health care costs compared with conventional hemodialysis.ObjectiveWe surveyed practicing Canadian nephrologists to assess knowledge, perceptions, and practice pattern on the use of incremental hemodialysis.Design/settingWe distributed a cross-sectional, web-based survey. We asked about incremental hemodialysis prescribing practices, including frequency of prescription, clinical factors used to determine suitability for treatment, and barriers to implementation. The survey was conducted from September 21 to October 30, 2020.ParticipantsWe distributed the survey to practicing Canadian nephrologists identified from a private membership list of the Canadian Society of Nephrology (CSN), as well as to nephrologists named on a publicly available national list of practicing Canadian nephrologists created from provincial College of Physician registries. These were samples of convenience.MethodsWe conducted descriptive analysis of categorical data including frequencies for nominal variables and measures of central tendency (mean) and dispersion (standard deviation) for ordinal variables. We used chi-square analysis to identify association between participant and practice characteristics and their opinions and attitudes toward incremental dialysis. We used simple thematic analysis on free-text responses on questions regarding the prescription of incremental hemodialysis, focusing on age and baseline management of cardiac and noncardiac comorbidities.ResultsThe response rate was 35% (243/691). Most (138/211, 65%) of the participants prescribed incremental hemodialysis using an individualized approach at the nephrologist's discretion. Most participants (200/203, 98%) did not report any policy for implementation. Residual urine output was identified as the most important factor for eligibility (112/172, 65%), followed by electrolyte stability (76/172, 44%) and patient goals of care (69/117, 40%). Most participants agreed that dialysis prescriptions should take residual kidney function into consideration; however, 74% of the participants disagreed with a statement that there was strong evidence supporting incremental hemodialysis. Barriers identified included patient safety, patient acceptance of dose escalation, and logistics of scheduling. Despite these barriers, 82% of participants felt that that incremental hemodialysis is feasible with their current resources and 78% agreed that with specific criteria, it is a safe option.LimitationsThe generalizability of our study is limited by its response rate of 35%; however, this is comparable with typical response rates seen in electronic surveys. Most participants practice in an academic setting, which may have introduced bias to the results.ConclusionsDespite the perception of limited evidence and a lack of guidance on implementation, incremental hemodialysis is frequently practiced by Canadian nephrologists. Barriers to implementation were identified, highlighting the need for research to guide practice.

Project description:Recently, increasing attention has been focused on the investigation of the human brain connectome that describes the patterns of structural and functional connectivity networks of the human brain. Many studies of the human connectome have demonstrated that the brain network follows a small-world topology with an intrinsically cohesive modular structure and includes several network hubs in the medial parietal regions. However, most of these studies have only focused on undirected connections between regions in which the directions of information flow are not taken into account. How the brain regions causally influence each other and how the directed network of human brain is topologically organized remain largely unknown. Here, we applied linear multivariate Granger causality analysis (GCA) and graph theoretical approaches to a resting-state functional MRI dataset with a large cohort of young healthy participants (n = 86) to explore connectivity patterns of the population-based whole-brain functional directed network. This directed brain network exhibited prominent small-world properties, which obviously improved previous results of functional MRI studies showing weak small-world properties in the directed brain networks in terms of a kernel-based GCA and individual analysis. This brain network also showed significant modular structures associated with 5 well known subsystems: fronto-parietal, visual, paralimbic/limbic, subcortical and primary systems. Importantly, we identified several driving hubs predominantly located in the components of the attentional network (e.g., the inferior frontal gyrus, supplementary motor area, insula and fusiform gyrus) and several driven hubs predominantly located in the components of the default mode network (e.g., the precuneus, posterior cingulate gyrus, medial prefrontal cortex and inferior parietal lobule). Further split-half analyses indicated that our results were highly reproducible between two independent subgroups. The current study demonstrated the directions of spontaneous information flow and causal influences in the directed brain networks, thus providing new insights into our understanding of human brain functional connectome.

Dataset Information

Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures.

Publications

Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets