Adaptive algorithms for shaping behavior.
Ontology highlight
ABSTRACT:
SUBMITTER: Tong WL
PROVIDER: S-EPMC10723287 | biostudies-literature | 2023 Dec
REPOSITORIES: biostudies-literature
Publications
Adaptive algorithms for shaping behavior.
Tong William L WL Iyer Anisha A Murthy Venkatesh N VN Reddy Gautam G
bioRxiv : the preprint server for biology 20231205
Dogs and laboratory mice are commonly trained to perform complex tasks by guiding them through a curriculum of simpler tasks ('shaping'). What are the principles behind effective shaping strategies? Here, we propose a machine learning framework for shaping animal behavior, where an autonomous teacher agent decides its student's task based on the student's transcript of successes and failures on previously assigned tasks. Using autonomous teachers that plan a curriculum in a common sequence learn ...[more]
Similar Datasets
Project description:The acts of learning and memory are thought to emerge from the modifications of synaptic connections between neurons, as guided by sensory feedback during behavior. However, much is unknown about how such synaptic processes can sculpt and are sculpted by neuronal population dynamics and an interaction with the environment. Here, we embodied a simulated network, inspired by dissociated cortical neuronal cultures, with an artificial animal (an animat) through a sensory-motor loop consisting of structured stimuli, detailed activity metrics incorporating spatial information, and an adaptive training algorithm that takes advantage of spike timing dependent plasticity. By using our design, we demonstrated that the network was capable of learning associations between multiple sensory inputs and motor outputs, and the animat was able to adapt to a new sensory mapping to restore its goal behavior: move toward and stay within a user-defined area. We further showed that successful learning required proper selections of stimuli to encode sensory inputs and a variety of training stimuli with adaptive selection contingent on the animat's behavior. We also found that an individual network had the flexibility to achieve different multi-task goals, and the same goal behavior could be exhibited with different sets of network synaptic strengths. While lacking the characteristic layered structure of in vivo cortical tissue, the biologically inspired simulated networks could tune their activity in behaviorally relevant manners, demonstrating that leaky integrate-and-fire neural networks have an innate ability to process information. This closed-loop hybrid system is a useful tool to study the network properties intermediating synaptic plasticity and behavioral adaptation. The training algorithm provides a stepping stone towards designing future control systems, whether with artificial neural networks or biological animats themselves.
| S-EPMC2265558 | biostudies-literature
Project description:The goal of this review is to describe in what ways feedback or adaptive stimulation may be delivered and adjusted based on relevant biomarkers. Specific treatment mechanisms underlying therapeutic brain stimulation remain unclear, in spite of the demonstrated efficacy in a number of nervous system diseases. Brain stimulation appears to exert widespread influence over specific neural networks that are relevant to specific disease entities. In awake patients, activation or suppression of these neural networks can be assessed by either symptom alleviation (i.e., tremor, rigidity, seizures) or physiological criteria, which may be predictive of expected symptomatic treatment. Secondary verification of network activation through specific biomarkers that are linked to symptomatic disease improvement may be useful for several reasons. For example, these biomarkers could aid optimal intraoperative localization, possibly improve efficacy or efficiency (i.e., reduced power needs), and provide long-term adaptive automatic adjustment of stimulation parameters. Possible biomarkers for use in portable or implanted devices span from ongoing physiological brain activity, evoked local field potentials (LFPs), and intermittent pathological activity, to wearable devices, biochemical, blood flow, optical, or magnetic resonance imaging (MRI) changes, temperature changes, or optogenetic signals. First, however, potential biomarkers must be correlated directly with symptom or disease treatment and network activation. Although numerous biomarkers are under consideration for a variety of stimulation indications the feasibility of these approaches has yet to be fully determined. Particularly, there are critical questions whether the use of adaptive systems can improve efficacy over continuous stimulation, facilitate adjustment of stimulation interventions and improve our understanding of the role of abnormal network function in disease mechanisms.
| S-EPMC5641319 | biostudies-other
Project description:This study presents a particle filter based framework to track cardiac surface from a time sequence of single magnetic resonance imaging (MRI) slices with the future goal of utilizing the presented framework for interventional cardiovascular magnetic resonance procedures, which rely on the accurate and online tracking of the cardiac surface from MRI data. The framework exploits a low-order parametric deformable model of the cardiac surface. A stochastic dynamic system represents the cardiac surface motion. Deformable models are employed to introduce shape prior to control the degree of the deformations. Adaptive filters are used to model complex cardiac motion in the dynamic model of the system. Particle filters are utilized to recursively estimate the current state of the system over time. The proposed method is applied to recover biventricular deformations and validated with a numerical phantom and multiple real cardiac MRI datasets. The algorithm is evaluated with multiple experiments using fixed and varying image slice planes at each time step. For the real cardiac MRI datasets, the average root-mean-square tracking errors of 2.61 mm and 3.42 mm are reported respectively for the fixed and varying image slice planes. This work serves as a proof-of-concept study for modeling and tracking the cardiac surface deformations via a low-order probabilistic model with the future goal of utilizing this method for the targeted interventional cardiac procedures under MR image guidance. For the real cardiac MRI datasets, the presented method was able to track the points-of-interests located on different sections of the cardiac surface within a precision of 3 pixels. The analyses show that the use of deformable cardiac surface tracking algorithm can pave the way for performing precise targeted intracardiac ablation procedures under MRI guidance. The main contributions of this work are twofold. First, it presents a framework for the tracking of whole cardiac surface from a time sequence of single image slices. Second, it employs adaptive filters to incorporate motion information in the tracking of nonrigid cardiac surface motion for temporal coherence.
| S-EPMC9877032 | biostudies-literature
Project description:BackgroundAdaptive therapy aims to tackle cancer drug resistance by leveraging resource competition between drug-sensitive and resistant cells. Here, we present a theoretical study of intra-tumoral competition during adaptive therapy, to investigate under which circumstances it will be superior to aggressive treatment.MethodsWe develop and analyse a simple, 2-D, on-lattice, agent-based tumour model in which cells are classified as fully drug-sensitive or resistant. Subsequently, we compare this model to its corresponding non-spatial ordinary differential equation model, and fit it to longitudinal prostate-specific antigen data from 65 prostate cancer patients undergoing intermittent androgen deprivation therapy following biochemical recurrence.ResultsLeveraging the individual-based nature of our model, we explicitly demonstrate competitive suppression of resistance during adaptive therapy, and examine how different factors, such as the initial resistance fraction or resistance costs, alter competition. This not only corroborates our theoretical understanding of adaptive therapy, but also reveals that competition of resistant cells with each other may play a more important role in adaptive therapy in solid tumours than was previously thought. To conclude, we present two case studies, which demonstrate the implications of our work for: (i) mathematical modelling of adaptive therapy, and (ii) the intra-tumoral dynamics in prostate cancer patients during intermittent androgen deprivation treatment, a precursor of adaptive therapy.ConclusionOur work shows that the tumour's spatial architecture is an important factor in adaptive therapy and provides insights into how adaptive therapy leverages both inter- and intra-specific competition to control resistance.
| S-EPMC9053239 | biostudies-literature
Project description:Environmental conditions during early life can adaptively shape the phenotype for the prevailing environment. Recently, it has been suggested that adolescence represents an additional temporal window for adaptive developmental plasticity, though supporting evidence is scarce. Previous work has shown that male guinea pigs living in large mixed-sex colonies develop a low-aggressive phenotype as part of a queuing strategy that is adaptive for integrating into large unfamiliar colonies. By contrast, males living in pairs during adolescence become highly aggressive towards strangers. Here, we tested whether the high-aggressive phenotype is adaptive under conditions of low population density, namely when directly competing with a single opponent for access to females. For that purpose, we established groups of one pair-housed male (PM), one colony-housed male (CM) and two females. PMs directed more aggression towards the male competitor and more courtship and mating towards females than did CMs. In consequence, PMs attained the dominant position in most cases and sired significantly more offspring. Moreover, they showed distinctly higher testosterone concentrations and elevated cortisol levels, which probably promoted enhanced aggressiveness while mobilizing necessary energy. Taken together, our results provide the clearest evidence to date for adaptive shaping of the phenotype by environmental influences during adolescence.
| S-EPMC5326539 | biostudies-literature
Project description:Animals can alter their foraging behavior through associative learning, where an encounter with an essential resource (e.g., food or a reproductive opportunity) is associated with nearby environmental cues (e.g., volatiles). This can subsequently improve the animal's foraging efficiency. However, when these associated cues are encountered again, the anticipated resource is not always present. Such an unrewarding experience, also called a memory-extinction experience, can change an animal's response to the associated cues. Although some studies are available on the mechanisms of this process, they rarely focus on cues and rewards that are relevant in an animal's natural habitat. In this study, we tested the effect of different types of ecologically relevant memory-extinction experiences on the conditioned plant volatile preferences of the parasitic wasp Cotesia glomerata that uses these cues to locate its caterpillar hosts. These extinction experiences consisted of contact with only host traces (frass and silk), contact with nonhost traces, or oviposition in a nonhost near host traces, on the conditioned plant species. Our results show that the lack of oviposition, after contacting host traces, led to the temporary alteration of the conditioned plant volatile preference in C. glomerata, but this effect was plant species-specific. These results provide novel insights into how ecologically relevant memory-extinction experiences can fine-tune an animal's foraging behavior. This fine-tuning of learned behavior can be beneficial when the lack of finding a resource accurately predicts current, but not future foraging opportunities. Such continuous reevaluation of obtained information helps animals to prevent maladaptive foraging behavior.
| S-EPMC8528537 | biostudies-literature
Project description:Measurements to estimate parameters of a model are commonplace in the physical sciences, where the traditional approach to automation is to use a sequence of preselected settings followed by least-squares fitting of a model function to the data. This measure-then-fit approach is simple and effective and entirely appropriate for many applications but when measurement resources are limited, efficiency becomes more important. To increase efficiency, Bayesian experiment design allows measurement settings to be chosen adaptively based on accumulated data and utility, the predicted improvement in results as a function of settings. However, the calculation of utility has been judged too impractical for most applications. In this paper, we introduce computational methods and simplified algorithms that accelerate utility calculations by over an order of magnitude, with only slight degradation in measurement efficiency. The methods eliminate utility calculation as a barrier to practical application of efficient adaptive measurement.
| S-EPMC9791979 | biostudies-literature
Project description:BackgroundWe have previously conducted computer-based tournaments to compare the yield of alternative approaches to deploying mobile HIV testing services in settings where the prevalence of undetected infection may be characterized by 'hotspots'. We report here on three refinements to our prior assessments and their implications for decision-making. Specifically, (1) enlarging the number of geographic zones; (2) including spatial correlation in the prevalence of undetected infection; and (3) evaluating a prospective search algorithm that accounts for such correlation.MethodsBuilding on our prior work, we used a simulation model to create a hypothetical city consisting of up to 100 contiguous geographic zones. Each zone was randomly assigned a prevalence of undetected HIV infection. We employed a user-defined weighting scheme to correlate infection levels between adjacent zones. Over 180 days, search algorithms selected a zone in which to conduct a fixed number of HIV tests. Algorithms were permitted to observe the results of their own prior testing activities and to use that information in choosing where to test in subsequent rounds. The algorithms were (1) Thompson sampling (TS), an adaptive Bayesian search strategy; (2) Besag York Mollié (BYM), a Bayesian hierarchical model; and (3) Clairvoyance, a benchmarking strategy with access to perfect information.ResultsOver 250 tournament runs, BYM detected 65.3% (compared to 55.1% for TS) of the cases identified by Clairvoyance. BYM outperformed TS in all sensitivity analyses, except when there was a small number of zones (i.e., 16 zones in a 4 × 4 grid), wherein there was no significant difference in the yield of the two strategies. Though settings of no, low, medium, and high spatial correlation in the data were examined, differences in these levels did not have a significant effect on the relative performance of BYM versus TS.ConclusionsBYM narrowly outperformed TS in our simulation, suggesting that small improvements in yield can be achieved by accounting for spatial correlation. However, the comparative simplicity with which TS can be implemented makes a field evaluation critical to understanding the practical value of either of these algorithms as an alternative to existing approaches for deploying HIV testing resources.
| S-EPMC6120098 | biostudies-literature
Project description:Monarch butterflies in North America migrate south each autumn, but the mechanisms that initiate their migratory flight remain incompletely understood. We investigated environmental, developmental, and genetic factors that contribute to directional flight by testing summer and autumn-generation monarchs in three flight simulators: two at ground level (with and without wind blockage) and a novel balloon-based system that raised butterflies 30 meters into the air. Monarchs reared under autumn-like conditions in a growth chamber during the summer were also tested to explore the influence of developmental cues. Autumn generation monarchs demonstrated significant southwestern flight orientation, observed exclusively in the balloon simulator, underscoring the importance of high-altitude flight for migratory behavior. Summer generation monarchs reared under autumn-like conditions displayed southward orientation, larger wing sizes, and partial reproductive diapause, indicating specific seasonal environmental cues that are sufficient to induce migratory traits. In contrast, a lab line of monarchs reared in captivity since 2016 exhibited diminished wing size and reduced orientation ability, even when raised outdoors in the autumn, consistent with a loss of migratory traits in the absence of migration. Surprisingly, butterflies in the balloon simulator tended to orient upwind, which suggests that wind may also serves as a directional cue during migration. These findings highlight the critical roles of altitude, wind, and environmental cues in monarch migration and validate the balloon flight simulator as a powerful tool for studying migratory behavior. This research advances our understanding of the initiation of monarch migration and informs strategies for conservation efforts amidst environmental change.
| S-EPMC11761016 | biostudies-literature
Project description:Recently, adaptive filtering algorithms were designed using hyperbolic functions, such as hyperbolic cosine and tangent function. However, most of those algorithms have few parameters that need to be set, and the adaptive estimation accuracy and convergence performance can be improved further. More importantly, the hyperbolic sine function has not been discussed. In this paper, a family of adaptive filtering algorithms is proposed using hyperbolic sine function (HSF) and inverse hyperbolic sine function (IHSF) function. Specifically, development of a robust adaptive filtering algorithm based on HSF, and extend the HSF algorithm to another novel adaptive filtering algorithm based on IHSF; then continue to analyze the computational complexity for HSF and IHSF; finally, validation of the analyses and superiority of the proposed algorithm via simulations. The HSF and IHSF algorithms can attain superior steady-state performance and stronger robustness in impulsive interference than several existing algorithms for different system identification scenarios, under Gaussian noise and impulsive interference, demonstrate the superior performance achieved by HSF and IHSF over existing adaptive filtering algorithms with different hyperbolic functions.
| S-EPMC8504740 | biostudies-literature