Motor Synergies Measurement Reveals the Relevant Role of Variability in Reward-Based Learning.
Ontology highlight
ABSTRACT:
SUBMITTER: Caballero C
PROVIDER: S-EPMC8513037 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Binary feedback, providing information solely about task success or failure, can be sufficient to drive motor learning. While binary feedback can induce explicit adjustments in movement strategy, it remains unclear if this type of feedback also induce implicit learning. We examined this question in a center-out reaching task by gradually moving an invisible reward zone away from a visual target to a final rotation of 7.5° or 25° in a between-group design. Participants received binary feedback, indicating if the movement intersected the reward zone. By the end of the training, both groups modified their reach angle by about 95% of the rotation. We quantified implicit learning by measuring performance in a subsequent no-feedback aftereffect phase, in which participants were told to forgo any adopted movement strategies and reach directly to the visual target. The results showed a small, but robust (2-3°) aftereffect in both groups, highlighting that binary feedback elicits implicit learning. Notably, for both groups, reaches to two flanking generalization targets were biased in the same direction as the aftereffect. This pattern is at odds with the hypothesis that implicit learning is a form of use-dependent learning. Rather, the results suggest that binary feedback can be sufficient to recalibrate a sensorimotor map.
| S-EPMC10327077 | biostudies-literature
Project description:Binary feedback, providing information solely about task success or failure, can be sufficient to drive motor learning. While binary feedback can induce explicit adjustments in movement strategy, it remains unclear if this type of feedback also induces implicit learning. We examined this question in a center-out reaching task by gradually moving an invisible reward zone away from a visual target to a final rotation of 7.5° or 25° in a between-group design. Participants received binary feedback, indicating if the movement intersected the reward zone. By the end of the training, both groups modified their reach angle by about 95% of the rotation. We quantified implicit learning by measuring performance in a subsequent no-feedback aftereffect phase, in which participants were told to forgo any adopted movement strategies and reach directly to the visual target. The results showed a small, but robust (2-3°) aftereffect in both groups, highlighting that binary feedback elicits implicit learning. Notably, for both groups, reaches to two flanking generalization targets were biased in the same direction as the aftereffect. This pattern is at odds with the hypothesis that implicit learning is a form of use-dependent learning. Rather, the results suggest that binary feedback can be sufficient to recalibrate a sensorimotor map.
| S-EPMC10471724 | biostudies-literature
Project description:The addition of rewarding feedback to motor learning tasks has been shown to increase the retention of learning, spurring interest in its possible utility for rehabilitation. However, motor tasks using rewarding feedback have repeatedly been shown to lead to great interindividual variability in performance. Understanding the causes of such variability is vital for maximizing the potential benefits of reward-based motor learning. Thus, using a large human cohort of both sexes (n = 241), we examined whether spatial (SWM), verbal, and mental rotation (RWM) working memory capacity and dopamine-related genetic profiles were associated with performance in two reward-based motor tasks. The first task assessed the participant's ability to follow a slowly shifting reward region based on hit/miss (binary) feedback. The second task investigated the participant's capacity to preserve performance with binary feedback after adapting to the rotation with full visual feedback. Our results demonstrate that higher SWM is associated with greater success and an enhanced capacity to reproduce a successful motor action, measured as change in reach angle following reward. In contrast, higher RWM was predictive of an increased propensity to express an explicit strategy when required to make large reach angle adjustments. Therefore, SWM and RWM were reliable, but dissociable, predictors of success during reward-based motor learning. Change in reach direction following failure was also a strong predictor of success rate, although we observed no consistent relationship with working memory. Surprisingly, no dopamine-related genotypes predicted performance. Therefore, working memory capacity plays a pivotal role in determining individual ability in reward-based motor learning.SIGNIFICANCE STATEMENT Reward-based motor learning tasks have repeatedly been shown to lead to idiosyncratic behaviors that cause varying degrees of task success. Yet, the factors determining an individual's capacity to use reward-based feedback are unclear. Here, we assessed a wide range of possible candidate predictors, and demonstrate that domain-specific working memory plays an essential role in determining individual capacity to use reward-based feedback. Surprisingly, genetic variations in dopamine availability were not found to play a role. This is in stark contrast with seminal work in the reinforcement and decision-making literature, which show strong and replicated effects of the same dopaminergic genes in decision-making. Therefore, our results provide novel insights into reward-based motor learning, highlighting a key role for domain-specific working memory capacity.
| S-EPMC6867814 | biostudies-literature
Project description:Exploration in reward-based motor learning is observable in experimental data as increased variability. In order to quantify exploration, we compare three methods for estimating other sources of variability: sensorimotor noise. We use a task in which participants could receive stochastic binary reward feedback following a target-directed weight shift. Participants first performed six baseline blocks without feedback, and next twenty blocks alternating with and without feedback. Variability was assessed based on trial-to-trial changes in movement endpoint. We estimated sensorimotor noise by the median squared trial-to-trial change in movement endpoint for trials in which no exploration is expected. We identified three types of such trials: trials in baseline blocks, trials in the blocks without feedback, and rewarded trials in the blocks with feedback. We estimated exploration by the median squared trial-to-trial change following non-rewarded trials minus sensorimotor noise. As expected, variability was larger following non-rewarded trials than following rewarded trials. This indicates that our reward-based weight-shifting task successfully induced exploration. Most importantly, our three estimates of sensorimotor noise differed: the estimate based on rewarded trials was significantly lower than the estimates based on the two types of trials without feedback. Consequently, the estimates of exploration also differed. We conclude that the quantification of exploration depends critically on the type of trials used to estimate sensorimotor noise. We recommend the use of variability following rewarded trials.
| S-EPMC7117770 | biostudies-literature
Project description:Reward timing, that is, the delay after which reward is delivered following an action is known to strongly influence reinforcement learning. Here, we asked if reward timing could also modulate how people learn and consolidate new motor skills. In 60 healthy participants, we found that delaying reward delivery by a few seconds influenced motor learning. Indeed, training with a short reward delay (1 s) induced continuous improvements in performance, whereas a long reward delay (6 s) led to initially high learning rates that were followed by an early plateau in the learning curve and a lower performance at the end of training. Participants who learned the skill with a long reward delay also exhibited reduced overnight memory consolidation. Overall, our data show that reward timing affects the dynamics and consolidation of motor learning, a finding that could be exploited in future rehabilitation programs.
| S-EPMC9095742 | biostudies-literature
Project description:The frontopolar cortex (FPC) contributes to tracking the reward of alternative choices during decision making, as well as their reliability. Whether this FPC function extends to reward gradients associated with continuous movements during motor learning remains unknown. We used anodal transcranial direct current stimulation (tDCS) over the right FPC to investigate its role in reward-based motor learning. Nineteen healthy human participants practiced novel sequences of finger movements on a digital piano with corresponding auditory feedback. Their aim was to use trialwise reward feedback to discover a hidden performance goal along a continuous dimension: timing. We additionally modulated the contralateral motor cortex (left M1) activity, and included a control sham stimulation. Right FPC-tDCS led to faster learning compared to lM1-tDCS and sham through regulation of motor variability. Bayesian computational modelling revealed that in all stimulation protocols, an increase in the trialwise expectation of reward was followed by greater exploitation, as shown previously. Yet, this association was weaker in lM1-tDCS suggesting a less efficient learning strategy. The effects of frontopolar stimulation were dissociated from those induced by lM1-tDCS and sham, as motor exploration was more sensitive to inferred changes in the reward tendency (volatility). The findings suggest that rFPC-tDCS increases the sensitivity of motor exploration to updates in reward volatility, accelerating reward-based motor learning.
| S-EPMC8514446 | biostudies-literature
Project description:We investigated whether physical requirements and motor variability decreased over days in novices during a repetitive screwing task. Fifty-seven subjects performed one hour of repetitive screwing and fastening on three days, separated by 2⁻7 days. The average physical requirement and relative cycle-to-cycle variability (coefficient of variation, i.e., CV) were calculated from continuous recordings of electromyography of four arm muscles (biceps brachii, triceps brachii, flexor carpi radialis, extensor digitorum), forearm acceleration, and electrocardiography. Muscle activity levels, heart rate, and forearm acceleration decreased from day 1 to day 2 (range: ~4% to ~20%) and/or 3 (range: ~4% to ~28%). Not all muscles showed a similar pattern. Activity of the extensor digitorum and biceps brachii decreased already between days 1 and 2 (range: ~6% to ~13%), whereas activity of the flexor carpi radialis and triceps brachii decreased between days 1 and 3 (range: ~13% to ~20%). No changes in physical requirement were detected between days 2 and 3. Relative motor variability did not change across days, except that variability of forearm acceleration increased from day 1 to 3 (~5%). This study found consistent changes in physical requirements and indicated that several arm muscles show earlier decreases of muscular activity, like the extensor digitorum, compared to other body parts, like the flexor carpi radialis. Moreover, movement strategies may develop differently than muscle activation strategies, based on the different developments of physical requirements and motor variability. The development of physical requirements in industrial tasks is part of daily living and starts at task onset, highlighting the importance of task familiarization and the randomization of experimental conditions in scientific studies.
| S-EPMC6479693 | biostudies-literature
Project description:During active (or REM) sleep, infant mammals exhibit myoclonic twitches of skeletal muscles throughout the body, generating jerky, discrete movements of the distal limbs. Hundreds of thousands of limb twitches are produced daily, and sensory feedback from these movements is a substantial driver of infant brain activity, suggesting that they contribute to motor learning and sensorimotor integration. It is not known whether the production of twitches is random or spatiotemporally structured, or whether the patterning of twitching changes with age; such information is critical for understanding how twitches contribute to development.We used high-speed videography and 3D motion tracking to assess the spatiotemporal structure of twitching at forelimb joints in 2- and 8-day-old rats. At both ages, twitches exhibited highly structured spatiotemporal properties at multiple timescales, including synergistic and multijoint movements within and across forelimbs. Hierarchical cluster analysis and latent class analysis revealed developmental changes in twitching quantity and patterning. Critically, we found evidence for a selectionist process whereby movement patterns at the early age compete for retention and expression over development.These findings indicate that twitches are not produced randomly but are highly structured at multiple timescales. This structure has important implications for understanding brain and spinal mechanisms that produce twitching, and the role that sensory feedback from twitching plays in sensorimotor system development. We propose that twitches represent a heretofore-overlooked form of motor exploration that helps animals probe the biomechanics of their limbs, build motor synergies, and lay a foundation for complex, automatic, and goal-directed wake movements.
| S-EPMC3823644 | biostudies-literature
Project description:When learning a movement based on binary success information, one is more variable following failure than following success. Theoretically, the additional variability post-failure might reflect exploration of possibilities to obtain success. When average behavior is changing (as in learning), variability can be estimated from differences between subsequent movements. Can one estimate exploration reliably from such trial-to-trial changes when studying reward-based motor learning? To answer this question, we tried to reconstruct the exploration underlying learning as described by four existing reward-based motor learning models. We simulated learning for various learner and task characteristics. If we simply determined the additional change post-failure, estimates of exploration were sensitive to learner and task characteristics. We identified two pitfalls in quantifying exploration based on trial-to-trial changes. Firstly, performance-dependent feedback can cause correlated samples of motor noise and exploration on successful trials, which biases exploration estimates. Secondly, the trial relative to which trial-to-trial change is calculated may also contain exploration, which causes underestimation. As a solution, we developed the additional trial-to-trial change (ATTC) method. By moving the reference trial one trial back and subtracting trial-to-trial changes following specific sequences of trial outcomes, exploration can be estimated reliably for the three models that explore based on the outcome of only the previous trial. Since ATTC estimates are based on a selection of trial sequences, this method requires many trials. In conclusion, if exploration is a binary function of previous trial outcome, the ATTC method allows for a model-free quantification of exploration.
| S-EPMC8382626 | biostudies-literature
Project description:Rapid learning can be critical to ensure elite performance in a changing world or to recover basic movement after neural injuries. Recently it was shown that the variability of follow-through movements affects the rate of motor memory formation. Here we investigate if lead-in movement has a similar effect on learning rate. We hypothesized that both modality and variability of lead-in movement would play critical roles, with simulations suggesting that only changes in active lead-in variability would exhibit slower learning. We tested this experimentally using a two-movement paradigm, with either visual or active initial lead-in movements preceeding a second movement performed in a force field. As predicted, increasing active lead-in variability reduced the rate of motor adaptation, whereas changes in visual lead-in variability had little effect. This demonstrates that distinct neural tuning activity is induced by different lead-in modalities, subsequently influencing the access to, and switching between, distinct motor memories.
| S-EPMC5552870 | biostudies-other