Melioration Learning in Two-Person Games.
Ontology highlight
ABSTRACT:
SUBMITTER: Zschache J
PROVIDER: S-EPMC5112854 | biostudies-literature | 2016
REPOSITORIES: biostudies-literature
Publications
Melioration learning is an empirically well-grounded model of reinforcement learning. By means of computer simulations, this paper derives predictions for several repeatedly played two-person games from this model. The results indicate a likely convergence to a pure Nash equilibrium of the game. If no pure equilibrium exists, the relative frequencies of choice may approach the predictions of the mixed Nash equilibrium. Yet in some games, no stable state is reached. ...[more]
Similar Datasets
Project description:Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.
| S-EPMC3459907 | biostudies-literature
Project description:In contrast to traditional professional sports, there are few standardized metrics in professional esports (competitive multiplayer video games) for assessing a player's skill and ability. We assessed the performance of professional-level players in Aim LabTM, a first-person shooter training and assessment game, with two target-shooting tasks. These tasks differed primarily in target size: the task with large targets provided an incentive to be fast but imprecise and the task with large targets provided an incentive to be precise but slow. Each player's motor acuity was measured by characterizing the speed-accuracy trade-off in shot behavior: shot time (elapsed time for a player to shoot at a target) and shot spatial error (distance from center of a target). We also characterized the fine-grained kinematics of players' mouse movements. Our findings demonstrate that: 1) movement kinematics depended on task demands; 2) individual differences in motor acuity were significantly correlated with kinematics; and 3) performance, combined across the two target sizes, was poorly characterized by Fitts Law. Our approach to measuring motor acuity has widespread applications not only in esports assessment and training, but also in basic (motor psychophysics) and clinical (gamified rehabilitation) research.
| S-EPMC9744923 | biostudies-literature
Project description:Game theory is the standard tool used to model strategic interactions in evolutionary biology and social science. Traditionally, game theory studies the equilibria of simple games. However, is this useful if the game is complicated, and if not, what is? We define a complicated game as one with many possible moves, and therefore many possible payoffs conditional on those moves. We investigate two-person games in which the players learn based on a type of reinforcement learning called experience-weighted attraction (EWA). By generating games at random, we characterize the learning dynamics under EWA and show that there are three clearly separated regimes: (i) convergence to a unique fixed point, (ii) a huge multiplicity of stable fixed points, and (iii) chaotic behavior. In case (iii), the dimension of the chaotic attractors can be very high, implying that the learning dynamics are effectively random. In the chaotic regime, the total payoffs fluctuate intermittently, showing bursts of rapid change punctuated by periods of quiescence, with heavy tails similar to what is observed in fluid turbulence and financial markets. Our results suggest that, at least for some learning algorithms, there is a large parameter regime for which complicated strategic interactions generate inherently unpredictable behavior that is best described in the language of dynamical systems theory.
| S-EPMC3557065 | biostudies-literature
Project description:We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple 'unbiased' reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual's behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is 'rational'.
| S-EPMC7864454 | biostudies-literature
Project description:The gaze behavior in sports and other applied settings has been studied for more than 20 years. A common finding is related to the "quiet eye" (QE), predicting that the duration of the last fixation before a critical event is associated with higher performance. Unlike previous studies conducted in applied settings with mobile eye trackers, we investigate the QE in a context similar to esport, in which participants click the mouse to hit targets presented on a computer screen under different levels of cognitive load. Simultaneously, eye and mouse movements were tracked using a high-end remote eye tracker at 300 Hz. Consistent with previous studies, we found that longer QE fixations were associated with higher performance. Increasing the cognitive load delayed the onset of the QE fixation, but had no significant influence on the QE duration. We discuss the implications of our results in the context of how the QE is defined, the quality of the eye-tracker data, and the type of analysis applied to QE data.
| S-EPMC8606425 | biostudies-literature
Project description:Games have a long history as benchmarks for progress in artificial intelligence. Approaches using search and learning produced strong performance across many perfect information games, and approaches using game-theoretic reasoning and learning demonstrated strong performance for specific imperfect information poker variants. We introduce Student of Games, a general-purpose algorithm that unifies previous approaches, combining guided search, self-play learning, and game-theoretic reasoning. Student of Games achieves strong empirical performance in large perfect and imperfect information games-an important step toward truly general algorithms for arbitrary environments. We prove that Student of Games is sound, converging to perfect play as available computation and approximation capacity increases. Student of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold'em poker, and defeats the state-of-the-art agent in Scotland Yard, an imperfect information game that illustrates the value of guided search, learning, and game-theoretic reasoning.
| S-EPMC10651118 | biostudies-literature
Project description:We propose a method for learning multi-agent policies to compete against multiple opponents. The method consists of recurrent neural network-based actor-critic networks and deterministic policy gradients that promote cooperation between agents by communication. The learning process does not require access to opponents' parameters or observations because the agents are trained separately from the opponents. The actor networks enable the agents to communicate using forward and backward paths while the critic network helps to train the actors by delivering them gradient signals based on their contribution to the global reward. Moreover, to address nonstationarity due to the evolving of other agents, we propose approximate model learning using auxiliary prediction networks for modeling the state transitions, reward function, and opponent behavior. In the test phase, we use competitive multi-agent environments to demonstrate by comparison the usefulness and superiority of the proposed method in terms of learning efficiency and goal achievements. The comparison results show that the proposed method outperforms the alternatives.
| S-EPMC6739057 | biostudies-literature
Project description:Across many domains of interaction, both natural and artificial, individuals use past experience to shape future behaviors. The results of such learning processes depend on what individuals wish to maximize. A natural objective is one's own success. However, when two such "selfish" learners interact with each other, the outcome can be detrimental to both, especially when there are conflicts of interest. Here, we explore how a learner can align incentives with a selfish opponent. Moreover, we consider the dynamics that arise when learning rules themselves are subject to evolutionary pressure. By combining extensive simulations and analytical techniques, we demonstrate that selfish learning is unstable in most classical two-player repeated games. If evolution operates on the level of long-run payoffs, selection instead favors learning rules that incorporate social (other-regarding) preferences. To further corroborate these results, we analyze data from a repeated prisoner's dilemma experiment. We find that selfish learning is insufficient to explain human behavior when there is a trade-off between payoff maximization and fairness.
| S-EPMC9802390 | biostudies-literature
Project description:BackgroundSpecific learning difficulties (SpLD) include several disorders such as dyslexia, dyscalculia, and dysgraphia, and the children with these SpLD receive special education. However, the studies and the educational material so far focus mainly on one specific disorder.ObjectiveThis study's primary goal is to develop comprehensive training material for different types of SpLD, with five serious games addressing different aspects of the SpLD. The second focus is measuring the impact of adaptive difficulty level adjustment in the children's and their educators' usability and technology acceptance perception. Receiving feedback from the children and their educators, and refining the games according to their suggestions have also been essential in this two-phase study.MethodsA total of 10 SpLD educators and 23 children with different types of SpLD tested the prototypes of the five serious games (ie, Word game, Memory game, Category game, Space game, and Math game), gave detailed feedback, answered the System Usability Scale and Technology Acceptance Model (TAM) questionnaires, and applied think-aloud protocols during game play.ResultsThe games' standard and adaptive versions were analyzed in terms of average playtime and the number of false answers. Detailed analyses of the interviews, with word clouds and player performances, were also provided. The TAM questionnaires' average and mean values and box plots of each data acquisition session for the children and the educators were also reported via System Usability Scale and TAM questionnaires. The TAM results of the educators had an average of 8.41 (SD 0.87) out of 10 in the first interview and an average of 8.71 (SD 0.64) out of 10 in the second interview. The children had an average of 9.07 (SD 0.56) out of 10 in the first interview.ConclusionsBoth the educators and the children with SpLD enjoyed playing the games, gave positive feedback, and suggested new ways for improvement. The results showed that these games provide thorough training material for different types of SpLD with personalized and tailored difficulty systems. The final version of the proposed games will become a part of the special education centers' supplementary curriculum and training materials, making new enhancements and improvements possible in the future.
| S-EPMC8204245 | biostudies-literature
Project description:People are often generous even when it is not in their apparent self-interest to do so as demonstrated by numerous experiments using the dictator game (DG). More recent research using DGs has varied the number of dictators and recipients and used these games to investigate the bystander effect and congestible altruism. These studies have found that people are less generous when there are others who could be generous (the bystander effect) and also less generous when there are multiple recipients (congestible altruism) though the sum of their generosity to all recipients increases. A working hypothesis is proposed that the context-sensitive generosity observed in n-person DGs can be explained as equitable behavior. From an evolutionary perspective, explaining the evolution of equitable behavior is challenging at best. To provide an evolutionary explanation, a second working hypothesis is proposed: equitable offers evolve because they reduce resource deficits produced by variability in the accumulation of resources and thereby minimize the length of interbirth intervals (IBIs) and increase fitness. Based on this working hypothesis, an evolutionary model was developed for n-person DGs to investigate the evolution of equitable offers as a resource allocation problem when reproduction is constrained by IBIs. Simulations demonstrated that equitable offers could evolve in group-structured populations when there is a cost (i.e., longer IBIs) to running resource deficits. Mean evolved offers also varied as a function of the number of dictators and recipients in patterns consistent with the bystander effect and congestible altruism. Equitable offers evolved because they reduced resource variability among group members and thereby reduced resource deficits, which insured higher average rates of reproduction for more equitable groups of agents. Implications of these results are discussed.
| S-EPMC8324864 | biostudies-literature