Melioration Learning in Two-Person Games.
Ontology highlight
ABSTRACT:
SUBMITTER: Zschache J
PROVIDER: S-EPMC5112854 | biostudies-literature | 2016
REPOSITORIES: biostudies-literature
Publications
Melioration learning is an empirically well-grounded model of reinforcement learning. By means of computer simulations, this paper derives predictions for several repeatedly played two-person games from this model. The results indicate a likely convergence to a pure Nash equilibrium of the game. If no pure equilibrium exists, the relative frequencies of choice may approach the predictions of the mixed Nash equilibrium. Yet in some games, no stable state is reached. ...[more]
Similar Datasets
Project description:Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.
| S-EPMC3459907 | biostudies-literature
Project description:Game theory is the standard tool used to model strategic interactions in evolutionary biology and social science. Traditionally, game theory studies the equilibria of simple games. However, is this useful if the game is complicated, and if not, what is? We define a complicated game as one with many possible moves, and therefore many possible payoffs conditional on those moves. We investigate two-person games in which the players learn based on a type of reinforcement learning called experience-weighted attraction (EWA). By generating games at random, we characterize the learning dynamics under EWA and show that there are three clearly separated regimes: (i) convergence to a unique fixed point, (ii) a huge multiplicity of stable fixed points, and (iii) chaotic behavior. In case (iii), the dimension of the chaotic attractors can be very high, implying that the learning dynamics are effectively random. In the chaotic regime, the total payoffs fluctuate intermittently, showing bursts of rapid change punctuated by periods of quiescence, with heavy tails similar to what is observed in fluid turbulence and financial markets. Our results suggest that, at least for some learning algorithms, there is a large parameter regime for which complicated strategic interactions generate inherently unpredictable behavior that is best described in the language of dynamical systems theory.
| S-EPMC3557065 | biostudies-literature
Project description:We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple 'unbiased' reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual's behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is 'rational'.
| S-EPMC7864454 | biostudies-literature
Project description:The gaze behavior in sports and other applied settings has been studied for more than 20 years. A common finding is related to the "quiet eye" (QE), predicting that the duration of the last fixation before a critical event is associated with higher performance. Unlike previous studies conducted in applied settings with mobile eye trackers, we investigate the QE in a context similar to esport, in which participants click the mouse to hit targets presented on a computer screen under different levels of cognitive load. Simultaneously, eye and mouse movements were tracked using a high-end remote eye tracker at 300 Hz. Consistent with previous studies, we found that longer QE fixations were associated with higher performance. Increasing the cognitive load delayed the onset of the QE fixation, but had no significant influence on the QE duration. We discuss the implications of our results in the context of how the QE is defined, the quality of the eye-tracker data, and the type of analysis applied to QE data.
| S-EPMC8606425 | biostudies-literature
Project description:We propose a method for learning multi-agent policies to compete against multiple opponents. The method consists of recurrent neural network-based actor-critic networks and deterministic policy gradients that promote cooperation between agents by communication. The learning process does not require access to opponents' parameters or observations because the agents are trained separately from the opponents. The actor networks enable the agents to communicate using forward and backward paths while the critic network helps to train the actors by delivering them gradient signals based on their contribution to the global reward. Moreover, to address nonstationarity due to the evolving of other agents, we propose approximate model learning using auxiliary prediction networks for modeling the state transitions, reward function, and opponent behavior. In the test phase, we use competitive multi-agent environments to demonstrate by comparison the usefulness and superiority of the proposed method in terms of learning efficiency and goal achievements. The comparison results show that the proposed method outperforms the alternatives.
| S-EPMC6739057 | biostudies-literature
Project description:BackgroundSpecific learning difficulties (SpLD) include several disorders such as dyslexia, dyscalculia, and dysgraphia, and the children with these SpLD receive special education. However, the studies and the educational material so far focus mainly on one specific disorder.ObjectiveThis study's primary goal is to develop comprehensive training material for different types of SpLD, with five serious games addressing different aspects of the SpLD. The second focus is measuring the impact of adaptive difficulty level adjustment in the children's and their educators' usability and technology acceptance perception. Receiving feedback from the children and their educators, and refining the games according to their suggestions have also been essential in this two-phase study.MethodsA total of 10 SpLD educators and 23 children with different types of SpLD tested the prototypes of the five serious games (ie, Word game, Memory game, Category game, Space game, and Math game), gave detailed feedback, answered the System Usability Scale and Technology Acceptance Model (TAM) questionnaires, and applied think-aloud protocols during game play.ResultsThe games' standard and adaptive versions were analyzed in terms of average playtime and the number of false answers. Detailed analyses of the interviews, with word clouds and player performances, were also provided. The TAM questionnaires' average and mean values and box plots of each data acquisition session for the children and the educators were also reported via System Usability Scale and TAM questionnaires. The TAM results of the educators had an average of 8.41 (SD 0.87) out of 10 in the first interview and an average of 8.71 (SD 0.64) out of 10 in the second interview. The children had an average of 9.07 (SD 0.56) out of 10 in the first interview.ConclusionsBoth the educators and the children with SpLD enjoyed playing the games, gave positive feedback, and suggested new ways for improvement. The results showed that these games provide thorough training material for different types of SpLD with personalized and tailored difficulty systems. The final version of the proposed games will become a part of the special education centers' supplementary curriculum and training materials, making new enhancements and improvements possible in the future.
| S-EPMC8204245 | biostudies-literature
Project description:People are often generous even when it is not in their apparent self-interest to do so as demonstrated by numerous experiments using the dictator game (DG). More recent research using DGs has varied the number of dictators and recipients and used these games to investigate the bystander effect and congestible altruism. These studies have found that people are less generous when there are others who could be generous (the bystander effect) and also less generous when there are multiple recipients (congestible altruism) though the sum of their generosity to all recipients increases. A working hypothesis is proposed that the context-sensitive generosity observed in n-person DGs can be explained as equitable behavior. From an evolutionary perspective, explaining the evolution of equitable behavior is challenging at best. To provide an evolutionary explanation, a second working hypothesis is proposed: equitable offers evolve because they reduce resource deficits produced by variability in the accumulation of resources and thereby minimize the length of interbirth intervals (IBIs) and increase fitness. Based on this working hypothesis, an evolutionary model was developed for n-person DGs to investigate the evolution of equitable offers as a resource allocation problem when reproduction is constrained by IBIs. Simulations demonstrated that equitable offers could evolve in group-structured populations when there is a cost (i.e., longer IBIs) to running resource deficits. Mean evolved offers also varied as a function of the number of dictators and recipients in patterns consistent with the bystander effect and congestible altruism. Equitable offers evolved because they reduced resource variability among group members and thereby reduced resource deficits, which insured higher average rates of reproduction for more equitable groups of agents. Implications of these results are discussed.
| S-EPMC8324864 | biostudies-literature
Project description:Repeated games have provided an explanation of how mutual cooperation can be achieved even if defection is more favourable in a one-shot game in the Prisoner's Dilemma situation. Recently found zero-determinant (ZD) strategies have substantially been investigated in evolutionary game theory. The original memory-one ZD strategies unilaterally enforce linear relationships between average pay-offs of players. Here, we extend the concept of ZD strategies to memory-two strategies in repeated games. Memory-two ZD strategies unilaterally enforce linear relationships between correlation functions of pay-offs and pay-offs of the previous round. Examples of memory-two ZD strategy in the repeated Prisoner's Dilemma game are provided, some of which generalize the tit-for-tat strategy to a memory-two case. Extension of ZD strategies to memory-n case with n ≥ ~2 is also straightforward.
| S-EPMC8150048 | biostudies-literature
Project description:The discriminative parts of people's appearance play a significant role in their re-identification across non overlapping camera views. However, just focusing on the discriminative or attention regions without catering the contextual information does not always help. It is more important to learn the attention with reference to their spatial locations in context of the whole image. Current person re-identification (re-id) approaches either use separate modules or classifiers to learn both of these; the attention and its context, resulting in highly expensive person re-id solutions. In this work, instead of handling attentions and the context separately, we employ a unified attention and context mapping (ACM) block within the convolutional layers of network, without any additional computational resources overhead. The ACM block captures the attention regions as well as the relevant contextual information in a stochastic manner and enriches the final person representations for robust person re-identification. We evaluate the proposed method on 04 public benchmarks of person re-identification i.e., Market1501, DukeMTMC-Reid, CUHK03 and MSMT17 and find that the ACM block consistently improves the performance of person re-identification over the baseline networks.
| S-EPMC8114801 | biostudies-literature
Project description:Action video game play benefits performance in an array of sensory, perceptual, and attentional tasks that go well beyond the specifics of game play [1-9]. That a training regimen may induce improvements in so many different skills is notable because the majority of studies on training-induced learning report improvements on the trained task but limited transfer to other, even closely related, tasks ([10], but see also [11-13]). Here we ask whether improved probabilistic inference may explain such broad transfer. By using a visual perceptual decision making task [14, 15], the present study shows for the first time that action video game experience does indeed improve probabilistic inference. A neural model of this task [16] establishes how changing a single parameter, namely the strength of the connections between the neural layer providing the momentary evidence and the layer integrating the evidence over time, captures improvements in action-gamers behavior. These results were established in a visual, but also in a novel auditory, task, indicating generalization across modalities. Thus, improved probabilistic inference provides a general mechanism for why action video game playing enhances performance in a wide variety of tasks. In addition, this mechanism may serve as a signature of training regimens that are likely to produce transfer of learning.
| S-EPMC2956114 | biostudies-literature