Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Reinforcement learning produces dominant strategies for the Iterated Prisoner's Dilemma.

ABSTRACT: We present tournament results and several powerful strategies for the Iterated Prisoner's Dilemma created using reinforcement learning techniques (evolutionary and particle swarm algorithms). These strategies are trained to perform well against a corpus of over 170 distinct opponents, including many well-known and classic strategies. All the trained strategies win standard tournaments against the total collection of other opponents. The trained strategies and one particular human made designed strategy are the top performers in noisy tournaments also.

SUBMITTER: Harper M

PROVIDER: S-EPMC5724862 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Reinforcement learning produces dominant strategies for the Iterated Prisoner's Dilemma.

Harper Marc M Knight Vincent V Jones Martin M Koutsovoulos Georgios G Glynatsi Nikoleta E NE Campbell Owen O

PloS one 20171211 12

We present tournament results and several powerful strategies for the Iterated Prisoner's Dilemma created using reinforcement learning techniques (evolutionary and particle swarm algorithms). These strategies are trained to perform well against a corpus of over 170 distinct opponents, including many well-known and classic strategies. All the trained strategies win standard tournaments against the total collection of other opponents. The trained strategies and one particular human made designed s ...[more]

PMID: 29228001

Similar Datasets

Misperception influence on zero-determinant strategies in iterated Prisoner's Dilemma.

Project description:Zero-determinant (ZD) strategies have attracted wide attention in Iterated Prisoner's Dilemma (IPD) games, since the player equipped with ZD strategies can unilaterally enforce the two players' expected utilities subjected to a linear relation. On the other hand, uncertainties, which may be caused by misperception, occur in IPD inevitably in practical circumstances. To better understand the situation, we consider the influence of misperception on ZD strategies in IPD, where the two players, player X and player Y, have different cognitions, but player X detects the misperception and it is believed to make ZD strategies by player Y. We provide a necessary and sufficient condition for the ZD strategies in IPD with misperception, where there is also a linear relationship between players' utilities in player X's cognition. Then we explore bounds of players' expected utility deviation from a linear relationship in player X's cognition with also improving its own utility.

| S-EPMC8956668 | biostudies-literature

Critical dynamics in the evolution of stochastic strategies for the iterated prisoner's dilemma.

Project description:The observed cooperation on the level of genes, cells, tissues, and individuals has been the object of intense study by evolutionary biologists, mainly because cooperation often flourishes in biological systems in apparent contradiction to the selfish goal of survival inherent in Darwinian evolution. In order to resolve this paradox, evolutionary game theory has focused on the Prisoner's Dilemma (PD), which incorporates the essence of this conflict. Here, we encode strategies for the iterated Prisoner's Dilemma (IPD) in terms of conditional probabilities that represent the response of decision pathways given previous plays. We find that if these stochastic strategies are encoded as genes that undergo Darwinian evolution, the environmental conditions that the strategies are adapting to determine the fixed point of the evolutionary trajectory, which could be either cooperation or defection. A transition between cooperative and defective attractors occurs as a function of different parameters such as mutation rate, replacement rate, and memory, all of which affect a player's ability to predict an opponent's behavior. These results imply that in populations of players that can use previous decisions to plan future ones, cooperation depends critically on whether the players can rely on facing the same strategies that they have adapted to. Defection, on the other hand, is the optimal adaptive response in environments that change so quickly that the information gathered from previous plays cannot usefully be integrated for a response.

| S-EPMC2951343 | biostudies-literature

Evolution of extortion in Iterated Prisoner's Dilemma games.

Project description:Iterated games are a fundamental component of economic and evolutionary game theory. They describe situations where two players interact repeatedly and have the ability to use conditional strategies that depend on the outcome of previous interactions, thus allowing for reciprocation. Recently, a new class of strategies has been proposed, so-called "zero-determinant" strategies. These strategies enforce a fixed linear relationship between one's own payoff and that of the other player. A subset of those strategies allows "extortioners" to ensure that any increase in one player's own payoff exceeds that of the other player by a fixed percentage. Here, we analyze the evolutionary performance of this new class of strategies. We show that in reasonably large populations, they can act as catalysts for the evolution of cooperation, similar to tit-for-tat, but that they are not the stable outcome of natural selection. In very small populations, however, extortioners hold their ground. Extortion strategies do particularly well in coevolutionary arms races between two distinct populations. Significantly, they benefit the population that evolves at the slower rate, an example of the so-called "Red King" effect. This may affect the evolution of interactions between host species and their endosymbionts.

| S-EPMC3637695 | biostudies-literature

Extortion can outperform generosity in the iterated prisoner's dilemma.

Project description:Zero-determinant (ZD) strategies, as discovered by Press and Dyson, can enforce a linear relationship between a pair of players' scores in the iterated prisoner's dilemma. Particularly, the extortionate ZD strategies can enforce and exploit cooperation, providing a player with a score advantage, and consequently higher scores than those from either mutual cooperation or generous ZD strategies. In laboratory experiments in which human subjects were paired with computer co-players, we demonstrate that both the generous and the extortionate ZD strategies indeed enforce a unilateral control of the reward. When the experimental setting is sufficiently long and the computerized nature of the opponent is known to human subjects, the extortionate strategy outperforms the generous strategy. Human subjects' cooperation rates when playing against extortionate and generous ZD strategies are similar after learning has occurred. More than half of extortionate strategists finally obtain an average score higher than that from mutual cooperation.

| S-EPMC4832060 | biostudies-literature

From extortion to generosity, evolution in the Iterated Prisoner's Dilemma.

Project description:Recent work has revealed a new class of "zero-determinant" (ZD) strategies for iterated, two-player games. ZD strategies allow a player to unilaterally enforce a linear relationship between her score and her opponent's score, and thus to achieve an unusual degree of control over both players' long-term payoffs. Although originally conceived in the context of classical two-player game theory, ZD strategies also have consequences in evolving populations of players. Here, we explore the evolutionary prospects for ZD strategies in the Iterated Prisoner's Dilemma (IPD). Several recent studies have focused on the evolution of "extortion strategies," a subset of ZD strategies, and have found them to be unsuccessful in populations. Nevertheless, we identify a different subset of ZD strategies, called "generous ZD strategies," that forgive defecting opponents but nonetheless dominate in evolving populations. For all but the smallest population sizes, generous ZD strategies are not only robust to being replaced by other strategies but can selectively replace any noncooperative ZD strategy. Generous strategies can be generalized beyond the space of ZD strategies, and they remain robust to invasion. When evolution occurs on the full set of all IPD strategies, selection disproportionately favors these generous strategies. In some regimes, generous strategies outperform even the most successful of the well-known IPD strategies, including win-stay-lose-shift.

| S-EPMC3780848 | biostudies-literature

The effect of psychopathy on cooperative strategies in an iterated Prisoner's Dilemma experiment with emotional feedback.

Project description:As decision-making research becomes more popular, the inclusion of personality traits has emerged as a focal point for an exhaustive analysis of human behaviour. In this study, we investigate the impact of psychopathic traits on cooperation in an iterated Prisoner's Dilemma game with emotional facial feedback. Firstly, we observed how receiving a facial feedback after each decision affected players with different psychopathic trait scores, and how being informed about the opponent's identity influenced cooperative behaviour. Secondly, we analysed the strategies adopted by each player, and how these choices were correlated with their psychopathic traits. Although our results showed no effect of different emotional content in the feedback on cooperation, we observed more cooperative behaviours in those players who were told their opponent was another fellow human, compared to those who were told it was a computer. Moreover, fearless dominance had a very small but consistent negative effect on overall cooperation and on the tendency to maintain cooperative behaviours. We also found that players' personality scores affected the strategies they chose to play throughout the game. Hence, our experiment adds complexity to the body of work investigating psychopathic traits and social interactions, considering not only the environment of facial feedback but also the role of deception in experimental games.

| S-EPMC6381135 | biostudies-literature

From rationality to cooperativeness: The totally mixed Nash equilibrium in Markov strategies in the iterated Prisoner's Dilemma.

Project description:In this research, the social behavior of the participants in a Prisoner's Dilemma laboratory game is explained on the basis of the quantal response equilibrium concept and the representation of the game in Markov strategies. In previous research, we demonstrated that social interaction during the experiment has a positive influence on cooperation, trust, and gratefulness. This research shows that the quantal response equilibrium concept agrees only with the results of experiments on cooperation in Prisoner's Dilemma prior to social interaction. However, quantal response equilibrium does not explain of participants' behavior after social interaction. As an alternative theoretical approach, an examination was conducted of iterated Prisoner's Dilemma game in Markov strategies. We built a totally mixed Nash equilibrium in this game; the equilibrium agrees with the results of the experiments both before and after social interaction.

| S-EPMC5708610 | biostudies-literature

Recognising and evaluating the effectiveness of extortion in the Iterated Prisoner's Dilemma.

Project description:Establishing and maintaining mutual cooperation in agent-to-agent interactions can be viewed as a question of direct reciprocity and readily applied to the Iterated Prisoner's Dilemma. Agents cooperate, at a small cost to themselves, in the hope of obtaining a future benefit. Zero-determinant strategies, introduced in 2012, have a subclass of strategies that are provably extortionate. In the established literature, most of the studies of the effectiveness or lack thereof, of zero-determinant strategies is done by placing some zero-determinant strategy in a specific scenario (collection of agents) and evaluating its performance either numerically or theoretically. Extortionate strategies are algebraically rigid and memory-one by definition, and requires complete knowledge of a strategy (the memory-one cooperation probabilities). The contribution of this work is a method to detect extortionate behaviour from the history of play of an arbitrary strategy. This inverts the paradigm of most studies: instead of observing the effectiveness of some theoretically extortionate strategies, the largest known collection of strategies will be observed and their intensity of extortion quantified empirically. Moreover, we show that the lack of adaptability of extortionate strategies extends via this broader definition.

| S-EPMC11280246 | biostudies-literature

Win-Stay-Lose-Shift as a self-confirming equilibrium in the iterated Prisoner's Dilemma.

Project description:Evolutionary game theory assumes that players replicate a highly scored player's strategy through genetic inheritance. However, when learning occurs culturally, it is often difficult to recognize someone's strategy just by observing the behaviour. In this work, we consider players with memory-one stochastic strategies in the iterated Prisoner's Dilemma, with an assumption that they cannot directly access each other's strategy but only observe the actual moves for a certain number of rounds. Based on the observation, the observer has to infer the resident strategy in a Bayesian way and chooses his or her own strategy accordingly. By examining the best-response relations, we argue that players can escape from full defection into a cooperative equilibrium supported by Win-Stay-Lose-Shift in a self-confirming manner, provided that the cost of cooperation is low and the observational learning supplies sufficiently large uncertainty.

| S-EPMC8242928 | biostudies-literature

Oxytocin and vasopressin modulation of prisoner's dilemma strategies.

Project description:BackgroundThe neuropeptides oxytocin and vasopressin have been repeatedly implicated in social decision making by enhancing social salience and, generally, cooperation. The iterated and sequential version of the prisoner's dilemma (PD) game is a social dilemma paradigm eliciting strategies of cooperation versus competition.AimsWe aimed to characterise the role of PD players' sex, game partner type (computer vs. human) and oxytocin or vasopressin inhalation on the player's strategy preference.MethodsParticipants (153 men; 151 women) were randomised to intranasal 24 IU oxytocin, 20 IU vasopressin or placebo, double-blind, and played the PD. We examined main and interactive effects of sex, drug and partner type on strategy preference.ResultsWe found a pervasive preference for a tit-for-tat strategy (i.e. general sensitivity to the partner's choices) over unconditional cooperation, particularly when against a human rather than a computer partner. Oxytocin doubled this sensitivity in women (i.e. the preference for tit-for-tat over unconditional cooperation strategies) when playing against computers, which suggests a tendency to anthropomorphise them, and doubled women's unconditional cooperation preference when playing against humans. Vasopressin doubled sensitivity to the partner's previous choices (i.e. for tit-for-tat over unconditional cooperation) across sexes and partner types.ConclusionsThese findings suggest that women may be more sensitive to oxytocin's social effects of anthropomorphism of non-humans and of unconditional cooperation with humans, which may be consistent with evolutionary pressures for maternal care, and that vasopressin, irrespective of sex and partner type, may be generally sensitising humans to others' behaviour.

| S-EPMC7583454 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data