A Minimally Constrained Environment for the Study of Cooperation

Abstract

We describe a simple environment to study cooperation between two agents and a method of achieving cooperation in that environment. The environment consists of randomly generated normal form games with uniformly distributed payoffs. Agents play multiple games against each other, each game drawn independently from the random distribution. This environment provides a good model of the difficulties of cooperating in an ever changing world. Tit-for-Tat cannot be used because moves are not labeled as "cooperate" or "defect", fictitious play cannot be used because the agent never sees the same game twice, and approaches suitable for stochastic games cannot be used because the set of states is not finite. Our agent identifies cooperative moves by assigning an attitude to its opponent and to itself. The attitude determines how much a player values its opponents payoff, i.e how much the player is willing to deviate from strictly self-interested behavior. To cooperate, our agent estimates the attitude of its opponent by observing its moves and reciprocates by setting its own attitude accordingly. We show how the opponent's attitude can be estimated using a particle filter, even when the opponent is changing its attitude.