Stability of learning dynamics in two-agent, imperfect-information games

Butterworth, John M. and Shapiro, Jonathan L. (2009) Stability of learning dynamics in two-agent, imperfect-information games. In: FOGA 09, January 9 - 11, 2009, Orlando, Florida USA.

PDF foga15-butterworth.pdf Download (444kB)

Abstract

One issue in multi-agent co-adaptive learning concerns convergence. When two (or more) agents play a game with different information and different payoffs, the general behaviour tends to be oscillation around a Nash equilibrium. Several algorithms have been proposed to force convergence to mixed-strategy Nash equilibria in imperfect-information games when the agents are aware of their opponent's strategy. We consider the effect on one such algorithm, the lagging anchor algorithm, when each agent must also infer the gradient information from observations, in the infinitesimal time-step limit. Use of an estimated gradient, either by opponent modelling or stochastic gradient ascent, destabilises the algorithm in a region of parameter space. There are two phases of behaviour. If the rate of estimation is low, the Nash equilibrium becomes unstable in the mean. If the rate is high, the Nash equilibrium is an attractive fixed point in the mean, but the uncertainty acts as narrow-band coloured noise, which causes dampened oscillations.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Reinforcement learning, game theory, learning in games, CICADA
Subjects:	MSC 2010, the AMS's Mathematics Subject Classification > 68 Computer science MSC 2010, the AMS's Mathematics Subject Classification > 91 Game theory, economics, social and behavioral sciences
Depositing User:	Dr Jonathan Shapiro
Date Deposited:	28 Apr 2009
Last Modified:	20 Oct 2017 14:12
URI:	https://eprints.maths.manchester.ac.uk/id/eprint/1261

Actions (login required)

View Item