Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity

• Main Authors: Garnelo, Marta, Czarnecki, Wojciech Marian, Liu, Siqi, Tirumala, Dhruva, Oh, Junhyuk, Gidel, Gauthier, van Hasselt, Hado, Balduzzi, David
• Format: Journal Article
• Language: English
• Published: 08-10-2021
• Subjects: Computer Science - Artificial Intelligence
• DOI: 10.48550/arxiv.2110.04041

Strategic diversity is often essential in games: in multi-player games, for
example, evaluating a player against a diverse set of strategies will yield a
more accurate estimate of its performance. Furthermore, in games with
non-transitivities diversity allows a player to cover several winning
strategies. However, despite the significance of strategic diversity, training
agents that exhibit diverse behaviour remains a challenge. In this paper we
study how to construct diverse populations of agents by carefully structuring
how individuals within a population interact. Our approach is based on
interaction graphs, which control the flow of information between agents during
training and can encourage agents to specialise on different strategies,
leading to improved overall performance. We provide evidence for the importance
of diversity in multi-agent training and analyse the effect of applying
different interaction graphs on the training trajectories, diversity and
performance of populations in a range of games. This is an extended version of
the long abstract published at AAMAS.