Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search

• Main Authors: Light, Jonathan, Cai, Min, Chen, Weiqin, Wang, Guanzhi, Chen, Xiusi, Cheng, Wei, Yue, Yisong, Hu, Ziniu
• Format: Journal Article
• Language: English
• Published: 20-08-2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language
• DOI: 10.48550/arxiv.2408.10635

In this paper, we propose a new method STRATEGIST that utilizes LLMs to
acquire new skills for playing multi-agent games through a self-improvement
process. Our method gathers quality feedback through self-play simulations with
Monte Carlo tree search and LLM-based reflection, which can then be used to
learn high-level strategic skills such as how to evaluate states that guide the
low-level execution. We showcase how our method can be used in both action
planning and dialogue generation in the context of games, achieving good
performance on both tasks. Specifically, we demonstrate that our method can
help train agents with better performance than both traditional reinforcement
learning-based approaches and other LLM-based skill learning approaches in
games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.
STRATEGIST helps bridge the gap between foundation models and symbolic
decision-making methods through its bi-level approach, leading to more robust
decision-making.