Finding path and cycle counting formulae in graphs with Deep Reinforcement Learning

• Main Authors: Piquenot, Jason, Bérar, Maxime, Héroux, Pierre, Ramel, Jean-Yves, Raveaux, Romain, Adam, Sébastien
• Format: Journal Article
• Language: English
• Published: 02-10-2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Formal Languages and Automata Theory
• DOI: 10.48550/arxiv.2410.01661

This paper presents Grammar Reinforcement Learning (GRL), a reinforcement
learning algorithm that uses Monte Carlo Tree Search (MCTS) and a transformer
architecture that models a Pushdown Automaton (PDA) within a context-free
grammar (CFG) framework. Taking as use case the problem of efficiently counting
paths and cycles in graphs, a key challenge in network analysis, computer
science, biology, and social sciences, GRL discovers new matrix-based formulas
for path/cycle counting that improve computational efficiency by factors of two
to six w.r.t state-of-the-art approaches. Our contributions include: (i) a
framework for generating gramformers that operate within a CFG, (ii) the
development of GRL for optimizing formulas within grammatical structures, and
(iii) the discovery of novel formulas for graph substructure counting, leading
to significant computational improvements.