Graph Positional and Structural Encoder

• Main Authors: Cantürk, Semih, Liu, Renming, Lapointe-Gagné, Olivier, Létourneau, Vincent, Wolf, Guy, Beaini, Dominique, Rampášek, Ladislav
• Format: Journal Article
• Language: English
• Published: 13-07-2023
• Subjects: Computer Science - Learning
• DOI: 10.48550/arxiv.2307.07107

Positional and structural encodings (PSE) enable better identifiability of
nodes within a graph, rendering them essential tools for empowering modern
GNNs, and in particular graph Transformers. However, designing PSEs that work
optimally for all graph prediction tasks is a challenging and unsolved problem.
Here, we present the Graph Positional and Structural Encoder (GPSE), the
first-ever graph encoder designed to capture rich PSE representations for
augmenting any GNN. GPSE learns an efficient common latent representation for
multiple PSEs, and is highly transferable: The encoder trained on a particular
graph dataset can be used effectively on datasets drawn from markedly different
distributions and modalities. We show that across a wide range of benchmarks,
GPSE-enhanced models can significantly outperform those that employ explicitly
computed PSEs, and at least match their performance in others. Our results pave
the way for the development of foundational pre-trained graph encoders for
extracting positional and structural information, and highlight their potential
as a more powerful and efficient alternative to explicitly computed PSEs and
existing self-supervised pre-training approaches. Our framework and pre-trained
models are publicly available at https://github.com/G-Taxonomy-Workgroup/GPSE.
For convenience, GPSE has also been integrated into the PyG library to
facilitate downstream applications.