Graph-based Neural Acceleration for Nonnegative Matrix Factorization

• Main Authors: Sjölund, Jens, Bånkestad, Maria
• Format: Journal Article
• Language: English
• Published: 01-02-2022
• Subjects: Computer Science - Learning; Computer Science - Numerical Analysis; Mathematics - Numerical Analysis; Mathematics - Optimization and Control; Statistics - Machine Learning
• DOI: 10.48550/arxiv.2202.00264

We describe a graph-based neural acceleration technique for nonnegative
matrix factorization that builds upon a connection between matrices and
bipartite graphs that is well-known in certain fields, e.g., sparse linear
algebra, but has not yet been exploited to design graph neural networks for
matrix computations. We first consider low-rank factorization more broadly and
propose a graph representation of the problem suited for graph neural networks.
Then, we focus on the task of nonnegative matrix factorization and propose a
graph neural network that interleaves bipartite self-attention layers with
updates based on the alternating direction method of multipliers. Our empirical
evaluation on synthetic and two real-world datasets shows that we attain
substantial acceleration, even though we only train in an unsupervised fashion
on smaller synthetic instances.