Algebraic Machine Learning

• Main Authors: Martin-Maroto, Fernando, de Polavieja, Gonzalo G
• Format: Journal Article
• Language: English
• Published: 14-03-2018
• Subjects: Computer Science - Discrete Mathematics; Computer Science - Learning; Mathematics - Commutative Algebra; Mathematics - Rings and Algebras
• DOI: 10.48550/arxiv.1803.05252

Machine learning algorithms use error function minimization to fit a large
set of parameters in a preexisting model. However, error minimization
eventually leads to a memorization of the training dataset, losing the ability
to generalize to other datasets. To achieve generalization something else is
needed, for example a regularization method or stopping the training when error
in a validation dataset is minimal. Here we propose a different approach to
learning and generalization that is parameter-free, fully discrete and that
does not use function minimization. We use the training data to find an
algebraic representation with minimal size and maximal freedom, explicitly
expressed as a product of irreducible components. This algebraic representation
is shown to directly generalize, giving high accuracy in test data, more so the
smaller the representation. We prove that the number of generalizing
representations can be very large and the algebra only needs to find one. We
also derive and test a relationship between compression and error rate. We give
results for a simple problem solved step by step, hand-written character
recognition, and the Queens Completion problem as an example of unsupervised
learning. As an alternative to statistical learning, algebraic learning may
offer advantages in combining bottom-up and top-down information, formal
concept derivation from data and large-scale parallelization.