Random Features for Compositional Kernels

• Main Authors: Daniely, Amit, Frostig, Roy, Gupta, Vineet, Singer, Yoram
• Format: Journal Article
• Language: English
• Published: 22-03-2017
• Subjects: Computer Science - Learning
• DOI: 10.48550/arxiv.1703.07872

We describe and analyze a simple random feature scheme (RFS) from prescribed
compositional kernels. The compositional kernels we use are inspired by the
structure of convolutional neural networks and kernels. The resulting scheme
yields sparse and efficiently computable features. Each random feature can be
represented as an algebraic expression over a small number of (random) paths in
a composition tree. Thus, compositional random features can be stored
compactly. The discrete nature of the generation process enables de-duplication
of repeated features, further compacting the representation and increasing the
diversity of the embeddings. Our approach complements and can be combined with
previous random feature schemes.