On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location

• Main Authors: Kayhan, Osman Semih, van Gemert, Jan C
• Format: Journal Article
• Language: English
• Published: 16-03-2020
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Learning
• DOI: 10.48550/arxiv.2003.07064

In this paper we challenge the common assumption that convolutional layers in
modern CNNs are translation invariant. We show that CNNs can and will exploit
the absolute spatial location by learning filters that respond exclusively to
particular absolute locations by exploiting image boundary effects. Because
modern CNNs filters have a huge receptive field, these boundary effects operate
even far from the image boundary, allowing the network to exploit absolute
spatial location all over the image. We give a simple solution to remove
spatial location encoding which improves translation invariance and thus gives
a stronger visual inductive bias which particularly benefits small data sets.
We broadly demonstrate these benefits on several architectures and various
applications such as image classification, patch matching, and two video
classification datasets.