Probing with Noise: Unpicking the Warp and Weft of Embeddings

• Main Authors: Klubička, Filip, Kelleher, John D
• Format: Journal Article
• Language: English
• Published: 21-10-2022
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language; Computer Science - Learning
• DOI: 10.48550/arxiv.2210.12206

Improving our understanding of how information is encoded in vector space can
yield valuable interpretability insights. Alongside vector dimensions, we argue
that it is possible for the vector norm to also carry linguistic information.
We develop a method to test this: an extension of the probing framework which
allows for relative intrinsic interpretations of probing results. It relies on
introducing noise that ablates information encoded in embeddings, grounded in
random baselines and confidence intervals. We apply the method to
well-established probing tasks and find evidence that confirms the existence of
separate information containers in English GloVe and BERT embeddings. Our
correlation analysis aligns with the experimental findings that different
encoders use the norm to encode different kinds of information: GloVe stores
syntactic and sentence length information in the vector norm, while BERT uses
it to encode contextual incongruity.