BERTian Poetics: Constrained Composition with Masked LMs

BERTian Poetics: Constrained Composition with Masked LMs

Masked language models have recently been interpreted as energy-based sequence models that can be generated from using a Metropolis--Hastings sampler. This short paper demonstrates how this can be instrumentalized for constrained composition and explores the poetics implied by such a usage. Our focu...

Full description

Saved in:
Bibliographic Details
Main Authors: Akiki, Christopher, Potthast, Martin
Format: Journal Article
Language:English
Published: 28-10-2021
Subjects:
Computer Science - Computation and Language
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Masked language models have recently been interpreted as energy-based sequence models that can be generated from using a Metropolis--Hastings sampler. This short paper demonstrates how this can be instrumentalized for constrained composition and explores the poetics implied by such a usage. Our focus on constraints makes it especially apt to understand the generated text through the poetics of the OuLiPo movement.
DOI:10.48550/arxiv.2110.15181