Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification

• Main Authors: Edwards, Aleksandra, Ushio, Asahi, Camacho-Collados, Jose, de Ribaupierre, Hélène, Preece, Alun
• Format: Journal Article
• Language: English
• Published: 17-11-2021
• Subjects: Computer Science - Computation and Language
• DOI: 10.48550/arxiv.2111.09064

Data augmentation techniques are widely used for enhancing the performance of
machine learning models by tackling class imbalance issues and data sparsity.
State-of-the-art generative language models have been shown to provide
significant gains across different NLP tasks. However, their applicability to
data augmentation for text classification tasks in few-shot settings have not
been fully explored, especially for specialised domains. In this paper, we
leverage GPT-2 (Radford A et al, 2019) for generating artificial training
instances in order to improve classification performance. Our aim is to analyse
the impact the selection process of seed training examples have over the
quality of GPT-generated samples and consequently the classifier performance.
We perform experiments with several seed selection strategies that, among
others, exploit class hierarchical structures and domain expert selection. Our
results show that fine-tuning GPT-2 in a handful of label instances leads to
consistent classification improvements and outperform competitive baselines.
Finally, we show that guiding this process through domain expert selection can
lead to further improvements, which opens up interesting research avenues for
combining generative models and active learning.