Speechworthy Instruction-tuned Language Models

• Main Authors: Cho, Hyundong, Jedema, Nicolaas, Ribeiro, Leonardo F. R, Sharma, Karishma, Szekely, Pedro, Moschitti, Alessandro, Janssen, Ruben, May, Jonathan
• Format: Journal Article
• Language: English
• Published: 22-09-2024
• Subjects: Computer Science - Artificial Intelligence
• DOI: 10.48550/arxiv.2409.14672

Current instruction-tuned language models are exclusively trained with
textual preference data and thus are often not aligned with the unique
requirements of other modalities, such as speech. To better align language
models with the speech domain, we explore (i) prompting strategies grounded in
radio-industry best practices and (ii) preference learning using a novel
speech-based preference data of 20K samples, generated with a wide spectrum of
prompts that induce varying dimensions of speech-suitability and labeled by
annotators who listen to response pairs. Both human and automatic evaluation
show that both prompting and preference learning increase the
speech-suitability of popular instruction-tuned LLMs. Interestingly, we find
that prompting and preference learning can be additive; combining them achieves
the best win rates in head-to-head comparison, resulting in responses that are
preferred or tied to the base model in 76.2% of comparisons on average. Lastly,
we share lexical, syntactical, and qualitative analyses to showcase how each
method contributes to improving the speech-suitability of generated responses.