Search Results - "Gallé, Matthias"

The Smallest Grammar Problem as Constituents Choice and Minimal Grammar Parsing by Carrascosa, Rafael, Coste, François, Gallé, Matthias, Infante-Lopez, Gabriel

“…The smallest grammar problem—namely, finding a smallest context-free grammar that generates exactly one sequence—is of practical and theoretical importance in…”
Get full text

The Rediscovery Hypothesis: Language Models Need to Meet Linguistics by Nikoulina, Vassilina, Tezekbayev, Maxat, Kozhakhmet, Nuradil, Babazhanova, Madina, Gallé, Matthias, Assylbekov, Zhenisbek

“…There is an ongoing debate in the NLP community whether modern language models contain linguistic knowledge, recovered through so-called probes. In this paper,…”
Get full text

xkcd-repeats: A new taxonomy of repeats defined by their context diversity by Gallé, Matthias, Tealdi, Matías

“…The context in which a substring appears is an important notion to identify – for example – its semantic meaning. However, existing definitions from…”
Get full text

BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model by Akiki, Christopher, Pistilli, Giada, Mieskes, Margot, Gallé, Matthias, Wolf, Thomas, Ilić, Suzana, Jernite, Yacine

“…The BigScience Workshop was a value-driven initiative that spanned one and half years of interdisciplinary research and culminated in the creation of ROOTS, a…”
Get full text

Searching for smallest grammars on large sequences and application to DNA by Carrascosa, Rafael, Coste, François, Gallé, Matthias, Infante-Lopez, Gabriel

“…Motivated by the inference of the structure of genomic sequences, we address here the smallest grammar problem. In previous work, we introduced a new…”
Get full text

LLMCRIT: Teaching Large Language Models to Use Criteria by Yuan, Weizhe, Liu, Pengfei, Gallé, Matthias

“…Humans follow criteria when they execute tasks, and these criteria are directly used to assess the quality of task completion. Therefore, having models learn…”
Get full text

What Can I Do Now? Guiding Users in a World of Automated Decisions by Gallé, Matthias

“…More and more processes governing our lives use in some part an automatic decision step, where -- based on a feature vector derived from an applicant -- an…”
Get full text

Improving Reward Models with Synthetic Critiques by Ye, Zihuiwen, Greenlee-Scott, Fraser, Bartolo, Max, Blunsom, Phil, Campos, Jon Ander, Gallé, Matthias

“…Reward models (RMs) play a critical role in aligning language models through the process of reinforcement learning from human feedback. RMs are trained to…”
Get full text

Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs by Ahmadian, Arash, Cremer, Chris, Gallé, Matthias, Fadaee, Marzieh, Kreutzer, Julia, Pietquin, Olivier, Üstün, Ahmet, Hooker, Sara

“…AI alignment in the shape of Reinforcement Learning from Human Feedback (RLHF) is increasingly treated as a crucial ingredient for high performance large…”
Get full text

Speeding Up Entmax by Tezekbayev, Maxat, Nikoulina, Vassilina, Gallé, Matthias, Assylbekov, Zhenisbek

“…Softmax is the de facto standard in modern neural networks for language processing when it comes to normalizing logits. However, by producing a dense…”
Get full text

Unsupervised and Distributional Detection of Machine-Generated Text by Gallé, Matthias, Rozen, Jos, Kruszewski, Germán, Elsahar, Hady

“…The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or…”
Get full text

Multilingual Unsupervised Neural Machine Translation with Denoising Adapters by Üstün, Ahmet, Bérard, Alexandre, Besacier, Laurent, Gallé, Matthias

“…We consider the problem of multilingual unsupervised machine translation, translating to and from languages that only have monolingual data by using auxiliary…”
Get full text

The Generalized Smallest Grammar Problem by Siyari, Payam, Gallé, Matthias

“…The Smallest Grammar Problem -- the problem of finding the smallest context-free grammar that generates exactly one given sequence -- has never been…”
Get full text

BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model by Akiki, Christopher, Pistilli, Giada, Mieskes, Margot, Gallé, Matthias, Wolf, Thomas, Ilić, Suzana, Jernite, Yacine

“…The BigScience Workshop was a value-driven initiative that spanned one and half years of interdisciplinary research and culminated in the creation of ROOTS, a…”
Get full text

On Leakage of Code Generation Evaluation Datasets by Matton, Alexandre, Sherborne, Tom, Aumiller, Dennis, Tommasone, Elena, Alizadeh, Milad, He, Jingyi, Ma, Raymond, Voisin, Maxime, Gilsenan-McMahon, Ellen, Gallé, Matthias

“…In this paper, we consider contamination by code generation test sets, in particular in their use in modern large language models. We discuss three possible…”
Get full text

Self-Supervised and Controlled Multi-Document Opinion Summarization by Elsahar, Hady, Coavoux, Maximin, Gallé, Matthias, Rozen, Jos

“…We address the problem of unsupervised abstractive summarization of collections of user generated reviews with self-supervision and control. We propose a…”
Get full text

The Rediscovery Hypothesis: Language Models Need to Meet Linguistics by Nikoulina, Vassilina, Tezekbayev, Maxat, Kozhakhmet, Nuradil, Babazhanova, Madina, Gallé, Matthias, Assylbekov, Zhenisbek

“…Journal of Artificial Intelligence Vol. 72 (2021) 1343-1384 There is an ongoing debate in the NLP community whether modern language models contain linguistic…”
Get full text

"Roles for the boys?" Mining cast lists for gender and role distributions over time by Radford, William, Gallé, Matthias

“…Film and television play an important role in popular culture, however studies that require watching and annotating video are time-consuming and expensive to…”
Get full text

Character-based NMT with Transformer by Gupta, Rohit, Besacier, Laurent, Dymetman, Marc, Gallé, Matthias

“…Character-based translation has several appealing advantages, but its performance is in general worse than a carefully tuned BPE baseline. In this paper we…”
Get full text

On the Evaluation of Machine Translation for Terminology Consistency by Alam, Md Mahfuz ibn, Anastasopoulos, Antonios, Besacier, Laurent, Cross, James, Gallé, Matthias, Koehn, Philipp, Nikoulina, Vassilina

“…As neural machine translation (NMT) systems become an important part of professional translator pipelines, a growing body of work focuses on combining NMT with…”
Get full text