Search Results - "Heinzerling, Benjamin"

Examining the effect of whitening on static and contextualized word embeddings by Sasaki, Shota, Heinzerling, Benjamin, Suzuki, Jun, Inui, Kentaro

“…Static word embeddings (SWE) and contextualized word embeddings (CWE) are the foundation of modern natural language processing. However, these embeddings…”
Get full text

Monotonic Representation of Numeric Properties in Language Models by Heinzerling, Benjamin, Inui, Kentaro

“…Language models (LMs) can express factual knowledge involving numeric properties such as Karl Popper was born in 1902. However, how this information is encoded…”
Get full text

Representational Analysis of Binding in Language Models by Dai, Qin, Heinzerling, Benjamin, Inui, Kentaro

“…Entity tracking is essential for complex reasoning. To perform in-context entity tracking, language models (LMs) must bind an entity to its attribute (e.g.,…”
Get full text

Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries by Heinzerling, Benjamin, Inui, Kentaro

“…Pretrained language models have been suggested as a possible alternative or complement to structured knowledge bases. However, this emerging LM-as-KB paradigm…”
Get full text

Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction by Dai, Qin, Heinzerling, Benjamin, Inui, Kentaro

“…Bi-encoder architectures for distantly-supervised relation extraction are designed to make use of the complementary information found in text and knowledge…”
Get full text

The Curse of Popularity: Popular Entities have Catastrophic Side Effects when Deleting Knowledge from Language Models by Takahashi, Ryosuke, Kamoda, Go, Heinzerling, Benjamin, Sakaguchi, Keisuke, Inui, Kentaro

“…Language models (LMs) encode world knowledge in their internal parameters through training. However, LMs may learn personal and confidential information from…”
Get full text

ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluation by Brassard, Ana, Heinzerling, Benjamin, Kudo, Keito, Sakaguchi, Keisuke, Inui, Kentaro

“…Evaluating the quality of free-text explanations is a multifaceted, subjective, and labor-intensive task. Large language models (LLMs) present an appealing…”
Get full text

Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation by Heinzerling, Benjamin, Strube, Michael

“…Pretrained contextual and non-contextual subword embeddings have become available in over 250 languages, allowing massively multilingual NLP. However, while…”
Get full text

Test-time Augmentation for Factual Probing by Kamoda, Go, Heinzerling, Benjamin, Sakaguchi, Keisuke, Inui, Kentaro

“…Factual probing is a method that uses prompts to test if a language model "knows" certain world knowledge facts. A problem in factual probing is that small…”
Get full text

The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces by El-Shangiti, Ahmed Oumar, Hiraoka, Tatsuya, AlQuabeh, Hilal, Heinzerling, Benjamin, Inui, Kentaro

“…This paper investigates whether large language models (LLMs) utilize numerical attributes encoded in a low-dimensional subspace of the embedding space when…”
Get full text

Tracing and Manipulating Intermediate Values in Neural Math Problem Solvers by Matsumoto, Yuta, Heinzerling, Benjamin, Yoshikawa, Masashi, Inui, Kentaro

“…How language models process complex input that requires multiple steps of inference is not well understood. Previous research has shown that information about…”
Get full text

BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages by Heinzerling, Benjamin, Strube, Michael

“…We present BPEmb, a collection of pre-trained subword unit embeddings in 275 languages, based on Byte-Pair Encoding (BPE). In an evaluation using fine-grained…”
Get full text

COPA-SSE: Semi-structured Explanations for Commonsense Reasoning by Brassard, Ana, Heinzerling, Benjamin, Kavumba, Pride, Inui, Kentaro

“…We present Semi-Structured Explanations for COPA (COPA-SSE), a new crowdsourced dataset of 9,747 semi-structured, English common sense explanations for Choice…”
Get full text

Learning to Learn to be Right for the Right Reasons by Kavumba, Pride, Heinzerling, Benjamin, Brassard, Ana, Inui, Kentaro

“…Improving model generalization on held-out data is one of the core objectives in commonsense reasoning. Recent work has shown that models trained on the…”
Get full text

Fine-Grained Entity Typing in Hyperbolic Space by López, Federico, Heinzerling, Benjamin, Strube, Michael

“…How can we represent hierarchical information present in large type inventories for entity typing? We study the ability of hyperbolic embeddings to capture…”
Get full text

Riposte! A Large Corpus of Counter-Arguments by Reisert, Paul, Heinzerling, Benjamin, Inoue, Naoya, Kiyono, Shun, Inui, Kentaro

“…Constructive feedback is an effective method for improving critical thinking skills. Counter-arguments (CAs), one form of constructive feedback, have been…”
Get full text

Revisiting Selectional Preferences for Coreference Resolution by Heinzerling, Benjamin, Moosavi, Nafise Sadat, Strube, Michael

“…Selectional preferences have long been claimed to be essential for coreference resolution. However, they are mainly modeled only implicitly by current…”
Get full text

When Choosing Plausible Alternatives, Clever Hans can be Clever by Kavumba, Pride, Inoue, Naoya, Heinzerling, Benjamin, Singh, Keshav, Reisert, Paul, Inui, Kentaro

“…Pretrained language models, such as BERT and RoBERTa, have shown large improvements in the commonsense reasoning benchmark COPA. However, recent work found…”
Get full text

On the Importance of Subword Information for Morphological Tasks in Truly Low-Resource Languages by Zhu, Yi, Heinzerling, Benjamin, Vulić, Ivan, Strube, Michael, Reichart, Roi, Korhonen, Anna

“…Recent work has validated the importance of subword information for word representation learning. Since subwords increase parameter sharing ability in neural…”
Get full text