Search Results - "Kim, Najoung"

Automatic Scoring of Semantic Fluency by Kim, Najoung, Kim, Jung-Ho, Wolters, Maria K, MacPherson, Sarah E, Park, Jong C

“…In neuropsychological assessment, semantic fluency is a widely accepted measure of executive function and access to semantic memory. While fluency scores are…”
Get full text

Finding Structure in One Child's Linguistic Experience by Wang, Wentao, Vong, Wai Keen, Kim, Najoung, Lake, Brenden M.

“…Neural network models have recently made striking progress in natural language processing, but they are typically trained on orders of magnitude more language…”
Get full text

Generating novel experimental hypotheses from language models: A case study on cross-dative generalization by Misra, Kanishka, Kim, Najoung

“…Neural network language models (LMs) have been shown to successfully capture complex linguistic knowledge. However, their utility for understanding language…”
Get full text

Abstraction via exemplars? A representational case study on lexical category inference in BERT by Misra, Kanishka, Kim, Najoung

“…Exemplar based accounts are often considered to be in direct opposition to pure linguistic abstraction in explaining language learners' ability to generalize…”
Get full text

Entity Tracking in Language Models by Kim, Najoung, Schuster, Sebastian

“…Keeping track of how states of entities change as a text or dialog unfolds is a key prerequisite to discourse understanding. Yet, there have been few…”
Get full text

Is artificial intelligence still intelligence? LLMs generalize to novel adjective-noun pairs, but don't mimic the full human distribution by Ross, Hayley, Davidson, Kathryn, Kim, Najoung

“…Inferences from adjective-noun combinations like "Is artificial intelligence still intelligence?" provide a good test bed for LLMs' understanding of meaning…”
Get full text

Code Pretraining Improves Entity Tracking Abilities of Language Models by Kim, Najoung, Schuster, Sebastian, Toshniwal, Shubham

“…Recent work has provided indirect evidence that pretraining language models on code improves the ability of models to track state changes of discourse entities…”
Get full text

Syn-QA2: Evaluating False Assumptions in Long-tail Questions with Synthetic QA Datasets by Daswani, Ashwin, Sawant, Rohan, Kim, Najoung

“…Sensitivity to false assumptions (or false premises) in information-seeking questions is critical for robust question-answering (QA) systems. Recent work has…”
Get full text

COGS: A Compositional Generalization Challenge Based on Semantic Interpretation by Kim, Najoung, Linzen, Tal

“…Natural language is characterized by compositionality: the meaning of a complex expression is constructed from the meanings of its constituent parts. To…”
Get full text

Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models by Kim, Najoung, Linzen, Tal, Smolensky, Paul

“…Human linguistic capacity is often characterized by compositionality and the generalization it enables -- human learners can produce and comprehend novel…”
Get full text

Personas as a Way to Model Truthfulness in Language Models by Joshi, Nitish, Rando, Javier, Saparov, Abulhair, Kim, Najoung, He, He

“…Large language models (LLMs) are trained on vast amounts of text from the internet, which contains both factual and misleading information about the world…”
Get full text

Reconstruction Probing by Kim, Najoung, Khilnani, Jatin, Warstadt, Alex, Qaddoumi, Abed

“…We propose reconstruction probing, a new analysis method for contextualized representations based on reconstruction probabilities in masked language models…”
Get full text

LAMBADA: Backward Chaining for Automated Reasoning in Natural Language by Kazemi, Mehran, Kim, Najoung, Bhatia, Deepti, Xu, Xin, Ramachandran, Deepak

“…Remarkable progress has been made on automated reasoning with natural text, by using Language Models (LMs) and methods such as Chain-of-Thought and…”
Get full text

(QA)$^2$: Question Answering with Questionable Assumptions by Kim, Najoung, Htut, Phu Mon, Bowman, Samuel R, Petty, Jackson

“…Naturally occurring information-seeking questions often contain questionable assumptions -- assumptions that are false or unverifiable. Questions containing…”
Get full text

Inverse scaling can become U-shaped by Wei, Jason, Kim, Najoung, Tay, Yi, Le, Quoc V

“…Scaling up language models has been empirically shown to improve performance on a wide range of downstream tasks. However, if we were to observe worse…”
Get full text

SLOG: A Structural Generalization Benchmark for Semantic Parsing by Li, Bingzhi, Donatelli, Lucia, Koller, Alexander, Linzen, Tal, Yao, Yuekun, Kim, Najoung

“…The goal of compositional generalization benchmarks is to evaluate how well models generalize to new complex linguistic expressions. Existing benchmarks often…”
Get full text

BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Information by Kazemi, Mehran, Yuan, Quan, Bhatia, Deepti, Kim, Najoung, Xu, Xin, Imbrasaite, Vaiva, Ramachandran, Deepak

“…Automated reasoning with unstructured natural text is a key requirement for many potential applications of NLP and for developing robust AI systems. Recently,…”
Get full text

Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples by Saparov, Abulhair, Pang, Richard Yuanzhe, Padmakumar, Vishakh, Joshi, Nitish, Kazemi, Seyed Mehran, Kim, Najoung, He, He

“…Given the intractably large size of the space of proofs, any model that is capable of general deductive reasoning must generalize to proofs of greater…”
Get full text

Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering by Kim, Najoung, Pavlick, Ellie, Ayan, Burcu Karagol, Ramachandran, Deepak

“…Many Question-Answering (QA) datasets contain unanswerable questions, but their treatment in QA systems remains primitive. Our analysis of the Natural…”
Get full text

Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks by Wu, Zhaofeng, Qiu, Linlu, Ross, Alexis, Akyürek, Ekin, Chen, Boyuan, Wang, Bailin, Kim, Najoung, Andreas, Jacob, Kim, Yoon

“…The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these…”
Get full text