Search Results - "Suhr, Alane"

Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior by Kojima, Noriyuki, Suhr, Alane, Artzi, Yoav

“…We study continual learning for natural language instruction generation, by observing human users’ instruction execution. We focus on a collaborative scenario,…”
Get full text

TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments by Chen, Howard, Suhr, Alane, Misra, Dipendra, Snavely, Noah, Artzi, Yoav

“…We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task. We introduce the Touchdown task and…”
Get full text

Reasoning and Learning in Interactive Natural Language Systems by Suhr, Alane Laughlin

“…Systems that support expressive, situated natural language interactions are essential for expanding access to complex computing systems, such as robots and…”
Get full text

Continual Learning for Instruction Following from Realtime Feedback by Suhr, Alane, Artzi, Yoav

“…We propose and deploy an approach to continually train an instruction-following agent from feedback provided by users during collaborative interactions. During…”
Get full text

Grounding Language in Multi-Perspective Referential Communication by Tang, Zineng, Mao, Lingjun, Suhr, Alane

“…We introduce a task and dataset for referring expression generation and comprehension in multi-agent embodied environments. In this task, two agents in a…”
Get full text

Using Language Models to Disambiguate Lexical Choices in Translation by Barua, Josh, Subramanian, Sanjay, Yin, Kayo, Suhr, Alane

“…In translation, a concept represented by a single word in a source language can have multiple variations in a target language. The task of lexical selection…”
Get full text

NLVR2 Visual Bias Analysis by Suhr, Alane, Artzi, Yoav

“…NLVR2 (Suhr et al., 2019) was designed to be robust for language bias through a data collection process that resulted in each natural language sentence…”
Get full text

Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting by Sclar, Melanie, Choi, Yejin, Tsvetkov, Yulia, Suhr, Alane

“…As large language models (LLMs) are adopted as a fundamental component of language technologies, it is crucial to accurately characterize their performance…”
Get full text

Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior by Kojima, Noriyuki, Suhr, Alane, Artzi, Yoav

“…We study continual learning for natural language instruction generation, by observing human users' instruction execution. We focus on a collaborative scenario,…”
Get full text

Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation by Suhr, Alane, Artzi, Yoav

“…We propose a learning approach for mapping context-dependent sequential instructions to actions. We address the problem of discourse and state dependencies…”
Get full text

DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning by Bai, Hao, Zhou, Yifei, Cemri, Mert, Pan, Jiayi, Suhr, Alane, Levine, Sergey, Kumar, Aviral

“…Training corpuses for vision language models (VLMs) typically lack sufficient amounts of decision-centric data. This renders off-the-shelf VLMs sub-optimal for…”
Get full text

Autonomous Evaluation and Refinement of Digital Agents by Pan, Jiayi, Zhang, Yichi, Tomlin, Nicholas, Zhou, Yifei, Levine, Sergey, Suhr, Alane

“…We show that domain-general automatic evaluators can significantly improve the performance of agents for web navigation and device control. We experiment with…”
Get full text

Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker by Sclar, Melanie, Kumar, Sachin, West, Peter, Suhr, Alane, Choi, Yejin, Tsvetkov, Yulia

“…ACL 2023 Theory of Mind (ToM)$\unicode{x2014}$the ability to reason about the mental states of other people$\unicode{x2014}$is a key element of our social…”
Get full text

Analysis of Language Change in Collaborative Instruction Following by Effenberger, Anna, Yan, Eva, Singh, Rhia, Suhr, Alane, Artzi, Yoav

“…We analyze language change over time in a collaborative, goal-oriented instructional task, where utility-maximizing participants form conventions and increase…”
Get full text

Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling by Nottingham, Kolby, Ammanabrolu, Prithviraj, Suhr, Alane, Choi, Yejin, Hajishirzi, Hannaneh, Singh, Sameer, Fox, Roy

“…Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of the world. However, if initialized with knowledge of high-level…”
Get full text

Abstract Visual Reasoning with Tangram Shapes by Ji, Anya, Kojima, Noriyuki, Rush, Noah, Suhr, Alane, Vong, Wai Keen, Hawkins, Robert D, Artzi, Yoav

“…We introduce KiloGram, a resource for studying abstract visual reasoning in humans and machines. Drawing on the history of tangram puzzles as stimuli in…”
Get full text

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning by Zhai, Yuexiang, Bai, Hao, Lin, Zipeng, Pan, Jiayi, Tong, Shengbang, Zhou, Yifei, Suhr, Alane, Xie, Saining, LeCun, Yann, Ma, Yi, Levine, Sergey

“…Large vision-language models (VLMs) fine-tuned on specialized visual instruction-following data have exhibited impressive language reasoning capabilities…”
Get full text

Fine-Grained Human Feedback Gives Better Rewards for Language Model Training by Wu, Zeqiu, Hu, Yushi, Shi, Weijia, Dziri, Nouha, Suhr, Alane, Ammanabrolu, Prithviraj, Smith, Noah A, Ostendorf, Mari, Hajishirzi, Hannaneh

“…Language models (LMs) often exhibit undesirable text generation behaviors, including generating false, toxic, or irrelevant outputs. Reinforcement learning…”
Get full text

We're Afraid Language Models Aren't Modeling Ambiguity by Liu, Alisa, Wu, Zhaofeng, Michael, Julian, Suhr, Alane, West, Peter, Koller, Alexander, Swayamdipta, Swabha, Smith, Noah A, Choi, Yejin

“…Ambiguity is an intrinsic feature of natural language. Managing ambiguity is a key part of human language understanding, allowing us to anticipate…”
Get full text

UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations by Zhao, Wenting, Chiu, Justin T, Hwang, Jena D, Brahman, Faeze, Hessel, Jack, Choudhury, Sanjiban, Choi, Yejin, Li, Xiang Lorraine, Suhr, Alane

“…Language technologies that accurately model the dynamics of events must perform commonsense reasoning. Existing work evaluating commonsense reasoning focuses…”
Get full text