Search Results - "Yogatama, Dani"

Adaptive Semiparametric Language Models by Yogatama, Dani, de Masson d’Autume, Cyprien, Kong, Lingpeng

“…We present a language model that combines a large parametric neural network (i.e., a transformer) with a non-parametric episodic memory component in an…”
Get full text

Questions Are All You Need to Train a Dense Passage Retriever by Sachan, Devendra Singh, Lewis, Mike, Yogatama, Dani, Zettlemoyer, Luke, Pineau, Joelle, Zaheer, Manzil

“…We introduce , a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled training data. Dense retrieval is…”
Get full text

Relational Memory-Augmented Language Models by Liu, Qi, Yogatama, Dani, Blunsom, Phil

“…We present a memory-augmented approach to condition an autoregressive language model on a knowledge graph. We represent the graph as a collection of relation…”
Get full text

Grandmaster level in StarCraft II using multi-agent reinforcement learning by Vinyals, Oriol, Babuschkin, Igor, Czarnecki, Wojciech M., Mathieu, Michaël, Dudzik, Andrew, Chung, Junyoung, Choi, David H., Powell, Richard, Ewalds, Timo, Georgiev, Petko, Oh, Junhyuk, Horgan, Dan, Kroiss, Manuel, Danihelka, Ivo, Huang, Aja, Sifre, Laurent, Cai, Trevor, Agapiou, John P., Jaderberg, Max, Vezhnevets, Alexander S., Leblond, Rémi, Pohlen, Tobias, Dalibard, Valentin, Budden, David, Sulsky, Yury, Molloy, James, Paine, Tom L., Gulcehre, Caglar, Wang, Ziyu, Pfaff, Tobias, Wu, Yuhuai, Ring, Roman, Yogatama, Dani, Wünsch, Dario, McKinney, Katrina, Smith, Oliver, Schaul, Tom, Lillicrap, Timothy, Kavukcuoglu, Koray, Hassabis, Demis, Apps, Chris, Silver, David

“…Many real-world applications require artificial agents to compete and coordinate with other agents in complex environments. As a stepping stone to this goal,…”
Get full text

Syntactic Structure Distillation Pretraining for Bidirectional Encoders by Kuncoro, Adhiguna, Kong, Lingpeng, Fried, Daniel, Yogatama, Dani, Rimell, Laura, Dyer, Chris, Blunsom, Phil

“…Textual representation learners trained on large amounts of data have achieved notable success on downstream tasks; intriguingly, they have also performed well…”
Get full text

Understanding In-Context Learning with a Pelican Soup Framework by Chiang, Ting-Rui, Yogatama, Dani

“…Many existing theoretical analyses of in-context learning for natural language processing are based on latent variable models that leaves gaps between theory…”
Get full text

The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining by Chiang, Ting-Rui, Yogatama, Dani

“…We analyze the masked language modeling pretraining objective function from the perspective of the distributional hypothesis. We investigate whether better…”
Get full text

Modelling Latent Skills for Multitask Language Generation by Cao, Kris, Yogatama, Dani

“…We present a generative model for multitask conditional language generation. Our guiding hypothesis is that a shared set of latent skills underlies many…”
Get full text

Causal Interventions on Causal Paths: Mapping GPT-2's Reasoning From Syntax to Semantics by Lee, Isabelle, Lum, Joshua, Liu, Ziyi, Yogatama, Dani

“…While interpretability research has shed light on some internal algorithms utilized by transformer-based LLMs, reasoning in natural language, with its deep…”
Get full text

LocateBench: Evaluating the Locating Ability of Vision Language Models by Chiang, Ting-Rui, Robinson, Joshua, Yu, Xinyan Velocity, Yogatama, Dani

“…The ability to locate an object in an image according to natural language instructions is crucial for many real-world applications. In this work we propose…”
Get full text

DeLLMa: Decision Making Under Uncertainty with Large Language Models by Liu, Ollie, Fu, Deqing, Yogatama, Dani, Neiswanger, Willie

“…The potential of large language models (LLMs) as decision support tools is increasingly being explored in fields such as business, engineering, and medicine,…”
Get full text

Relational Memory Augmented Language Models by Liu, Qi, Yogatama, Dani, Blunsom, Phil

“…We present a memory-augmented approach to condition an autoregressive language model on a knowledge graph. We represent the graph as a collection of relation…”
Get full text

Balancing Average and Worst-case Accuracy in Multitask Learning by Michel, Paul, Ruder, Sebastian, Yogatama, Dani

“…When training and evaluating machine learning models on a large number of tasks, it is important to not only look at average task accuracy -- which may be…”
Get full text

The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities by Wu, Zhaofeng, Yu, Xinyan Velocity, Yogatama, Dani, Lu, Jiasen, Kim, Yoon

“…Modern language models can process inputs across diverse languages and modalities. We hypothesize that models acquire this capability through learning a shared…”
Get full text

IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations by Fu, Deqing, Guo, Ruohao, Khalighinejad, Ghazal, Liu, Ollie, Dhingra, Bhuwan, Yogatama, Dani, Jia, Robin, Neiswanger, Willie

“…Current foundation models exhibit impressive capabilities when prompted either with text only or with both image and text inputs. But do their capabilities…”
Get full text

On Retrieval Augmentation and the Limitations of Language Model Training by Chiang, Ting-Rui, Yu, Xinyan Velocity, Robinson, Joshua, Liu, Ollie, Lee, Isabelle, Yogatama, Dani

“…Augmenting a language model (LM) with $k$-nearest neighbors ($k$NN) retrieval on its training data alone can decrease its perplexity, though the underlying…”
Get full text

Interpretable Diffusion via Information Decomposition by Kong, Xianghao, Liu, Ollie, Li, Han, Yogatama, Dani, Steeg, Greg Ver

“…Denoising diffusion models enable conditional generation and density modeling of complex relationships like images and text. However, the nature of the learned…”
Get full text

On the Cross-lingual Transferability of Monolingual Representations by Artetxe, Mikel, Ruder, Sebastian, Yogatama, Dani

“…State-of-the-art unsupervised multilingual models (e.g., multilingual BERT) have been shown to generalize in a zero-shot cross-lingual setting. This…”
Get full text

Dynamic Language Models for Streaming Text by Yogatama, Dani, Wang, Chong, Routledge, Bryan R., Smith, Noah A., Xing, Eric P.

“…We present a probabilistic language model that captures temporal dynamics and conditions on arbitrary context features. These context features serve as…”
Get full text

Adaptive Semiparametric Language Models by Yogatama, Dani, d'Autume, Cyprien de Masson, Kong, Lingpeng

“…We present a language model that combines a large parametric neural network (i.e., a transformer) with a non-parametric episodic memory component in an…”
Get full text