Search Results - "Ryabinin, Max"

Hypernymy Understanding Evaluation of Text-to-Image Models via WordNet Hierarchy by Baryshnikov, Anton, Ryabinin, Max

“…Text-to-image synthesis has recently attracted widespread attention due to rapidly improving quality and numerous practical applications. However, the language…”
Get full text

It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning by Tikhonov, Alexey, Ryabinin, Max

“…Commonsense reasoning is one of the key problems in natural language processing, but the relative scarcity of labeled data holds back the progress for…”
Get full text

Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements by Voronov, Anton, Wolf, Lena, Ryabinin, Max

“…Large language models demonstrate a remarkable capability for learning to solve new tasks from a few examples. The prompt template, or the way the input…”
Get full text

Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts by Ryabinin, Max, Gusev, Anton

“…Advances in Neural Information Processing Systems 33 (2020) 3659-3672 Many recent breakthroughs in deep learning were achieved by training increasingly larger…”
Get full text

Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language by Wang, Jiayi, Lu, Yao, Weber, Maurice, Ryabinin, Max, Chen, Yihong, Tang, Raphael, Stenetorp, Pontus

“…English, as a very high-resource language, enables the pretraining of high-quality large language models (LLMs). The same cannot be said for most other…”
Get full text

Is This Loss Informative? Faster Text-to-Image Customization by Tracking Objective Dynamics by Voronov, Anton, Khoroshikh, Mikhail, Babenko, Artem, Ryabinin, Max

“…Text-to-image generation models represent the next step of evolution in image synthesis, offering a natural way to achieve flexible yet fine-grained control…”
Get full text

SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient by Ryabinin, Max, Dettmers, Tim, Diskin, Michael, Borzunov, Alexander

“…Many deep learning applications benefit from using large models with billions of parameters. Training these models is notoriously expensive due to the need for…”
Get full text

Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets by Ryabinin, Max, Malinin, Andrey, Gales, Mark

“…Ensembles of machine learning models yield improved system performance as well as robust and interpretable uncertainty estimates; however, their inference…”
Get full text

SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices by Svirschevski, Ruslan, May, Avner, Chen, Zhuoming, Chen, Beidi, Jia, Zhihao, Ryabinin, Max

“…As large language models gain widespread adoption, running them efficiently becomes crucial. Recent works on LLM inference use speculative decoding to achieve…”
Get full text

Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding by Chen, Zhuoming, May, Avner, Svirschevski, Ruslan, Huang, Yuhsun, Ryabinin, Max, Jia, Zhihao, Chen, Beidi

“…As the usage of large language models (LLMs) grows, performing efficient inference with these models becomes increasingly important. While speculative decoding…”
Get full text

Distributed Inference and Fine-tuning of Large Language Models Over The Internet by Borzunov, Alexander, Ryabinin, Max, Chumachenko, Artem, Baranchuk, Dmitry, Dettmers, Tim, Belkada, Younes, Samygin, Pavel, Raffel, Colin

“…Large language models (LLMs) are useful in many NLP tasks and become more capable with size, with the best open-source models having over 50 billion…”
Get full text

Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees by Beznosikov, Aleksandr, Richtárik, Peter, Diskin, Michael, Ryabinin, Max, Gasnikov, Alexander

“…https://proceedings.neurips.cc/paper_files/paper/2022/hash/5ac1428c23b5da5e66d029646ea3206d-Abstract-Conference.html Variational inequalities in general and…”
Get full text

Secure Distributed Training at Scale by Gorbunov, Eduard, Borzunov, Alexander, Diskin, Michael, Ryabinin, Max

“…Many areas of deep learning benefit from using increasingly larger neural networks trained on public data, as is the case for pre-trained models for NLP and…”
Get full text

Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices by Ryabinin, Max, Gorbunov, Eduard, Plokhotnyuk, Vsevolod, Pekhimenko, Gennady

“…Training deep neural networks on large datasets can often be accelerated by using multiple compute nodes. This approach, known as distributed training, can…”
Get full text

RuCoLA: Russian Corpus of Linguistic Acceptability by Mikhailov, Vladislav, Shamardina, Tatiana, Ryabinin, Max, Pestova, Alena, Smurov, Ivan, Artemova, Ekaterina

“…Linguistic acceptability (LA) attracts the attention of the research community due to its many uses, such as testing the grammatical knowledge of language…”
Get full text

Petals: Collaborative Inference and Fine-tuning of Large Models by Borzunov, Alexander, Baranchuk, Dmitry, Dettmers, Tim, Ryabinin, Max, Belkada, Younes, Chumachenko, Artem, Samygin, Pavel, Raffel, Colin

“…Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B,…”
Get full text

Training Transformers Together by Borzunov, Alexander, Ryabinin, Max, Dettmers, Tim, Lhoest, Quentin, Saulnier, Lucile, Diskin, Michael, Jernite, Yacine, Wolf, Thomas

“…The infrastructure necessary for training state-of-the-art models is becoming overly expensive, which makes training such models affordable only to large…”
Get full text

The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models by Hong, Giwon, Gema, Aryo Pradipta, Saxena, Rohit, Du, Xiaotang, Nie, Ping, Zhao, Yu, Perez-Beltrachini, Laura, Ryabinin, Max, He, Xuanli, Fourrier, Clémentine, Minervini, Pasquale

“…Large Language Models (LLMs) have transformed the Natural Language Processing (NLP) landscape with their remarkable ability to understand and generate…”
Get full text

Embedding Words in Non-Vector Space with Unsupervised Graph Learning by Ryabinin, Max, Popov, Sergei, Prokhorenkova, Liudmila, Voita, Elena

“…It has become a de-facto standard to represent words as elements of a vector space (word2vec, GloVe). While this approach is convenient, it is unnatural for…”
Get full text

FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU by Sheng, Ying, Zheng, Lianmin, Yuan, Binhang, Li, Zhuohan, Ryabinin, Max, Fu, Daniel Y, Xie, Zhiqiang, Chen, Beidi, Barrett, Clark, Gonzalez, Joseph E, Liang, Percy, Ré, Christopher, Stoica, Ion, Zhang, Ce

“…The high computational and memory requirements of large language model (LLM) inference make it feasible only with multiple high-end accelerators. Motivated by…”
Get full text