Search Results - "Kobayashi, Seijin"

Comparison of Hepatic Resection and Radiofrequency Ablation for Small Hepatocellular Carcinoma: A Meta-Analysis of 16,103 Patients by Xu, Qinghua, Kobayashi, Seijin, Ye, Xun, Meng, Xia

“…We performed a meta-analysis to evaluate the therapeutic effects of radiofrequency ablation (RFA) and surgical hepatic resection (HR) in the treatment of small…”
Get full text

Wheel Defect Detection With Machine Learning by Krummenacher, Gabriel, Ong, Cheng Soon, Koller, Stefan, Kobayashi, Seijin, Buhmann, Joachim M.

“…Wheel defects on railway wagons have been identified as an important source of damage to the railway infrastructure and rolling stock. They also cause noise…”
Get full text

Comparison of Hepatic Resection and Radiofrequency Ablation for Small Hepatocellular Carcinoma: A Meta-Analysis of 16,103 Patients by Qinghua, Xu, Seijin, Kobayashi, Xia, Meng

Get full text

Weight decay induces low-rank attention layers by Kobayashi, Seijin, Akram, Yassir, Von Oswald, Johannes

“…The effect of regularizers such as weight decay when training deep neural networks is not well understood. We study the influence of weight decay as well as…”
Get full text

Learning Randomized Algorithms with Transformers by von Oswald, Johannes, Kobayashi, Seijin, Akram, Yassir, Steger, Angelika

“…Randomization is a powerful tool that endows algorithms with remarkable properties. For instance, randomized algorithms excel in adversarial settings, often…”
Get full text

Attention as a Hypernetwork by Schug, Simon, Kobayashi, Seijin, Akram, Yassir, Sacramento, João, Pascanu, Razvan

“…Transformers can under some circumstances generalize to novel problem instances whose constituent parts might have been encountered during training but whose…”
Get full text

Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis by Meulemans, Alexander, Schug, Simon, Kobayashi, Seijin, Daw, Nathaniel, Wayne, Gregory

“…To make reinforcement learning more sample efficient, we need better credit assignment methods that measure an action's influence on future rewards. Building…”
Get full text

Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel by Kobayashi, Seijin, Aceituno, Pau Vilimelis, von Oswald, Johannes

“…Identifying unfamiliar inputs, also known as out-of-distribution (OOD) detection, is a crucial property of any decision making process. A simple and…”
Get full text

Gated recurrent neural networks discover attention by Zucchet, Nicolas, Kobayashi, Seijin, Akram, Yassir, von Oswald, Johannes, Larcher, Maxime, Steger, Angelika, Sacramento, João

“…Recent architectural developments have enabled recurrent neural networks (RNNs) to reach and even surpass the performance of Transformers on certain sequence…”
Get full text

When can transformers compositionally generalize in-context? by Kobayashi, Seijin, Schug, Simon, Akram, Yassir, Redhardt, Florian, von Oswald, Johannes, Pascanu, Razvan, Lajoie, Guillaume, Sacramento, João

“…Many tasks can be composed from a few independent components. This gives rise to a combinatorial explosion of possible tasks, only some of which might be…”
Get full text

Discovering modular solutions that generalize compositionally by Schug, Simon, Kobayashi, Seijin, Akram, Yassir, Wołczyk, Maciej, Proca, Alexandra, von Oswald, Johannes, Pascanu, Razvan, Sacramento, João, Steger, Angelika

“…Many complex tasks can be decomposed into simpler, independent parts. Discovering such underlying compositional structure has the potential to enable…”
Get full text

Multi-agent cooperation through learning-aware policy gradients by Meulemans, Alexander, Kobayashi, Seijin, von Oswald, Johannes, Scherrer, Nino, Elmoznino, Eric, Richards, Blake, Lajoie, Guillaume, Arcas, Blaise Agüera y, Sacramento, João

“…Self-interested individuals often fail to cooperate, posing a fundamental challenge for multi-agent learning. How can we achieve cooperation among…”
Get full text

Meta-Learning via Classifier(-free) Diffusion Guidance by Nava, Elvis, Kobayashi, Seijin, Yin, Yifei, Katzschmann, Robert K, Grewe, Benjamin F

“…We introduce meta-learning algorithms that perform zero-shot weight-space adaptation of neural network models to unseen tasks. Our methods repurpose the…”
Get full text

The least-control principle for local learning at equilibrium by Meulemans, Alexander, Zucchet, Nicolas, Kobayashi, Seijin, von Oswald, Johannes, Sacramento, João

“…Equilibrium systems are a powerful way to express neural computations. As special cases, they include models of great current interest in both neuroscience and…”
Get full text

Learning where to learn: Gradient sparsity in meta and continual learning by von Oswald, Johannes, Zhao, Dominic, Kobayashi, Seijin, Schug, Simon, Caccia, Massimo, Zucchet, Nicolas, Sacramento, João

“…Finding neural network weights that generalize well from small datasets is difficult. A promising approach is to learn a weight initialization such that a…”
Get full text

Uncovering mesa-optimization algorithms in Transformers by von Oswald, Johannes, Schlegel, Maximilian, Meulemans, Alexander, Kobayashi, Seijin, Niklasson, Eyvind, Zucchet, Nicolas, Scherrer, Nino, Miller, Nolan, Sandler, Mark, Arcas, Blaise Agüera y, Vladymyrov, Max, Pascanu, Razvan, Sacramento, João

“…Some autoregressive models exhibit in-context learning capabilities: being able to learn as an input sequence is processed, without undergoing any parameter…”
Get full text

Neural networks with late-phase weights by von Oswald, Johannes, Kobayashi, Seijin, Meulemans, Alexander, Henning, Christian, Grewe, Benjamin F, Sacramento, João

“…Published as a conference paper at ICLR 2021 The largely successful method of training neural networks is to learn their weights using some variant of…”
Get full text

Posterior Meta-Replay for Continual Learning by Henning, Christian, Cervera, Maria R, D'Angelo, Francesco, von Oswald, Johannes, Traber, Regina, Ehret, Benjamin, Kobayashi, Seijin, Grewe, Benjamin F, Sacramento, João

“…Learning a sequence of tasks without access to i.i.d. observations is a widely studied form of continual learning (CL) that remains challenging. In principle,…”
Get full text