Search Results - "Balles, Lukas"

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation by Ranjan, Anurag, Jampani, Varun, Balles, Lukas, Kim, Kihwan, Sun, Deqing, Wulff, Jonas, Black, Michael J.

“…We address the unsupervised learning of several interconnected problems in low-level vision: single view depth prediction, camera motion estimation, optical…”
Get full text

Cyclic (Amino)(aryl)carbenes Enter the Field of Chromophore Ligands: Expanded π System Leads to Unusually Deep Red Emitting Cu I Compounds by Gernert, Markus, Balles-Wolf, Lukas, Kerner, Florian, Müller, Ulrich, Schmiedel, Alexander, Holzapfel, Marco, Marian, Christel M, Pflaum, Jens, Lambert, Christoph, Steffen, Andreas

“…A series of copper(I) complexes bearing a cyclic (amino)(aryl)carbene (CAArC) ligand with various complex geometries have been investigated in great detail…”
Get full text

A Negative Result on Gradient Matching for Selective Backprop by Balles, Lukas, Archambeau, Cedric, Zappella, Giovanni

“…With increasing scale in model and dataset size, the training of deep neural networks becomes a massive computational burden. One approach to speed up the…”
Get full text

Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need by Wistuba, Martin, Sivaprasad, Prabhu Teja, Balles, Lukas, Zappella, Giovanni

“…Recent Continual Learning (CL) methods have combined pretrained Transformers with prompt tuning, a parameter-efficient fine-tuning (PEFT) technique. We argue…”
Get full text

Continual Learning with Low Rank Adaptation by Wistuba, Martin, Sivaprasad, Prabhu Teja, Balles, Lukas, Zappella, Giovanni

“…Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest. However, they…”
Get full text

Gradient-Matching Coresets for Rehearsal-Based Continual Learning by Balles, Lukas, Zappella, Giovanni, Archambeau, Cédric

“…The goal of continual learning (CL) is to efficiently update a machine learning model with new data without forgetting previously-learned knowledge. Most…”
Get full text

Gradient-matching coresets for continual learning by Balles, Lukas, Zappella, Giovanni, Archambeau, Cédric

“…We devise a coreset selection method based on the idea of gradient matching: The gradients induced by the coreset should match, as closely as possible, those…”
Get full text

Renate: A Library for Real-World Continual Learning by Wistuba, Martin, Ferianc, Martin, Balles, Lukas, Archambeau, Cedric, Zappella, Giovanni

“…Continual learning enables the incremental training of machine learning models on non-stationary data streams.While academic interest in the topic is high,…”
Get full text

Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients by Balles, Lukas, Hennig, Philipp

“…The ADAM optimizer is exceedingly popular in the deep learning community. Often it works very well, sometimes it doesn't. Why? We interpret ADAM as a…”
Get full text

The Geometry of Sign Gradient Descent by Balles, Lukas, Pedregosa, Fabian, Roux, Nicolas Le

“…Sign-based optimization methods have become popular in machine learning due to their favorable communication cost in distributed optimization and their…”
Get full text

Cyclic (Amino)(aryl)carbenes Enter the Field of Chromophore Ligands: Expanded π System Leads to Unusually Deep Red Emitting CuI Compounds by Gernert, Markus, Balles-Wolf, Lukas, Kerner, Florian, Müller, Ulrich, Schmiedel, Alexander, Holzapfel, Marco, Marian, Christel M, Pflaum, Jens, Lambert, Christoph, Steffen, Andreas

“…A series of copper­(I) complexes bearing a cyclic (amino)­(aryl)­carbene (CAArC) ligand with various complex geometries have been investigated in great detail…”
Get full text

u-$\mu$P: The Unit-Scaled Maximal Update Parametrization by Blake, Charlie, Eichenberg, Constantin, Dean, Josef, Balles, Lukas, Prince, Luke Y, Deiseroth, Björn, Cruz-Salinas, Andres Felipe, Luschi, Carlo, Weinbach, Samuel, Orr, Douglas

“…The Maximal Update Parametrization ($\mu$P) aims to make the optimal hyperparameters (HPs) of a model independent of its size, allowing them to be swept using…”
Get full text

Limitations of the Empirical Fisher Approximation for Natural Gradient Descent by Kunstner, Frederik, Balles, Lukas, Hennig, Philipp

“…Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to…”
Get full text

DeepOBS: A Deep Learning Optimizer Benchmark Suite by Schneider, Frank, Balles, Lukas, Hennig, Philipp

“…Because the choice and tuning of the optimizer affects the speed, and ultimately the performance of deep learning, there is significant past and recent…”
Get full text

PASHA: Efficient HPO and NAS with Progressive Resource Allocation by Bohdal, Ondrej, Balles, Lukas, Wistuba, Martin, Ermis, Beyza, Archambeau, Cédric, Zappella, Giovanni

“…Hyperparameter optimization (HPO) and neural architecture search (NAS) are methods of choice to obtain the best-in-class machine learning models, but in…”
Get full text

Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering by Chen, Ricky T. Q, Choi, Dami, Balles, Lukas, Duvenaud, David, Hennig, Philipp

“…Standard first-order stochastic optimization algorithms base their updates solely on the average mini-batch gradient, and it has been shown that tracking…”
Get full text

Coupling Adaptive Batch Sizes with Learning Rates by Balles, Lukas, Romero, Javier, Hennig, Philipp

“…Mini-batch stochastic gradient descent and variants thereof have become standard for large-scale empirical risk minimization like the training of neural…”
Get full text

Early Stopping without a Validation Set by Mahsereci, Maren, Balles, Lukas, Lassner, Christoph, Hennig, Philipp

“…Early stopping is a widely used technique to prevent poor generalization performance when training an over-expressive model by means of gradient-based…”
Get full text

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation by Ranjan, Anurag, Jampani, Varun, Balles, Lukas, Kim, Kihwan, Sun, Deqing, Wulff, Jonas, Black, Michael J

“…We address the unsupervised learning of several interconnected problems in low-level vision: single view depth prediction, camera motion estimation, optical…”
Get full text