Search Results - "Advani, Madhu S."

High-dimensional dynamics of generalization error in neural networks by Advani, Madhu S., Saxe, Andrew M., Sompolinsky, Haim

“…We perform an analysis of the average generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant…”
Get full text

Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning by Zhang, Yao, Saxe, Andrew M., Advani, Madhu S., Lee, Alpha A.

“…Finding parameters that minimise a loss function is at the core of many machine learning methods. The Stochastic Gradient Descent (SGD) algorithm is widely…”
Get full text

High-dimensional dynamics of generalization error in neural networks by Advani, Madhu S, Saxe, Andrew M

“…We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant…”
Get full text

Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup by Goldt, Sebastian, Advani, Madhu S, Saxe, Andrew M, Krzakala, Florent, Zdeborová, Lenka

“…J. Stat. Mech. 2020 124010 & NeurIPS 2019 Deep neural networks achieve stellar generalisation even when they have enough parameters to easily fit all their…”
Get full text

Generalisation dynamics of online learning in over-parameterised neural networks by Goldt, Sebastian, Advani, Madhu S, Saxe, Andrew M, Krzakala, Florent, Zdeborová, Lenka

“…Presented at the ICML 2019 Workshop on Theoretical Physics for Deep Learning Deep neural networks achieve stellar generalisation on a variety of problems,…”
Get full text

Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning by Zhang, Yao, Saxe, Andrew M, Advani, Madhu S, Lee, Alpha A

“…Finding parameters that minimise a loss function is at the core of many machine learning methods. The Stochastic Gradient Descent algorithm is widely used and…”
Get full text