optimization – Page 5 – Andrew Fairless, Ph.D.

What I Read: Why Deep Learning Works

By Andrew Fairless on August 23, 2021August 17, 2021

https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ Why Deep Learning Works Even Though It Shouldn’tRyan Moulton’s ArticlesRyan Moulton “Stop talking about minima…. Nobody ever trains their model remotely close to convergence…. What really needs further researchContinue readingWhat I Read: Why Deep Learning Works

What I Read: Gradient Pseudo-Swap

By Andrew Fairless on August 3, 2021July 12, 2021

https://deepnote.com/@martin-molnar/Gradient-Pseudo-Swap-F2u_aTbqTOisNYCH_JORwg Martin MolnarGradient Pseudo-Swap “When we have layers in a neural network we want to train with gradient descent, but those layers don’t have smooth gradients that can be used,Continue readingWhat I Read: Gradient Pseudo-Swap

What I Read: Model Free

By Andrew Fairless on July 26, 2021July 12, 2021

http://www.argmin.net/2018/04/19/pid/ The Best Things in Life Are Model FreeBen RechtApr 19, 2018 “The odd thing about statistics and ML research these days is that everyone knows about gradient descent, butContinue readingWhat I Read: Model Free

What I Read: Game theory for large-scale data analysis

By Andrew Fairless on June 7, 2021May 14, 2021

https://deepmind.com/blog/article/EigenGame 06 May 2021Game theory as an engine for large-scale data analysisBrian McWilliamsIan GempClaire Vernade “…we reformulated principal component analysis (PCA)… as a competitive multi-agent game we call EigenGame. PCAContinue readingWhat I Read: Game theory for large-scale data analysis

What I Read: Deep learning model compression

By Andrew Fairless on April 10, 2021April 10, 2021

https://rachitsingh.com/deep-learning-model-compression/ Deep learning model compressionRachit SinghWritten March 26, 2021 “Researchers and practitioners have come up with many methods for optimizing neural networks to run faster or with less memory usage.Continue readingWhat I Read: Deep learning model compression

What I Read: Machine learning is going real-time

By Andrew Fairless on March 3, 2021March 3, 2021

https://huyenchip.com/2020/12/27/real-time-machine-learning.html Machine learning is going real-timeDec 27, 2020Chip Huyen “Machine learning is going real-time, whether you’re ready or not. While the majority of companies are still debating whether there’s valueContinue readingWhat I Read: Machine learning is going real-time

What I Read: Mismatches between Optimization Analyses and Deep Learning

By Andrew Fairless on February 13, 2021January 14, 2021

http://www.offconvex.org/2020/10/21/intrinsicLR/ Mismatches between Traditional Optimization Analyses and Modern Deep LearningZhiyuan Li and Sanjeev AroraOct 21, 2020 “You may remember our previous blog post showing that it is possible to doContinue readingWhat I Read: Mismatches between Optimization Analyses and Deep Learning

What I Read: Reproducing Deep Double Descent

By Andrew Fairless on February 7, 2021January 11, 2021

https://hippocampus-garden.com/double_descent/ Reproducing Deep Double DescentShion HondaJune 13, 2020 “…you’ve probably heard of bias-variance trade-off… But, do you know that there is a continuation of the U-shaped test risk curve? Surprisingly,Continue readingWhat I Read: Reproducing Deep Double Descent

What I Read: Deep Double Descent: Where Bigger Models and More Data Hurt

By Andrew Fairless on February 6, 2021January 11, 2021

https://arxiv.org/abs/1912.02292 Deep Double Descent: Where Bigger Models and More Data HurtPreetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever “We show that a variety of modern deepContinue readingWhat I Read: Deep Double Descent: Where Bigger Models and More Data Hurt

Tag: optimization