gradient descent – Andrew Fairless, Ph.D.

What I Read: Giant Steps Can Solve Optimization Faster

By Andrew Fairless on September 27, 2023September 4, 2023

https://www.quantamagazine.org/risky-giant-steps-can-solve-optimization-problems-faster-20230811/ Risky Giant Steps Can Solve Optimization Problems FasterAllison ParshallAugust 11, 2023 “New results break with decades of conventional wisdom for the gradient descent algorithm.”

What I Read: Neural Tangent Kernel

By Andrew Fairless on November 2, 2022September 27, 2022

https://lilianweng.github.io/posts/2022-09-08-ntk/ Some Math behind Neural Tangent KernelLilian WengSeptember 8, 2022 “Neural tangent kernel… leads to great insights into why neural networks with enough width can consistently converge to a globalContinue readingWhat I Read: Neural Tangent Kernel

What I Read: Deep Learning Optimization Theory

By Andrew Fairless on December 15, 2021November 16, 2021

https://towardsdatascience.com/deep-learning-optimization-theory-introduction-148b3504b20f?gi=ff0bd10cc9fe Deep Learning Optimization Theory — IntroductionUnderstanding the theory of optimization in deep learning is crucial to enable progress. This post introduces the experimental and theoretical approaches to studying it.OmriContinue readingWhat I Read: Deep Learning Optimization Theory

What I Read: machine learning with differential privacy

By Andrew Fairless on November 29, 2021November 16, 2021

https://differentialprivacy.org/how-to-deploy-ml-with-dp/ How to deploy machine learning with differential privacy?Posted by Nicolas Papernot and Abhradeep Thakurta on October 25, 2021. “In many applications of machine learning, such as machine learning forContinue readingWhat I Read: machine learning with differential privacy

What I Read: Machine learning is just statistics + quantifier reversal

By Andrew Fairless on November 18, 2021November 16, 2021

https://jeremybernste.in/writing/ml-is-just-statistics Machine learning is just statistics + quantifier reversaljeremybernsteLos Angeles, 21 Oct 2021 “But in machine learning there’s a switcheroo—we select the sample of data first, and then we useContinue readingWhat I Read: Machine learning is just statistics + quantifier reversal

What I Read: First-Principles Theory of Neural Network Generalization

By Andrew Fairless on November 17, 2021November 16, 2021

https://natluk.net/a-first-principles-theory-of-neuralnetwork-generalization-the-berkeley-artificial-intelligence-research-blog/ A First-Principles Theory of Neural Network Generalization – The Berkeley Artificial Intelligence Research BlogNatLuk Community25 October 2021 “Perhaps the greatest of these mysteries has been the question of generalization:Continue readingWhat I Read: First-Principles Theory of Neural Network Generalization

What I Read: Mystery of Deep Learning

By Andrew Fairless on October 13, 2021October 12, 2021

https://www.quantamagazine.org/a-new-link-to-an-old-model-could-crack-the-mystery-of-deep-learning-20211011/ A New Link to an Old Model Could Crack the Mystery of Deep LearningTo help them explain the shocking success of deep neural networks, researchers are turning to olderContinue readingWhat I Read: Mystery of Deep Learning

What I Read: Learning Neural Network Subspaces

By Andrew Fairless on September 20, 2021September 4, 2021

https://arxiv.org/abs/2102.10472 Learning Neural Network SubspacesMitchell Wortsman, Maxwell Horton, Carlos Guestrin, Ali Farhadi, Mohammad Rastegari “These neural network subspaces contain diverse solutions that can be ensembled, approaching the ensemble performance ofContinue readingWhat I Read: Learning Neural Network Subspaces

What I Read: Computer Scientists Discover Limits of Major Research Algorithm

By Andrew Fairless on September 7, 2021August 17, 2021

https://www.quantamagazine.org/computer-scientists-discover-limits-of-major-research-algorithm-20210817/ Computer Scientists Discover Limits of Major Research AlgorithmNick ThiemeAugust 17, 2021 “Many aspects of modern applied research rely on a crucial algorithm called gradient descent…. researchers have never fullyContinue readingWhat I Read: Computer Scientists Discover Limits of Major Research Algorithm

What I Read: Why Deep Learning Works

By Andrew Fairless on August 23, 2021August 17, 2021

https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ Why Deep Learning Works Even Though It Shouldn’tRyan Moulton’s ArticlesRyan Moulton “Stop talking about minima…. Nobody ever trains their model remotely close to convergence…. What really needs further researchContinue readingWhat I Read: Why Deep Learning Works

Tag: gradient descent