overparameterization – Andrew Fairless, Ph.D.

What I Read: Statistical Intuitions

By Andrew Fairless on April 22, 2025January 18, 2025

https://arxiv.org/abs/2409.18842 Classical Statistical (In-Sample) Intuitions Don’t Generalize Well: A Note on Bias-Variance Tradeoffs, Overfitting and Moving from Fixed to Random DesignsAlicia Curth27 Sep 2024 “…we show that classical intuitions relatingContinue readingWhat I Read: Statistical Intuitions

What I Read: Why Bigger Neural Networks Do Better

By Andrew Fairless on April 11, 2022March 12, 2022

https://www.quantamagazine.org/computer-scientists-prove-why-bigger-neural-networks-do-better-20220210/ Computer Scientists Prove Why Bigger Neural Networks Do BetterMordechai RorvigStaff WriterFebruary 10, 2022 “Two researchers show that for neural networks to be able to remember better, they need farContinue readingWhat I Read: Why Bigger Neural Networks Do Better

What I Read: Generalization in Deep Learning

By Andrew Fairless on March 31, 2021April 3, 2021

https://ai.googleblog.com/2021/03/a-new-lens-on-understanding.html A New Lens on Understanding Generalization in Deep LearningWednesday, March 10, 2021Hanie Sedghi, Google Research and Preetum Nakkiran, Harvard University “…we find that models that train quickly on infiniteContinue readingWhat I Read: Generalization in Deep Learning

What I Read: The way we train AI is flawed

By Andrew Fairless on February 15, 2021January 18, 2021

https://www.technologyreview.com/2020/11/18/1012234/training-machine-learning-broken-real-world-heath-nlp-computer-vision/ The way we train AI is fundamentally flawedThe process used to build most of the machine-learning models we use today can’t tell if they will work in the realContinue readingWhat I Read: The way we train AI is flawed

What I Read: Reproducing Deep Double Descent

By Andrew Fairless on February 7, 2021January 11, 2021

https://hippocampus-garden.com/double_descent/ Reproducing Deep Double DescentShion HondaJune 13, 2020 “…you’ve probably heard of bias-variance trade-off… But, do you know that there is a continuation of the U-shaped test risk curve? Surprisingly,Continue readingWhat I Read: Reproducing Deep Double Descent

What I Read: Bayesian Neural Networks Need Not Concentrate

By Andrew Fairless on December 7, 2020December 13, 2020

https://jacobbuckman.com/2020-01-22-bayesian-neural-networks-need-not-concentrate/ Bayesian Neural Networks Need Not ConcentratePosted on January 22, 2020by Carles Gelada and Jacob Buckman “…it’s important that we, as a field, stop ignoring the crucial importance that theContinue readingWhat I Read: Bayesian Neural Networks Need Not Concentrate

Tag: overparameterization