regularization – Andrew Fairless, Ph.D.

What I Read: KL All You Need

By Andrew Fairless on August 21, 2024June 4, 2024

https://blog.alexalemi.com/kl-is-all-you-need.html KL is All You NeedAlexander A. Alemi2024-01-08 “…the core of essentially all modern machine learning methods is a single universal objective: Kullback-Leibler (KL) divergence minimization…. Understand KL, understand theContinue readingWhat I Read: KL All You Need

What I Read: polynomial monster

By Andrew Fairless on March 26, 2024February 5, 2024

https://alexshtf.github.io/2024/01/25/Bernstein-Basis.html Keeping the polynomial monster under controlAlex ShtoffJan 25, 2024 “…we saw that the Bernstein polynomials can be used to fit a high-degree polynomial curve with ease, without its shapeContinue readingWhat I Read: polynomial monster

What I Read: polynomial features

By Andrew Fairless on March 25, 2024February 5, 2024

https://alexshtf.github.io/2024/01/21/Bernstein.html Are polynomial features the root of all evil?Alex ShtoffJan 21, 2024 “There’s nothing inherently wrong with high degree polynomials, and in contrast to what is typically taught, high degreeContinue readingWhat I Read: polynomial features

What I Read: Self-Attention in GPT

By Andrew Fairless on March 4, 2024January 25, 2024

https://twiecki.io/blog/2024/01/04/ An Intuitive Guide to Self-Attention in GPT: The Venetian MasqueradeThomas WieckiJanuary 4, 2024 “In AI, especially with something as intricate as self-attention, it’s easy to get lost in theContinue readingWhat I Read: Self-Attention in GPT

What I Read: Bonsai Networks, RNNs

By Andrew Fairless on October 3, 2023September 4, 2023

https://cprimozic.net/blog/growing-sparse-computational-graphs-with-rnns/ Growing Bonsai Networks with RNNsCasey Primozic “This writeup introduces what I’m calling Bonsai Networks – extremely sparse computational graphs produced by training and pruning RNNs. They provide an interpretableContinue readingWhat I Read: Bonsai Networks, RNNs

What I Read: Ways Digital Minds Know

By Andrew Fairless on August 8, 2023July 9, 2023

https://moultano.wordpress.com/2023/06/28/the-many-ways-that-digital-minds-can-know/ The Many Ways that Digital Minds Can KnowRyan Moulton “…aspects of search engine quality are better analogies to describe the properties of LLMs than “generalization” and “memorization” are, andContinue readingWhat I Read: Ways Digital Minds Know

What I Read: What, Why ChatGPT

By Andrew Fairless on July 12, 2023June 15, 2023

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ What Is ChatGPT Doing … and Why Does It Work?February 14, 2023Stephen Wolfram “That ChatGPT can automatically generate something that reads even superficially like human-written text is remarkable, andContinue readingWhat I Read: What, Why ChatGPT

What I Read: Learning, not Enough Data Part 3

By Andrew Fairless on June 1, 2022May 23, 2022

https://lilianweng.github.io/posts/2022-04-15-data-gen/ Learning with not Enough Data Part 3: Data GenerationApril 15, 2022Lilian Weng “Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2).Continue readingWhat I Read: Learning, not Enough Data Part 3

What I Read: Policy Regulariser, Adversary

By Andrew Fairless on May 11, 2022April 25, 2022

https://deepmindsafetyresearch.medium.com/your-policy-regulariser-is-secretly-an-adversary-14684c743d45 Your Policy Regulariser is Secretly an AdversaryDeepMind Safety ResearchMar 24 By Rob Brekelmans, Tim Genewein, Jordi Grau-Moya, Grégoire Delétang, Markus Kunesch, Shane Legg, Pedro A. Ortega“Policy regularisation can beContinue readingWhat I Read: Policy Regulariser, Adversary

What I Read: Why Deep Learning Works

By Andrew Fairless on August 23, 2021August 17, 2021

https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ Why Deep Learning Works Even Though It Shouldn’tRyan Moulton’s ArticlesRyan Moulton “Stop talking about minima…. Nobody ever trains their model remotely close to convergence…. What really needs further researchContinue readingWhat I Read: Why Deep Learning Works

Tag: regularization