optimization – Page 5 – Andrew Fairless, Ph.D.

What I Read: Pathfinder, A parallel quasi-Newton algorithm

By Andrew Fairless on September 2, 2021August 17, 2021

https://statmodeling.stat.columbia.edu/2021/08/10/pathfinder-a-parallel-quasi-newton-algorithm-for-reaching-regions-of-high-probability-mass/ Pathfinder: A parallel quasi-Newton algorithm for reaching regions of high probability massAndrew Gelman10 August 2021, 9:07 am “In the world of Stan, we see three roles for Pathfinder: (1)Continue readingWhat I Read: Pathfinder, A parallel quasi-Newton algorithm

What I Read: Not Optimized By Jax, PyTorch, or Tensorflow

By Andrew Fairless on August 24, 2021August 17, 2021

Useful Algorithms That Are Not Optimized By Jax, PyTorch, or Tensorflow Useful Algorithms That Are Not Optimized By Jax, PyTorch, or TensorflowJuly 21 2021Christopher Rackauckas “It turns out that forContinue readingWhat I Read: Not Optimized By Jax, PyTorch, or Tensorflow

What I Read: Why Deep Learning Works

By Andrew Fairless on August 23, 2021August 17, 2021

https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ Why Deep Learning Works Even Though It Shouldn’tRyan Moulton’s ArticlesRyan Moulton “Stop talking about minima…. Nobody ever trains their model remotely close to convergence…. What really needs further researchContinue readingWhat I Read: Why Deep Learning Works

What I Read: Gradient Pseudo-Swap

By Andrew Fairless on August 3, 2021July 12, 2021

https://deepnote.com/@martin-molnar/Gradient-Pseudo-Swap-F2u_aTbqTOisNYCH_JORwg Martin MolnarGradient Pseudo-Swap “When we have layers in a neural network we want to train with gradient descent, but those layers don’t have smooth gradients that can be used,Continue readingWhat I Read: Gradient Pseudo-Swap

What I Read: Model Free

By Andrew Fairless on July 26, 2021July 12, 2021

http://www.argmin.net/2018/04/19/pid/ The Best Things in Life Are Model FreeBen RechtApr 19, 2018 “The odd thing about statistics and ML research these days is that everyone knows about gradient descent, butContinue readingWhat I Read: Model Free

What I Read: Game theory for large-scale data analysis

By Andrew Fairless on June 7, 2021May 14, 2021

https://deepmind.com/blog/article/EigenGame 06 May 2021Game theory as an engine for large-scale data analysisBrian McWilliamsIan GempClaire Vernade “…we reformulated principal component analysis (PCA)… as a competitive multi-agent game we call EigenGame. PCAContinue readingWhat I Read: Game theory for large-scale data analysis

What I Read: Deep learning model compression

By Andrew Fairless on April 10, 2021April 10, 2021

https://rachitsingh.com/deep-learning-model-compression/ Deep learning model compressionRachit SinghWritten March 26, 2021 “Researchers and practitioners have come up with many methods for optimizing neural networks to run faster or with less memory usage.Continue readingWhat I Read: Deep learning model compression

What I Read: Machine learning is going real-time

By Andrew Fairless on March 3, 2021March 3, 2021

https://huyenchip.com/2020/12/27/real-time-machine-learning.html Machine learning is going real-timeDec 27, 2020Chip Huyen “Machine learning is going real-time, whether you’re ready or not. While the majority of companies are still debating whether there’s valueContinue readingWhat I Read: Machine learning is going real-time

What I Read: Mismatches between Optimization Analyses and Deep Learning

By Andrew Fairless on February 13, 2021January 14, 2021

http://www.offconvex.org/2020/10/21/intrinsicLR/ Mismatches between Traditional Optimization Analyses and Modern Deep LearningZhiyuan Li and Sanjeev AroraOct 21, 2020 “You may remember our previous blog post showing that it is possible to doContinue readingWhat I Read: Mismatches between Optimization Analyses and Deep Learning

Tag: optimization