optimization – Page 4 – Andrew Fairless, Ph.D.

What I Read: Neuron Bursts Can Mimic Famous AI Learning Strategy

By Andrew Fairless on November 1, 2021October 24, 2021

https://www.quantamagazine.org/brain-bursts-can-mimic-famous-ai-learning-strategy-20211018/ Neuron Bursts Can Mimic Famous AI Learning StrategyAllison WhittenOctober 18, 2021“A new model of learning centers on bursts of neural activity that act as teaching signals — approximating backpropagation,Continue readingWhat I Read: Neuron Bursts Can Mimic Famous AI Learning Strategy

What I Read: Machine learning is not nonparametric statistics

By Andrew Fairless on October 20, 2021October 16, 2021

https://www.argmin.net/2021/10/13/prediction/ Machine learning is not nonparametric statistics.Ben RechtOct 13, 2021 “…I’ve been told by respected statisticians that machine learning is nothing more than nonparametric statistics. The longer I work inContinue readingWhat I Read: Machine learning is not nonparametric statistics

What I Learn: Scaling TensorFlow

By Andrew Fairless on October 12, 2021September 24, 2021

https://arxiv.org/abs/2109.09541 Scaling TensorFlow to 300 million predictions per secondJan Hartman, Davorin Kopič “In this work, we describe the process of scaling machine learning models implemented in the TensorFlow machine learningContinue readingWhat I Learn: Scaling TensorFlow

What I Read: Learning Neural Network Subspaces

By Andrew Fairless on September 20, 2021September 4, 2021

https://arxiv.org/abs/2102.10472 Learning Neural Network SubspacesMitchell Wortsman, Maxwell Horton, Carlos Guestrin, Ali Farhadi, Mohammad Rastegari “These neural network subspaces contain diverse solutions that can be ensembled, approaching the ensemble performance ofContinue readingWhat I Read: Learning Neural Network Subspaces

What I Read: Computer Scientists Discover Limits of Major Research Algorithm

By Andrew Fairless on September 7, 2021August 17, 2021

https://www.quantamagazine.org/computer-scientists-discover-limits-of-major-research-algorithm-20210817/ Computer Scientists Discover Limits of Major Research AlgorithmNick ThiemeAugust 17, 2021 “Many aspects of modern applied research rely on a crucial algorithm called gradient descent…. researchers have never fullyContinue readingWhat I Read: Computer Scientists Discover Limits of Major Research Algorithm

What I Read: Pathfinder, A parallel quasi-Newton algorithm

By Andrew Fairless on September 2, 2021August 17, 2021

https://statmodeling.stat.columbia.edu/2021/08/10/pathfinder-a-parallel-quasi-newton-algorithm-for-reaching-regions-of-high-probability-mass/ Pathfinder: A parallel quasi-Newton algorithm for reaching regions of high probability massAndrew Gelman10 August 2021, 9:07 am “In the world of Stan, we see three roles for Pathfinder: (1)Continue readingWhat I Read: Pathfinder, A parallel quasi-Newton algorithm

What I Read: Not Optimized By Jax, PyTorch, or Tensorflow

By Andrew Fairless on August 24, 2021August 17, 2021

Useful Algorithms That Are Not Optimized By Jax, PyTorch, or Tensorflow Useful Algorithms That Are Not Optimized By Jax, PyTorch, or TensorflowJuly 21 2021Christopher Rackauckas “It turns out that forContinue readingWhat I Read: Not Optimized By Jax, PyTorch, or Tensorflow

What I Read: Why Deep Learning Works

By Andrew Fairless on August 23, 2021August 17, 2021

https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ Why Deep Learning Works Even Though It Shouldn’tRyan Moulton’s ArticlesRyan Moulton “Stop talking about minima…. Nobody ever trains their model remotely close to convergence…. What really needs further researchContinue readingWhat I Read: Why Deep Learning Works

What I Read: Gradient Pseudo-Swap

By Andrew Fairless on August 3, 2021July 12, 2021

https://deepnote.com/@martin-molnar/Gradient-Pseudo-Swap-F2u_aTbqTOisNYCH_JORwg Martin MolnarGradient Pseudo-Swap “When we have layers in a neural network we want to train with gradient descent, but those layers don’t have smooth gradients that can be used,Continue readingWhat I Read: Gradient Pseudo-Swap

What I Read: Model Free

By Andrew Fairless on July 26, 2021July 12, 2021

http://www.argmin.net/2018/04/19/pid/ The Best Things in Life Are Model FreeBen RechtApr 19, 2018 “The odd thing about statistics and ML research these days is that everyone knows about gradient descent, butContinue readingWhat I Read: Model Free

Tag: optimization