pruning – Andrew Fairless, Ph.D.

What I Read: Bonsai Networks, RNNs

By Andrew Fairless on October 3, 2023September 4, 2023

https://cprimozic.net/blog/growing-sparse-computational-graphs-with-rnns/ Growing Bonsai Networks with RNNsCasey Primozic “This writeup introduces what I’m calling Bonsai Networks – extremely sparse computational graphs produced by training and pruning RNNs. They provide an interpretableContinue readingWhat I Read: Bonsai Networks, RNNs

What I Read: Models Memorize or Generalize?

By Andrew Fairless on October 2, 2023September 4, 2023

https://pair.withgoogle.com/explorables/grokking/ Do Machine Learning Models Memorize or Generalize?By Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg and Lucas DixonAugust 2023 “Large language models can certainly seem like theyContinue readingWhat I Read: Models Memorize or Generalize?

What I Learn: neural scaling, data pruning

By Andrew Fairless on April 11, 2023February 22, 2023

https://arxiv.org/abs/2206.14486 Beyond neural scaling laws: beating power law scaling via data pruningBen Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari S. Morcos[Submitted on 29 Jun 2022 (v1), last revised 15Continue readingWhat I Learn: neural scaling, data pruning

What I Read: Do Wide and Deep Networks Learn the Same Things?

By Andrew Fairless on May 28, 2021May 14, 2021

https://ai.googleblog.com/2021/05/do-wide-and-deep-networks-learn-same.html Do Wide and Deep Networks Learn the Same Things?Tuesday, May 4, 2021Posted by Thao Nguyen, AI Resident, Google Research

What I Read: Deep learning model compression

By Andrew Fairless on April 10, 2021April 10, 2021

https://rachitsingh.com/deep-learning-model-compression/ Deep learning model compressionRachit SinghWritten March 26, 2021 “Researchers and practitioners have come up with many methods for optimizing neural networks to run faster or with less memory usage.Continue readingWhat I Read: Deep learning model compression

What I Read: Can a neural network train other networks?

By Andrew Fairless on January 31, 2021January 6, 2021

https://towardsdatascience.com/can-a-neural-network-train-other-networks-cf371be516c6?gi=4765ee11be44 Can a neural network train other networks?An introduction to knowledge distillationTivadar DankaOct 5 “Now you have a huge model, which, although performs excellently, there is no way to deployContinue readingWhat I Read: Can a neural network train other networks?

Tag: pruning