grokking – Andrew Fairless, Ph.D.

What I Read: How Machines ‘Grok’ Data

By Andrew Fairless on June 25, 2024April 23, 2024

https://www.quantamagazine.org/how-do-machines-grok-data-20240412 How Do Machines ‘Grok’ Data?Anil Ananthaswamy4/12/24 “By apparently overtraining them, researchers have seen neural networks discover novel solutions to problems.”

What I Read: Bonsai Networks, RNNs

By Andrew Fairless on October 3, 2023September 4, 2023

https://cprimozic.net/blog/growing-sparse-computational-graphs-with-rnns/ Growing Bonsai Networks with RNNsCasey Primozic “This writeup introduces what I’m calling Bonsai Networks – extremely sparse computational graphs produced by training and pruning RNNs. They provide an interpretableContinue readingWhat I Read: Bonsai Networks, RNNs

What I Read: Models Memorize or Generalize?

By Andrew Fairless on October 2, 2023September 4, 2023

https://pair.withgoogle.com/explorables/grokking/ Do Machine Learning Models Memorize or Generalize?By Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg and Lucas DixonAugust 2023 “Large language models can certainly seem like theyContinue readingWhat I Read: Models Memorize or Generalize?

Tag: grokking