https://blog.kjamistan.com/how-memorization-happens-novelty.html How memorization happens: Novelty09 Dezember 2024 “…repeated text and images incentivize training data memorization, but that’s not the only training data that machine learning models memorize. Let’s take a
What I Read: impossible languages
https://www.quantamagazine.org/can-ai-models-show-us-how-people-learn-impossible-languages-point-a-way-20250113/ Can AI Models Show Us How People Learn? Impossible Languages Point a Way.Ben Brubaker1/13/25 11:00 AM “Certain grammatical rules never appear in any known language. By constructing artificial languages
What I Read: Autoencoders, Interpretability
https://adamkarvonen.github.io/machine_learning/2024/06/11/sae-intuitions.html An Intuitive Explanation of Sparse Autoencoders for LLM InterpretabilityAdam KarvonenJun 11, 2024 “Sparse Autoencoders (SAEs) have recently become popular for interpretability of machine learning models…”
What I Read: LLMs, School Math
https://towardsdatascience.com/understanding-llms-from-scratch-using-middle-school-math-e602d27ec876?gi=551c5bfd7f21 Understanding LLMs from Scratch Using Middle School MathRohit PatelOct 19, 2024 “In this article, we talk about how Large Language Models (LLMs) work, from scratch — assuming only that