https://www.quantamagazine.org/tiny-language-models-thrive-with-gpt-4-as-a-teacher-20231005/ Tiny Language Models Come of AgeBen Brubaker10/5/23 10:50 AM “To better understand how neural networks learn to simulate writing, researchers trained simpler versions on synthetic children’s stories.”
What I Read: To Understand Transformers, Focus on Attention
https://drscotthawley.github.io/blog/posts/Transformers1-Attention.html To Understand Transformers, Focus on AttentionScott H. HawleyAugust 21, 2023 “To Understand Transformers, Focus on Attention”