https://www.borealisai.com/research-blogs/tutorial-17-transformers-iii-training/ Tutorial #17: Transformers III Training08/06/2021P. Xu, S. Prince “…we discuss challenges with transformer training dynamics and introduce some of the tricks that practitioners use to get transformers to converge.”
What I Read: Machine Learning, Building Blocks of Computing
https://www.quantamagazine.org/machine-learning-reimagines-the-building-blocks-of-computing-20220315/ Machine Learning Reimagines the Building Blocks of ComputingNick ThiemeContributing WriterMarch 15, 2022 “Traditional algorithms power complicated computational tools like machine learning. A new approach, called algorithms with predictions, uses
What I Read: Deep Learning From First Principles
https://horace.io/brrr_intro.html Making Deep Learning Go Brrrr From First PrinciplesHorace He “So, you want to improve the performance of your deep learning model. How might you approach such a task?”
What I Read: Researchers Build AI That Builds AI
https://www.quantamagazine.org/researchers-build-ai-that-builds-ai-20220125/ Researchers Build AI That Builds AIAnil AnanthaswamyJanuary 25, 2022“By using hypernetworks, researchers can now preemptively fine-tune artificial neural networks, saving some of the time and expense of training.”