https://sergeylevine.substack.com/p/offline-rl-and-large-language-models Offline RL and Large Language ModelsSergey LevineDec 4 “What if the purpose of a language model should not be to generate text at all, at least not directly? …
What I Read: Transformers Training
https://www.borealisai.com/research-blogs/tutorial-17-transformers-iii-training/ Tutorial #17: Transformers III Training08/06/2021P. Xu, S. Prince “…we discuss challenges with transformer training dynamics and introduce some of the tricks that practitioners use to get transformers to converge.”