https://www.borealisai.com/research-blogs/tutorial-17-transformers-iii-training/
Tutorial #17: Transformers III Training
08/06/2021
P. Xu, S. Prince
“…we discuss challenges with transformer training dynamics and introduce some of the tricks that practitioners use to get transformers to converge.”