What I Read: Distributed Training, Finetuning

https://sumanthrh.com/post/distributed-and-efficient-finetuning/

Everything about Distributed Training and Efficient Finetuning
Sumanth R Hegde
Last updated on Oct 13, 2023

“practical guidelines and gotchas with multi-GPU and multi-node training”