What I Read: Frameworks Scaling Deep Learning Training

https://medium.com/dataseries/microsoft-and-google-open-sourced-these-frameworks-based-on-their-work-scaling-deep-learning-c0510e907038

Microsoft and Google Open Sourced These Frameworks Based on Their Work Scaling Deep Learning Training
Google and Microsoft have recently released new frameworks for distributed deep learning training.
Jesus Rodriguez
Oct 26

“As models scale in order to achieve higher levels of accuracy, the training of those models becomes increasingly challenging…. training parallelism is a must in order to scale deep learning models. To address those challenges, Microsoft and Google have devoted months of research and engineering that resulted in the release of GPipe and PipeDream respectively.”