transformer – Page 4 – Andrew Fairless, Ph.D.

What I Read: Accelerating PyTorch

By Andrew Fairless on September 6, 2023August 1, 2023

https://magazine.sebastianraschka.com/p/accelerating-pytorch-model-training Accelerating PyTorch Model TrainingUsing Mixed-Precision and Fully Sharded Data ParallelismSebastian Raschka, PhDJun 26, 2023 “…how to scale PyTorch model training with minimal code changes. The focus here is onContinue readingWhat I Read: Accelerating PyTorch

What I Read: What, Why ChatGPT

By Andrew Fairless on July 12, 2023June 15, 2023

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ What Is ChatGPT Doing … and Why Does It Work?February 14, 2023Stephen Wolfram “That ChatGPT can automatically generate something that reads even superficially like human-written text is remarkable, andContinue readingWhat I Read: What, Why ChatGPT

What I Read: In-Context Learning

By Andrew Fairless on July 11, 2023May 31, 2023

https://thegradient.pub/in-context-learning-in-context/ In-Context Learning, In ContextDaniel Bashir29.Apr.2023 “…in-context learning (ICL). In this paradigm, an LLM learns to solve a new task at inference time (without any change to its weights) byContinue readingWhat I Read: In-Context Learning

What I Read: Against LLM

By Andrew Fairless on June 29, 2023May 31, 2023

https://explosion.ai/blog/against-llm-maximalism Against LLM maximalismby Matthew HonnibalMay 18, 2023 “However, LLMs are not a direct solution to most of the NLP use-cases companies have been working on… if you want toContinue readingWhat I Read: Against LLM

What I Read: Few Shot, Recommenders, LLMs

By Andrew Fairless on May 30, 2023May 1, 2023

https://blog.reachsumit.com/posts/2023/04/llm-for-recsys/ Zero and Few Shot Recommender Systems based on Large Language ModelsSumit Kumar2023-04-10 “Many researchers have recently proposed different approaches to building recommender systems using LLMs. These methods convert differentContinue readingWhat I Read: Few Shot, Recommenders, LLMs

What I Read: Geometric Deep Learning

By Andrew Fairless on April 10, 2023February 22, 2023

https://thegradient.pub/towards-geometric-deep-learning/ Towards Geometric Deep LearningMichael Bronstein18.Feb.2023 “Geometric Deep Learning is an umbrella term for approaches considering a broad class of ML problems from the perspectives of symmetry and invariance.”

What I Read: Transformer Inference Optimization

By Andrew Fairless on March 16, 2023January 30, 2023

https://lilianweng.github.io/posts/2023-01-10-inference-optimization/ Large Transformer Model Inference OptimizationLilian WengJanuary 10, 2023 “The extremely high inference cost, in both time and memory, is a big bottleneck for adopting a powerful transformer for solvingContinue readingWhat I Read: Transformer Inference Optimization

What I Read: Realtime User Actions in Recommendation

By Andrew Fairless on February 27, 2023January 9, 2023

https://medium.com/pinterest-engineering/how-pinterest-leverages-realtime-user-actions-in-recommendation-to-boost-homefeed-engagement-volume-165ae2e8cde8 How Pinterest Leverages Realtime User Actions in Recommendation to Boost Homefeed Engagement VolumeXue Xia, Software Engineer, Homefeed Ranking; Neng Gu, Software Engineer, Content & User Understanding; Dhruvil Deven Badani,Continue readingWhat I Read: Realtime User Actions in Recommendation

What I Read: Transformers Training

By Andrew Fairless on January 19, 2023December 16, 2022

https://www.borealisai.com/research-blogs/tutorial-17-transformers-iii-training/ Tutorial #17: Transformers III Training08/06/2021P. Xu, S. Prince “…we discuss challenges with transformer training dynamics and introduce some of the tricks that practitioners use to get transformers to converge.”

Tag: transformer