transformer – Page 5 – Andrew Fairless, Ph.D.

What I Read: Geometric Deep Learning

By Andrew Fairless on April 10, 2023February 22, 2023

https://thegradient.pub/towards-geometric-deep-learning/ Towards Geometric Deep LearningMichael Bronstein18.Feb.2023 “Geometric Deep Learning is an umbrella term for approaches considering a broad class of ML problems from the perspectives of symmetry and invariance.”

What I Read: Transformer Inference Optimization

By Andrew Fairless on March 16, 2023January 30, 2023

https://lilianweng.github.io/posts/2023-01-10-inference-optimization/ Large Transformer Model Inference OptimizationLilian WengJanuary 10, 2023 “The extremely high inference cost, in both time and memory, is a big bottleneck for adopting a powerful transformer for solvingContinue readingWhat I Read: Transformer Inference Optimization

What I Read: Realtime User Actions in Recommendation

By Andrew Fairless on February 27, 2023January 9, 2023

https://medium.com/pinterest-engineering/how-pinterest-leverages-realtime-user-actions-in-recommendation-to-boost-homefeed-engagement-volume-165ae2e8cde8 How Pinterest Leverages Realtime User Actions in Recommendation to Boost Homefeed Engagement VolumeXue Xia, Software Engineer, Homefeed Ranking; Neng Gu, Software Engineer, Content & User Understanding; Dhruvil Deven Badani,Continue readingWhat I Read: Realtime User Actions in Recommendation

What I Read: Transformers Training

By Andrew Fairless on January 19, 2023December 16, 2022

https://www.borealisai.com/research-blogs/tutorial-17-transformers-iii-training/ Tutorial #17: Transformers III Training08/06/2021P. Xu, S. Prince “…we discuss challenges with transformer training dynamics and introduce some of the tricks that practitioners use to get transformers to converge.”

What I Read: Illustrated Stable Diffusion

By Andrew Fairless on November 29, 2022October 20, 2022

https://jalammar.github.io/illustrated-stable-diffusion/ The Illustrated Stable DiffusionJay AlammarVisualizing machine learning one concept at a time. “The ability to create striking visuals from text descriptions has a magical quality to it and pointsContinue readingWhat I Read: Illustrated Stable Diffusion

What I Read: Productizing Large Language Models

By Andrew Fairless on November 16, 2022September 27, 2022

https://blog.replit.com/llms Productizing Large Language ModelsWed Sep 21 2022by Amjad Masad, Samip Dahal, Luis Héctor Chávez “Large Language Models (LLMs)… can chat, write poetry, write code, and even do basic arithmetic.Continue readingWhat I Read: Productizing Large Language Models

What I Read: Transformers, Brain

By Andrew Fairless on November 9, 2022September 27, 2022

https://www.quantamagazine.org/how-ai-transformers-mimic-parts-of-the-brain-20220912/ How Transformers Seem to Mimic Parts of the BrainStephen OrnesSeptember 12, 2022 “Neural networks originally designed for language processing turn out to be great models of how our brainsContinue readingWhat I Read: Transformers, Brain

What I Read: Emergent Features

By Andrew Fairless on October 6, 2022August 22, 2022

LLM.int8() and Emergent Features LLM.int8() and Emergent Features2022-08-17 by Tim Dettmers “…I use advanced quantization methods to achieve no performance degradation transformer inference at scale that makes large models moreContinue readingWhat I Read: Emergent Features

What I Read: BLOOM Training

By Andrew Fairless on September 12, 2022August 3, 2022

https://huggingface.co/blog/bloom-megatron-deepspeed The Technology Behind BLOOM TrainingStas BekmanPublished July 14, 2022. “…training ever larger language models has become the norm… the hidden knowledge about how to train such models rarely getsContinue readingWhat I Read: BLOOM Training

What I Read: Transformers in computer vision

By Andrew Fairless on September 6, 2022August 3, 2022

https://theaisummer.com/transformers-computer-vision/ Transformers in computer vision: ViT architectures, tips, tricks and improvementsNikolas Adaloglouon2021-10-21 “You are probably already aware of the Vision Transformer (ViT)…. We will address questions like how can youContinue readingWhat I Read: Transformers in computer vision

Tag: transformer