transformer – Page 3 – Andrew Fairless, Ph.D.

What I Read: Visualizing Matrix Multiplication

By Andrew Fairless on November 30, 2023October 5, 2023

https://pytorch.org/blog/inside-the-matrix/ Inside the Matrix: Visualizing Matrix Multiplication, Attention and Beyondby Team PyTorchSeptember 25, 2023 “Matrix multiplications (matmuls) are the building blocks of today’s ML models. This note presents mm, aContinue readingWhat I Read: Visualizing Matrix Multiplication

What I Read: Optimizing LLM in production

By Andrew Fairless on November 6, 2023October 5, 2023

https://huggingface.co/blog/optimize-llm Optimizing your LLM in productionSeptember 15, 2023Patrick von Platen “…efficient LLM deployment…. pros and cons of adopting lower precision, provide a comprehensive exploration of the latest attention algorithms, andContinue readingWhat I Read: Optimizing LLM in production

What I Read: LLM Training, RLHF

By Andrew Fairless on October 30, 2023October 5, 2023

https://magazine.sebastianraschka.com/p/llm-training-rlhf-and-its-alternatives LLM Training: RLHF and Its AlternativesSebastian Raschka, PhDSep 10, 2023 “RLHF is an integral part of the modern LLM training pipeline due to its ability to incorporate human preferencesContinue readingWhat I Read: LLM Training, RLHF

What I Read: To Understand Transformers, Focus on Attention

By Andrew Fairless on October 16, 2023September 4, 2023

https://drscotthawley.github.io/blog/posts/Transformers1-Attention.html To Understand Transformers, Focus on AttentionScott H. HawleyAugust 21, 2023 “To Understand Transformers, Focus on Attention”

What I Read: LLM research

By Andrew Fairless on October 10, 2023September 4, 2023

https://huyenchip.com//2023/08/16/llm-research-open-challenges.html Open challenges in LLM research8/15/23 8:00 PM “Never before in my life had I seen so many smart people working on the same goal: making LLMs better… I noticedContinue readingWhat I Read: LLM research

What I Read: Multimodal, Embeddings

By Andrew Fairless on October 5, 2023September 4, 2023

https://app.twelvelabs.io/blog/multimodal-embeddings The Multimodal Evolution of Vector EmbeddingsJames LeAugust 9, 2023 “…companies collect and feed hundreds of millions of terabytes of multimodal data into deep learning models. As a result, embeddings…Continue readingWhat I Read: Multimodal, Embeddings

What I Read: Attention Off By One

By Andrew Fairless on September 13, 2023September 4, 2023

https://www.evanmiller.org/attention-is-off-by-one.html Attention Is Off By OneBy Evan MillerJuly 24, 2023 “…the current generation of AI models have an off-by-one error in a crucial place, and it’s making everyone’s Transformer modelsContinue readingWhat I Read: Attention Off By One

What I Read: LLMs

By Andrew Fairless on September 7, 2023August 1, 2023

https://willthompson.name/what-we-know-about-llms-primer What We Know About LLMs (Primer)Will Thompson (Twitter)July 23, 2023 “…it is worth reflecting on what we concretely know about LLMs at this point in time and how theseContinue readingWhat I Read: LLMs

What I Read: Accelerating PyTorch

By Andrew Fairless on September 6, 2023August 1, 2023

https://magazine.sebastianraschka.com/p/accelerating-pytorch-model-training Accelerating PyTorch Model TrainingUsing Mixed-Precision and Fully Sharded Data ParallelismSebastian Raschka, PhDJun 26, 2023 “…how to scale PyTorch model training with minimal code changes. The focus here is onContinue readingWhat I Read: Accelerating PyTorch

What I Read: What, Why ChatGPT

By Andrew Fairless on July 12, 2023June 15, 2023

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ What Is ChatGPT Doing … and Why Does It Work?February 14, 2023Stephen Wolfram “That ChatGPT can automatically generate something that reads even superficially like human-written text is remarkable, andContinue readingWhat I Read: What, Why ChatGPT

Tag: transformer