attention – Page 2 – Andrew Fairless, Ph.D.

What I Read: Scaling ChatGPT, Engineering Challenges

By Andrew Fairless on April 18, 2024March 1, 2024

https://newsletter.pragmaticengineer.com/p/scaling-chatgpt Scaling ChatGPT: Five Real-World Engineering ChallengesGergely OroszFeb 20, 2024 “Just one year after its launch, ChatGPT had more than 100M weekly users. In order to meet this explosive demand,Continue readingWhat I Read: Scaling ChatGPT, Engineering Challenges

What I Read: Self-Attention in GPT

By Andrew Fairless on March 4, 2024January 25, 2024

https://twiecki.io/blog/2024/01/04/ An Intuitive Guide to Self-Attention in GPT: The Venetian MasqueradeThomas WieckiJanuary 4, 2024 “In AI, especially with something as intricate as self-attention, it’s easy to get lost in theContinue readingWhat I Read: Self-Attention in GPT

What I Read: Research Directions

By Andrew Fairless on February 21, 2024January 7, 2024

https://nlpnewsletter.substack.com/p/nlp-research-in-the-era-of-llms NLP Research in the Era of LLMs5 Key Research Directions Without Much ComputeSebastian RuderDec 19, 2023 “In an era where running state-of-the-art models requires a garrison of expensive GPUs,Continue readingWhat I Read: Research Directions

What I Read: Visualizing Matrix Multiplication

By Andrew Fairless on November 30, 2023October 5, 2023

https://pytorch.org/blog/inside-the-matrix/ Inside the Matrix: Visualizing Matrix Multiplication, Attention and Beyondby Team PyTorchSeptember 25, 2023 “Matrix multiplications (matmuls) are the building blocks of today’s ML models. This note presents mm, aContinue readingWhat I Read: Visualizing Matrix Multiplication

What I Read: Optimizing LLM in production

By Andrew Fairless on November 6, 2023October 5, 2023

https://huggingface.co/blog/optimize-llm Optimizing your LLM in productionSeptember 15, 2023Patrick von Platen “…efficient LLM deployment…. pros and cons of adopting lower precision, provide a comprehensive exploration of the latest attention algorithms, andContinue readingWhat I Read: Optimizing LLM in production

What I Read: To Understand Transformers, Focus on Attention

By Andrew Fairless on October 16, 2023September 4, 2023

https://drscotthawley.github.io/blog/posts/Transformers1-Attention.html To Understand Transformers, Focus on AttentionScott H. HawleyAugust 21, 2023 “To Understand Transformers, Focus on Attention”

What I Read: Multimodal, Embeddings

By Andrew Fairless on October 5, 2023September 4, 2023

https://app.twelvelabs.io/blog/multimodal-embeddings The Multimodal Evolution of Vector EmbeddingsJames LeAugust 9, 2023 “…companies collect and feed hundreds of millions of terabytes of multimodal data into deep learning models. As a result, embeddings…Continue readingWhat I Read: Multimodal, Embeddings

What I Read: Attention Off By One

By Andrew Fairless on September 13, 2023September 4, 2023

https://www.evanmiller.org/attention-is-off-by-one.html Attention Is Off By OneBy Evan MillerJuly 24, 2023 “…the current generation of AI models have an off-by-one error in a crucial place, and it’s making everyone’s Transformer modelsContinue readingWhat I Read: Attention Off By One

What I Read: LLMs

By Andrew Fairless on September 7, 2023August 1, 2023

https://willthompson.name/what-we-know-about-llms-primer What We Know About LLMs (Primer)Will Thompson (Twitter)July 23, 2023 “…it is worth reflecting on what we concretely know about LLMs at this point in time and how theseContinue readingWhat I Read: LLMs

What I Read: What, Why ChatGPT

By Andrew Fairless on July 12, 2023June 15, 2023

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ What Is ChatGPT Doing … and Why Does It Work?February 14, 2023Stephen Wolfram “That ChatGPT can automatically generate something that reads even superficially like human-written text is remarkable, andContinue readingWhat I Read: What, Why ChatGPT

Tag: attention