transformer – Page 3 – Andrew Fairless, Ph.D.

What I Read: Limits of Transformers on Compositionality

By Andrew Fairless on February 13, 2024December 19, 2023

https://arxiv.org/abs/2305.18654 Faith and Fate: Limits of Transformers on CompositionalityNouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, JenaContinue readingWhat I Read: Limits of Transformers on Compositionality

What I Read: Distributed Training, Finetuning

By Andrew Fairless on December 20, 2023November 7, 2023

https://sumanthrh.com/post/distributed-and-efficient-finetuning/ Everything about Distributed Training and Efficient FinetuningSumanth R HegdeLast updated on Oct 13, 2023 “practical guidelines and gotchas with multi-GPU and multi-node training”

What I Read: Visualizing Matrix Multiplication

By Andrew Fairless on November 30, 2023October 5, 2023

https://pytorch.org/blog/inside-the-matrix/ Inside the Matrix: Visualizing Matrix Multiplication, Attention and Beyondby Team PyTorchSeptember 25, 2023 “Matrix multiplications (matmuls) are the building blocks of today’s ML models. This note presents mm, aContinue readingWhat I Read: Visualizing Matrix Multiplication

What I Read: Optimizing LLM in production

By Andrew Fairless on November 6, 2023October 5, 2023

https://huggingface.co/blog/optimize-llm Optimizing your LLM in productionSeptember 15, 2023Patrick von Platen “…efficient LLM deployment…. pros and cons of adopting lower precision, provide a comprehensive exploration of the latest attention algorithms, andContinue readingWhat I Read: Optimizing LLM in production

What I Read: LLM Training, RLHF

By Andrew Fairless on October 30, 2023October 5, 2023

https://magazine.sebastianraschka.com/p/llm-training-rlhf-and-its-alternatives LLM Training: RLHF and Its AlternativesSebastian Raschka, PhDSep 10, 2023 “RLHF is an integral part of the modern LLM training pipeline due to its ability to incorporate human preferencesContinue readingWhat I Read: LLM Training, RLHF

What I Read: To Understand Transformers, Focus on Attention

By Andrew Fairless on October 16, 2023September 4, 2023

https://drscotthawley.github.io/blog/posts/Transformers1-Attention.html To Understand Transformers, Focus on AttentionScott H. HawleyAugust 21, 2023 “To Understand Transformers, Focus on Attention”

What I Read: LLM research

By Andrew Fairless on October 10, 2023September 4, 2023

https://huyenchip.com//2023/08/16/llm-research-open-challenges.html Open challenges in LLM research8/15/23 8:00 PM “Never before in my life had I seen so many smart people working on the same goal: making LLMs better… I noticedContinue readingWhat I Read: LLM research

What I Read: Multimodal, Embeddings

By Andrew Fairless on October 5, 2023September 4, 2023

https://app.twelvelabs.io/blog/multimodal-embeddings The Multimodal Evolution of Vector EmbeddingsJames LeAugust 9, 2023 “…companies collect and feed hundreds of millions of terabytes of multimodal data into deep learning models. As a result, embeddings…Continue readingWhat I Read: Multimodal, Embeddings

What I Read: Attention Off By One

By Andrew Fairless on September 13, 2023September 4, 2023

https://www.evanmiller.org/attention-is-off-by-one.html Attention Is Off By OneBy Evan MillerJuly 24, 2023 “…the current generation of AI models have an off-by-one error in a crucial place, and it’s making everyone’s Transformer modelsContinue readingWhat I Read: Attention Off By One

What I Read: LLMs

By Andrew Fairless on September 7, 2023August 1, 2023

https://willthompson.name/what-we-know-about-llms-primer What We Know About LLMs (Primer)Will Thompson (Twitter)July 23, 2023 “…it is worth reflecting on what we concretely know about LLMs at this point in time and how theseContinue readingWhat I Read: LLMs

Tag: transformer