efficiency – Andrew Fairless, Ph.D.

What I Read: Visual Guide, Quantization

By Andrew Fairless on October 30, 2024August 3, 2024

https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization A Visual Guide to Quantization, Demystifying the Compression of Large Language ModelsMaarten GrootendorstJul 22, 2024 “…I will introduce the field of quantization in the context of language modeling andContinue readingWhat I Read: Visual Guide, Quantization

What I Read: History, Transformer

By Andrew Fairless on October 28, 2024July 22, 2024

Shaping the Future of AI from the History of TransformerStanford CS25: V4 I Hyung Won Chung of OpenAIStanford Online “I will provide a highly-opinionated view on the early history ofContinue readingWhat I Read: History, Transformer

What I Read: string ufuncs, NumPy 2.0

By Andrew Fairless on August 27, 2024June 15, 2024

https://labs.quansight.org/blog/numpy-string-ufuncs Writing fast string ufuncs for NumPy 2.0Lysandros NikolaouPublished May 21, 2024 “After a huge amount of work from many people, NumPy 2.0.0 will soon be released, the first NumPyContinue readingWhat I Read: string ufuncs, NumPy 2.0

What I Read: Ring Attention

By Andrew Fairless on July 8, 2024April 23, 2024

https://coconut-mode.com/posts/ring-attention Ring Attention ExplainedKilian Haefeli, Simon Zirui Guo, Bonnie Li10 Apr 2024 “Context length in Large Language Models has expanded rapidly…. What if we we could use multiple devices toContinue readingWhat I Read: Ring Attention

What I Read: Linear Algebra, Random

By Andrew Fairless on June 17, 2024April 15, 2024

https://youtu.be/6htbyY3rH1w?si=IXTrcoIReps_ftFq Is the Future of Linear Algebra.. Random?Mutual Information “Randomization is arguably the most exciting and innovative idea to have hit linear algebra in a long time.”

What I Read: Data Selection, LLMs

By Andrew Fairless on June 12, 2024April 15, 2024

https://www.cs.princeton.edu/~smalladi/blog/2024/04/04/dataselection Using LESS Data to Tune Models: Data Selection in the Era of LLMsMengzhou Xia and Sadhika MalladiApril 04 2024 “We describe how data selection for modern-day LLMs differs fromContinue readingWhat I Read: Data Selection, LLMs

What I Read: Mamba Explained

By Andrew Fairless on June 10, 2024April 15, 2024

https://thegradient.pub/mamba-explained Mamba ExplainedKola Ayonrinde27.Mar.2024 “Mamba promises similar performance (and crucially similar scaling laws) as the Transformer whilst being feasible at long sequence lengths (say 1 million tokens).”

What I Read: speed up code, Numba

By Andrew Fairless on June 5, 2024April 15, 2024

https://pythonspeed.com/articles/slow-numba The wrong way to speed up your code with Numbaby Itamar Turner-Trauring21 Mar 2024 “If your NumPy-based code is too slow, you can sometimes use Numba to speed itContinue readingWhat I Read: speed up code, Numba

What I Read: binary vector search

By Andrew Fairless on June 4, 2024April 15, 2024

https://blog.pgvecto.rs/my-binary-vector-search-is-better-than-your-fp32-vectors My binary vector search is better than your FP32 vectorsBinary Vector Search: The 30x Memory Reduction Revolution with Preserved AccuracyCe GaoMar 22, 2024 “…binary vector search… shows promise inContinue readingWhat I Read: binary vector search

What I Read: 1-bit LLMs, 1.58 Bits

By Andrew Fairless on May 14, 2024March 11, 2024

https://arxiv.org/abs/2402.17764 The Era of 1-bit LLMs: All Large Language Models are in 1.58 BitsShuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, JilongContinue readingWhat I Read: 1-bit LLMs, 1.58 Bits

Tag: efficiency