latency – Andrew Fairless, Ph.D.

What I Read: Generative AI Platform

By Andrew Fairless on October 29, 2024August 3, 2024

https://huyenchip.com//2024/07/25/genai-platform.html Building A Generative AI PlatformChip Huyen7/24/24 “This post outlines the common components of a generative AI platform, what they do, and how they are implemented.”

What I Read: PostgreSQL

By Andrew Fairless on July 23, 2024May 24, 2024

https://ottertune.com/blog/the-part-of-postgresql-we-hate-the-most The part of PostgreSQL we hate the mostBohan Zhang | Andy PavloApr 26, 2023 “But as much as we love PostgreSQL at OtterTune, certain aspects of it are notContinue readingWhat I Read: PostgreSQL

What I Read: Linear Algebra, Random

By Andrew Fairless on June 17, 2024April 15, 2024

https://youtu.be/6htbyY3rH1w?si=IXTrcoIReps_ftFq Is the Future of Linear Algebra.. Random?Mutual Information “Randomization is arguably the most exciting and innovative idea to have hit linear algebra in a long time.”

What I Read: speed up code, Numba

By Andrew Fairless on June 5, 2024April 15, 2024

https://pythonspeed.com/articles/slow-numba The wrong way to speed up your code with Numbaby Itamar Turner-Trauring21 Mar 2024 “If your NumPy-based code is too slow, you can sometimes use Numba to speed itContinue readingWhat I Read: speed up code, Numba

What I Read: binary vector search

By Andrew Fairless on June 4, 2024April 15, 2024

https://blog.pgvecto.rs/my-binary-vector-search-is-better-than-your-fp32-vectors My binary vector search is better than your FP32 vectorsBinary Vector Search: The 30x Memory Reduction Revolution with Preserved AccuracyCe GaoMar 22, 2024 “…binary vector search… shows promise inContinue readingWhat I Read: binary vector search

What I Read: Modal’s web infrastructure

By Andrew Fairless on May 27, 2024March 21, 2024

https://modal.com/blog/serverless-http Lambda on hard mode: Inside Modal’s web infrastructureEric ZhangMarch 14, 2024 “At Modal, we built an HTTP and WebSocket stack on our platform. In other words, your serverless functionsContinue readingWhat I Read: Modal’s web infrastructure

What I Read: 1-bit LLMs, 1.58 Bits

By Andrew Fairless on May 14, 2024March 11, 2024

https://arxiv.org/abs/2402.17764 The Era of 1-bit LLMs: All Large Language Models are in 1.58 BitsShuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, JilongContinue readingWhat I Read: 1-bit LLMs, 1.58 Bits

What I Read: Scaling ChatGPT, Engineering Challenges

By Andrew Fairless on April 18, 2024March 1, 2024

https://newsletter.pragmaticengineer.com/p/scaling-chatgpt Scaling ChatGPT: Five Real-World Engineering ChallengesGergely OroszFeb 20, 2024 “Just one year after its launch, ChatGPT had more than 100M weekly users. In order to meet this explosive demand,Continue readingWhat I Read: Scaling ChatGPT, Engineering Challenges

What I Read: Deploy Model

By Andrew Fairless on April 9, 2024March 1, 2024

https://outerbounds.com/blog/the-many-ways-to-deploy-a-model/ The Many Ways to Deploy a ModelHamel Husain, The Outerbounds TeamFebruary 6, 2024 “Over the past years, we have been helping companies deploy a wildly diverse set of MLContinue readingWhat I Read: Deploy Model

What I Read: misleading GPU, CPU benchmarks

By Andrew Fairless on March 21, 2024February 5, 2024

https://pythonspeed.com/articles/gpu-vs-cpu/ Beware of misleading GPU vs CPU benchmarksItamar Turner-TrauringLast updated 17 Jan 2024, originally created 17 Jan 2024 “Unfortunately, while those speed-ups are impressive, they are also misleading. GPU-based librariesContinue readingWhat I Read: misleading GPU, CPU benchmarks

Tag: latency