latency – Page 2 – Andrew Fairless, Ph.D.

What I Read: Scaling ChatGPT, Engineering Challenges

By Andrew Fairless on April 18, 2024March 1, 2024

https://newsletter.pragmaticengineer.com/p/scaling-chatgpt Scaling ChatGPT: Five Real-World Engineering ChallengesGergely OroszFeb 20, 2024 “Just one year after its launch, ChatGPT had more than 100M weekly users. In order to meet this explosive demand,Continue readingWhat I Read: Scaling ChatGPT, Engineering Challenges

What I Read: Deploy Model

By Andrew Fairless on April 9, 2024March 1, 2024

https://outerbounds.com/blog/the-many-ways-to-deploy-a-model/ The Many Ways to Deploy a ModelHamel Husain, The Outerbounds TeamFebruary 6, 2024 “Over the past years, we have been helping companies deploy a wildly diverse set of MLContinue readingWhat I Read: Deploy Model

What I Read: misleading GPU, CPU benchmarks

By Andrew Fairless on March 21, 2024February 5, 2024

https://pythonspeed.com/articles/gpu-vs-cpu/ Beware of misleading GPU vs CPU benchmarksItamar Turner-TrauringLast updated 17 Jan 2024, originally created 17 Jan 2024 “Unfortunately, while those speed-ups are impressive, they are also misleading. GPU-based librariesContinue readingWhat I Read: misleading GPU, CPU benchmarks

What I Read: vectorize wide PyTorch expressions

By Andrew Fairless on January 16, 2024November 7, 2023

https://probablymarcus.com/blocks/2023/10/19/vectorizing-wide-pytorch-expressions.html What happens when you vectorize wide PyTorch expressions?Marcus Lewis19 October 2023 “So we see that vectorization has three effects:It lets us keep the GPU busy even when inputs areContinue readingWhat I Read: vectorize wide PyTorch expressions

What I Read: Compiling NumPy

By Andrew Fairless on January 10, 2024November 7, 2023

https://pytorch.org/blog/compiling-numpy-code/ Compiling NumPy code into C++ or CUDA via torch.compileby Evgeni Burovski, Ralf Gommers and Mario LezcanoOctober 17, 2023 “This feature leverages PyTorch’s compiler to generate efficient fused vectorized codeContinue readingWhat I Read: Compiling NumPy

What I Read: Nvidia AI Supremacy Temporary

By Andrew Fairless on October 25, 2023October 5, 2023

https://petewarden.com/2023/09/10/why-nvidias-ai-supremacy-is-only-temporary/ Why Nvidia’s AI Supremacy is Only TemporarySeptember 10, 2023Pete Warden “…hopefully I’ve made a convincing case that there are strong structural reasons behind Nvidia’s success. Here’s how I seeContinue readingWhat I Read: Nvidia AI Supremacy Temporary

What I Read: speed up Python with Rust

By Andrew Fairless on August 24, 2023July 30, 2023

https://pythonspeed.com/articles/easiest-rust-python/ The easiest way to speed up Python with RustItamar Turner-Trauring7/9/23 8:00 PM “If you want to speed up some existing Python code, writing a compiled extension in Rust canContinue readingWhat I Read: speed up Python with Rust

What I Read: speed up Numba and NumPy

By Andrew Fairless on August 1, 2023June 15, 2023

https://pythonspeed.com/articles/speeding-up-numba/ Understanding CPUs can help speed up Numba and NumPy codeby Itamar Turner-TrauringLast updated 08 Jun 2023, originally created 07 Jun 2023 “In practice, your initial Numba code may beContinue readingWhat I Read: speed up Numba and NumPy

What I Read: smaller LLMs, more tokens

By Andrew Fairless on May 31, 2023May 1, 2023

https://www.harmdevries.com/post/model-size-vs-compute-overhead/ Go smol or go homeWhy we should train smaller LLMs on more tokensHarm de VriesApr 13, 2023 “However, for most use cases you should not train a compute-optimal LLMContinue readingWhat I Read: smaller LLMs, more tokens

What I Read: Bloom filter

By Andrew Fairless on April 26, 2023March 30, 2023

https://luminousmen.com/post/building-a-bloom-filter Building a Bloom filterluminousmen “In this post, we will explore the Bloom filter — a data structure that is ingenious in its simplicity and elegant in its design.”

Tag: latency