data engineering – Andrew Fairless, Ph.D.

What I Read: Polars vs pandas

By Andrew Fairless on March 19, 2025November 20, 2024

https://labs.quansight.org/blog/dataframe-group-by The Polars vs pandas difference nobody is talking aboutMarcoGorelliNovember 11, 2024 “We’ll then take a look at elementary aggregations with both the pandas and Polars APIs. Finally, we’ll lookContinue readingWhat I Read: Polars vs pandas

What I Read: Incremental Jobs, Data Quality

By Andrew Fairless on March 12, 2025November 20, 2024

https://jack-vanlightly.com/blog/2024/11/13/incremental-jobs-and-data-quality-are-on-a-collision-course-part-2-the-way-forward Incremental Jobs and Data Quality Are On a Collision Course – Part 2 – The Way ForwardJack VanlightlyNov 13, 2024 “The thing we’re missing right now is that theContinue readingWhat I Read: Incremental Jobs, Data Quality

What I Read: Declarative Data Stack

By Andrew Fairless on March 11, 2025November 17, 2024

https://www.rilldata.com/blog/the-rise-of-the-declarative-data-stack The Rise of the Declarative Data StackSimon SpätiOctober 16, 2024 “But what does this shift towards declarative data stacks mean? How does it change how we build and manageContinue readingWhat I Read: Declarative Data Stack

What I Read: data migrations

By Andrew Fairless on February 19, 2025November 9, 2024

https://yorickpeterse.com/articles/building-a-better-and-scalable-system-for-data-migrations Building a better and scalable system for data migrationsYorick PeterseOctober 24, 2024 “I’ve been thinking about what a better solution to data migrations might look like for a while,Continue readingWhat I Read: data migrations

What I Read: Big Data is Dead

By Andrew Fairless on November 4, 2024August 3, 2024

https://motherduck.com/blog/big-data-is-dead Big Data is DeadJordan Tigani2023/02/07 “Misunderstanding of this point leads to a lot of the discussion of Big Data, because techniques for dealing with large compute requirements are differentContinue readingWhat I Read: Big Data is Dead

What I Read: Data Flywheels, LLM

By Andrew Fairless on October 17, 2024July 20, 2024

https://www.sh-reya.com/blog/ai-engineering-flywheel Data Flywheels for LLM ApplicationsShreya ShankarJul 1, 2024 “This diagram illustrates my (idealized) architecture of an LLM pipeline, from input processing through evaluation and logging. It showcases ideas I’llContinue readingWhat I Read: Data Flywheels, LLM

What I Read: Musings on AI Engineering

By Andrew Fairless on September 16, 2024June 29, 2024

https://www.sh-reya.com/blog/ai-engineering-short Short Musings on AI Engineering and “Failed AI Projects”Shreya ShankarJun 24, 2024 “When we studied the traditional MLOps lifecycle, we found one common pattern across all successful ML products—theirContinue readingWhat I Read: Musings on AI Engineering

What I Read: SSD, Database

By Andrew Fairless on September 12, 2024June 29, 2024

https://cedardb.com/blog/ssd_latency Why Your SSD (Probably) Sucks and What Your Database Can Do About ItLukas VogelJune 18, 2024 “SSDs have effectively replaced spinning disks as the go-to solution for persistent storageContinue readingWhat I Read: SSD, Database

Tag: data engineering