data engineering – Page 2 – Andrew Fairless, Ph.D.

What I Read: PostgreSQL

By Andrew Fairless on July 23, 2024May 24, 2024

https://ottertune.com/blog/the-part-of-postgresql-we-hate-the-most The part of PostgreSQL we hate the mostBohan Zhang | Andy PavloApr 26, 2023 “But as much as we love PostgreSQL at OtterTune, certain aspects of it are notContinue readingWhat I Read: PostgreSQL

What I Read: anti-patterns, data reuse

By Andrew Fairless on July 10, 2024May 6, 2024

https://www.milesmcbain.com/posts/data-analysis-reuse Patterns and anti-patterns of data analysis reuseMiles McBainMarch 10, 2024 “When I look at that function signature I hear the screams of matrices being ground to a fine powderContinue readingWhat I Read: anti-patterns, data reuse

What I Read: predicate pushdown

By Andrew Fairless on May 29, 2024March 23, 2024

https://pola.rs/posts/predicate-pushdown-query-optimizer/ The power of predicate pushdownPolars development teamTue, 19 Mar 2024 “…we will dive into the query optimizer and explain how one of the most important optimization rules works: predicateContinue readingWhat I Read: predicate pushdown

What I Read: How fast process CSV file

By Andrew Fairless on May 8, 2024March 7, 2024

https://datapythonista.me/blog/how-fast-can-we-process-a-csv-file How fast can we process a CSV fileMarc GarciaThu 22 February 2024 “…we’ll see in this blog post how to process a CSV file as fast as possible.”

What I Read: Scaling ChatGPT, Engineering Challenges

By Andrew Fairless on April 18, 2024March 1, 2024

https://newsletter.pragmaticengineer.com/p/scaling-chatgpt Scaling ChatGPT: Five Real-World Engineering ChallengesGergely OroszFeb 20, 2024 “Just one year after its launch, ChatGPT had more than 100M weekly users. In order to meet this explosive demand,Continue readingWhat I Read: Scaling ChatGPT, Engineering Challenges

What I Read: Probabilistic Linkage, Data Deduplication

By Andrew Fairless on April 15, 2024March 1, 2024

https://www.robinlinacre.com/fellegi_sunter_accuracy/ Why Probabilistic Linkage is More Accurate than Fuzzy Matching For Data DeduplicationRobin Linacre2023-10-24 “How effectively do different approaches to record linkage exploit the information in the data to makeContinue readingWhat I Read: Probabilistic Linkage, Data Deduplication

What I Read: Deploy Model

By Andrew Fairless on April 9, 2024March 1, 2024

https://outerbounds.com/blog/the-many-ways-to-deploy-a-model/ The Many Ways to Deploy a ModelHamel Husain, The Outerbounds TeamFebruary 6, 2024 “Over the past years, we have been helping companies deploy a wildly diverse set of MLContinue readingWhat I Read: Deploy Model

What I Read: SQL order

By Andrew Fairless on March 28, 2024March 23, 2024

https://lukianovihor.medium.com/sql-order-of-query-execution-8c7cd926400 SQL — order of query executionIhor LukianovSep 24, 2023 “To maximize your query’s speed on any SQL engine, it’s essential to have an understanding of the SQL execution order.”

What I Read: Database Disassembly

By Andrew Fairless on March 27, 2024February 5, 2024

https://materializedview.io/p/databases-are-falling-apart Databases Are Falling Apart: Database Disassembly and Its ImplicationsChris RiccominiJan 29, 2024 “Why are engineers taking databases apart and putting them back together, again?”

Tag: data engineering