data engineering – Page 10 – Andrew Fairless, Ph.D.

What I Read: What is a Data Mesh?

By Andrew Fairless on July 6, 2021June 22, 2021

https://cnr.sh/essays/what-the-heck-data-mesh What the Heck is a Data Mesh?!Chris Riccomini on June 8, 2021 “Yet, organizations don’t think of data as a product…. It would be madness to propose centralizing allContinue readingWhat I Read: What is a Data Mesh?

What I Read: Hiring Data Scientists

By Andrew Fairless on June 21, 2021June 7, 2021

https://towardsdatascience.com/stop-hiring-data-scientists-improve-ml-product-team-retention-8ae04dac406e?gi=175fec4d5962 Stop (and Start) Hiring Data ScientistsThree “stop & start” tactics to improve talent retentionIan XiaoMay 14 “… we’ve been overly spotlighting the algorithmic work and success of data scienceContinue readingWhat I Read: Hiring Data Scientists

What I Read: Feature stores

By Andrew Fairless on May 31, 2021May 14, 2021

https://www.kdnuggets.com/2021/05/feature-stores-how-avoid-feeling-every-day-is-groundhog-day.html Feature stores – how to avoid feeling that every day is Groundhog DayBy Monte Zweben, CEO, Splice Machine. “Feature stores stop the duplication of each task in the MLContinue readingWhat I Read: Feature stores

What I Read: Models of Data Science teams

By Andrew Fairless on May 11, 2021April 17, 2021

https://www.marcosantoni.com/chess_vs_checkers_teams.html Marco SantoniModels of Data Science teams: Chess vs Checkers “One of the key decisions to take when building a data science team is the mix of roles…. I’ll compareContinue readingWhat I Read: Models of Data Science teams

What I Read: Understanding Key-Value Databases

By Andrew Fairless on April 29, 2021April 10, 2021

https://www.kdnuggets.com/2021/04/nosql-explained-understanding-key-value-databases.html NoSQL Explained: Understanding Key-Value DatabasesBy Alex Williams, Hosting Data UK “Among the four big NoSQL database types, key-value stores are probably the most popular ones due to their simplicityContinue readingWhat I Read: Understanding Key-Value Databases

What I Read: Common Errors when Debugging Airflow DAGs

By Andrew Fairless on April 14, 2021April 2, 2021

https://www.astronomer.io/blog/7-common-errors-to-check-when-debugging-airflow-dag 7 Common Errors to Check when Debugging Airflow DAGsTasks not running? DAG stuck? Logs nowhere to be found? We’ve been there. Here’s a list of common snags and someContinue readingWhat I Read: Common Errors when Debugging Airflow DAGs

What I Read: Kedro Pipelines with Airflow

By Andrew Fairless on March 30, 2021March 15, 2021

https://medium.com/quantumblack/kedro-airflow-0-4-0-orchestrating-kedro-pipelines-with-airflow-23fb1283f24d Kedro-Airflow 0.4.0 — Orchestrating Kedro Pipelines with AirflowThe Kedro team and Astronomer have released Kedro-Airflow 0.4.0 to help you develop modular, maintainable & reproducible code with orchestration superpowers!QuantumBlack, aContinue readingWhat I Read: Kedro Pipelines with Airflow

What I Read: Long Live Data Discovery

By Andrew Fairless on March 1, 2021March 1, 2021

https://towardsdatascience.com/data-catalogs-are-dead-long-live-data-discovery-a0dc8d02bd34 Data Catalogs Are Dead; Long Live Data DiscoveryWhy we need to rethink our approach to metadata management and data governanceBarr MosesDec 14 “Traditional data catalogs and governance methodologies typicallyContinue readingWhat I Read: Long Live Data Discovery

What I Watch: Future of Data Engineering

By Andrew Fairless on March 1, 2021February 27, 2021

https://youtu.be/ZZr9oE4Oa5U Future of Data EngineeringDec 16, 2019InfoQ “Chris Riccomini talks about the current state-of-the-art in data pipelines and data warehousing, and shares some of the solutions to current problems dealingContinue readingWhat I Watch: Future of Data Engineering

What I Read: Feature Store vs Data Warehouse

By Andrew Fairless on February 25, 2021February 18, 2021

https://www.logicalclocks.com/blog/feature-store-vs-data-warehouse ML Engineer Guide: Feature Store vs Data WarehouseOctober 8, 2020ML Best Practicesby Jim Dowling “The feature store is a data warehouse of features for machine learning (ML). Architecturally, itContinue readingWhat I Read: Feature Store vs Data Warehouse

Tag: data engineering