https://dantelore.com/posts/simplest-data-pipeline/ World’s Simplest Data Pipeline?Dan TaylorNovember 13, 2022 “Data Engineering is very simple…. Single responsibility, immutable datasets, schema on write and so on all help to improve predictability and separate
What I Read: Data Pipeline Smoke Tests
https://dagster.io/blog/smoke-test-data-pipeline The Unreasonable Effectiveness of Data Pipeline Smoke TestsSandy RyzaOctober 19, 2022 “Data practitioners waste time writing unit tests to catch bugs they could have caught with smoke tests.”
What I Read: data catalogs, metadata
https://towardsdatascience.com/forrester-changed-the-way-they-think-about-data-catalogs-and-heres-what-you-need-to-know-139e13a5869d?gi=7192c997040d Forrester changed the way they think about data catalogs, and here’s what you need to knowThe latest sign of a major shift in how we think about metadataPrukalpaOct 4
What I Read: ML Engineering
https://www.shreya-shankar.com/phd-year-one/ Thoughts on ML Engineering After a Year of my PhDJuly 18, 2022Shreya Shankar “There are two types of business-critical MLEs. The first is the Task MLE…”