Andy Leonard
I recently wrote a series of articles about Data Engineering Patterns. The four articles focus on the differences between software engineering and data engineering and the need to apply software engineering principles to data engineering. But I also cover why that’s not as simple as it sounds. the...
Andy Leonard
What a month of naming posts was really all about [originally posted at Engineer of Data Substack] I have spent a chunk of a career in the part of data that has no name. Not the dashboards. Not the models everyone lines up to demo. The layer underneath – the part that moves the data, in the right...
Andy Leonard
Governance is downstream of a name [originally posted at Engineer of Data Substack] Last week, I wrote: Observable Run – every execution leaves a durable, queryable record of what ran, with what, and how it ended. I called it a pattern. It is also a door. Behind it is the question every data team...
Andy Leonard
Crafting the beginning of a vocabulary [originally posted at Engineer of Data Substack] Last week, I wrote: Stateful problems need their own names. We never wrote them. You’ve written idempotent loads, restart logic, dependency chains, promotion configs. By hand. Correctly enough to ship. You...
Andy Leonard
Data engineering in a software engineering world [originally posted at Engineer of Data Substack] Your orchestration runs. It Just Works every night. You built something real and that is not nothing. The ache isn’t that it’s broken – it’s that it lives in three people’s heads and a folder of...