Practical articles on AI, Data Engineering, Metadata-Driven Development, Snowflake, Enterprise Architecture, and Modern Data Platforms.
Real-world engineering lessons, technical deep dives, and implementation guides by Amit Singh · 11 articles published
Showing 4 articles tagged #data engineering
Banks already know how to capture changes at the transaction level. But what happens when a downstream system needs to know that a customer's risk score moved from 42 to 67 - not which five transactions caused it? That is the architectural gap CDF on Materialized Views is designed to close.
Source-to-Target Mappings sit at the center of every data pipeline, yet they are treated as throwaway documents. Here is why that needs to change - and what becomes possible when it does.
The promise of AI-assisted data engineering is compelling. But can a language model actually read a Source-to-Target Mapping and produce artifacts that are production-ready? Here is an honest look at what works, what does not, and what the path forward looks like.
Data engineering teams spend a disproportionate amount of time on work that is mechanical, predictable, and automatable. This article quantifies that cost and examines what it means for team capacity, delivery speed, and engineer satisfaction.