3 ms·
It's very simple: Schema on write is upfront costs for lower per unit costs later. Schema on read is no upfront costs and then higher per unit costs thereafte
by kthejoker2 5y ago
It's very simple:
Schema on write is upfront costs for lower per unit costs later.
Schema on read is no upfront costs and then higher per unit costs thereafter.
The value of schema on write is based on how much lower the per unit costs will be, how many units you expect, and how much the upfront costs will be.
Most people in ELT circles completely discount the per unit costs (many vendors are indeed incentivized to maximize these costs)
And the flipside is a lot of ETL is somewhere between YAGNI and unnecessary extravagance.
But broadly anyone who recommends one or the other without showing even a back of the napkin analysis of these costs are not serious data engineers.
Ps
Three other real world metaphors for schema on read vs schema on write:
* renting vs a mortgage
* eating out every night vs learning how to cook
* (pre pandemic) living far away from work to save money vs living closer to work.