4 ms·
Short version: - data warehouse: schema on write. you have to know the end form before you load it. breaks every time upstream changes (a lot, in this world)
by claytonjy 4y ago
Short version:
- data warehouse: schema on write. you have to know the end form before you load it. breaks every time upstream changes (a lot, in this world)
- data lake: schema on read. load everything into S3 and deal with it later. Mongo for data platforms
- data lakehouse: something in between. store everything loosely like a lake, but have in-lakehouse processes present user-friendly transforms or views like a warehouse. Made possible by cheap storage (parquet on S3), reduces schema breakage by keeping both sides of the T in the same place
- DougBTX 4y agoMaterialised views for cloud storage?