3 ms·
Somewhat outdated view. This may be the current stack, but its outdated now and is slowly being replaced. The new view is not big data pipelines and ETL jobs, i
by zaptheimpaler 6y ago
Somewhat outdated view. This may be the current stack, but its outdated now and is slowly being replaced. The new view is not big data pipelines and ETL jobs, its lambda architecture, live aggregations/materialized views and simple SQL queries on large data warehouses that hide the underlying details. The batch model may still apply to ML I guess, but I'm no expert there.
- molsongolden 6y agoAny resources/guides you'd recommend?
- tomnipotent 6y agoThis is true for only a very limited subset of data producers that need real-time or near real-time data included in ML models. For 99% of the rest, batch processing is just fine and considerably more economical.