4 ms·
I'm not too deep into the data side of things, but this is interesting to think about. Aren't stuff like data lakes and warehouses supposed to address the need
by ajcp 6y ago
I'm not too deep into the data side of things, but this is interesting to think about.
Aren't stuff like data lakes and warehouses supposed to address the need for a centralized datastore?
Outside of perhaps an easy-to-apply interface, what benefit would a data hub provide over just streaming duplicates from all of your databases into a single data lake like Snowflake?
- willvarfar 6y agoWant to cross ref the erp database with that stuff team x has in a lake and join with what team z has in a dwh? You don’t need pipelines and jobs and working out where to store that data... you just need a hub! And you can query it ad hoc too. We used to have to copy and shuffle data into centralized systems so we can query and join it. Data hubs do away with all that. Stop needing to think about storage. Instead, it’s just about access. There have always been fringe tools, eg I once did some cool stuff with MySQL spider engine. But modern systems like Trino (formerly called Presto) make it easy. And, I predict, will hit the mainstream soon.
- ajcp 6y ago> Stop needing to think about storage. Instead, it’s just about access. 100% resonates when you put it that way. Thanks for the explanation!