4 ms·
Data Hubs. It’s queriable like a database but it doesn’t store your data - it proxies all your other databases and data lakes and stuff, and let’s you join bet
by willvarfar 6y ago
Data Hubs.
It’s queriable like a database but it doesn’t store your data - it proxies all your other databases and data lakes and stuff, and let’s you join between them.
Trino is a great example.
- ed25519FUUU 6y agoSounds like graphQL
- abraxas 6y agois that stuff like Presto?
- giantg2 6y agoI've used Presto within Hue. I'm not sure if Presto is, but I know Hue is.
- dswalter 6y agoTrino is the community-open-source fork of Presto (they were recently required to change the name from PrestoSQL, which they had been using to make the Presto lineage clear). (not affiliated in any way). https://trino.io/ https://trino.io/
- giantg2 6y agoI don't think these are really under the radar. We have Hue. We also have other apps that act as data hubs, but are slightly more constrained.
- ajcp 6y agoI'm not too deep into the data side of things, but this is interesting to think about. Aren't stuff like data lakes and warehouses supposed to address the need for a centralized datastore? Outside of perhaps an easy-to-apply interface, what benefit would a data hub provide over just streaming duplicates from all of your databases into a single data lake like Snowflake?
- willvarfar 6y agoWant to cross ref the erp database with that stuff team x has in a lake and join with what team z has in a dwh? You don’t need pipelines and jobs and working out where to store that data... you just need a hub! And you can query it ad hoc too. We used to have to copy and shuffle data into centralized systems so we can query and join it. Data hubs do away with all that. Stop needing to think about storage. Instead, it’s just about access. There have always been fringe tools, eg I once did some cool stuff with MySQL spider engine. But modern systems like Trino (formerly called Presto) make it easy. And, I predict, will hit the mainstream soon.
- ajcp 6y ago> Stop needing to think about storage. Instead, it’s just about access. 100% resonates when you put it that way. Thanks for the explanation!
- busterarm 6y agoPostgres Foreign Data Wrappers
- aheilbut 6y agolike DB2 Federation?
- deleted 6y ago[deleted]