3 ms·
I worked with a team on this problem once before. It is something definitely solvable, it does take some engineering time though. Also, that team I worked wit
by sixdimensional 6y ago
I worked with a team on this problem once before. It is something definitely solvable, it does take some engineering time though. Also, that team I worked with missed something pretty important - you'll never solve this problem completely when data becomes too big to dynamically join (or even cache) across heterogeneous systems - but it is something I think you could maybe solve differently using event queues and materialized views for specific complex cases.
Have you guys thought about looking at some other tools that might be symbiotic? Hasura's interface would be an incredible companion to something like Dremio (also open source), since it is a fully-SQL engine across a number of heterogeneous systems, with probably the most advanced caching I have ever seen in a system like it (Apache Arrow in-memory and columnar storage). If you can make your query interface adapted to tools like Dremio, you shouldn't have any problem exploiting caching in those kinds of platforms - might be even more efficient than building into Hasura for some use cases.
Edit: I think there is a newer type of interface architecture forming here, and I think GraphQL represents one of the pieces of the puzzle, would be really interested to see more detail on Hasura's take on that.