3 ms·

How is this different from Apache flink real time SQL support? https://flink.apache.org/2020/07/28/flink-sql-demo-building-e2e-streaming-application.html https

• deleted 6y ago
  [deleted]
• frankmcsherry 6y ago
  I think the main distinction is around "interactivity" and how long it takes from typing a query to getting results out. Once you stand up a Flink dataflow, it should move along a brisk clip. But standing up a new dataflow is relatively heavy-weight for them; typically you have to re-flow all of the data that feeds the query. Materialize has a different architecture that allows more state-sharing between operators, and allows many queries to spin up in milliseconds. Primarily, this is when your query depends on existing relational data in pre-indexed form (e.g. joining by primary and foreign keys). You can read a bit more in an overview blog post [0] and in more detail in a VLDB paper [1]. I'm sure there are a number of other quirks distinguishing Flink and Materialize, probably several in their favor, but this is the high-order bit for me. [0]: https://materialize.com/materialize-under-the-hood/ https://materialize.com/materialize-under-the-hood/ [1]: http://www.vldb.org/pvldb/vol13/p1793-mcsherry.pdf http://www.vldb.org/pvldb/vol13/p1793-mcsherry.pdf