3 ms·
I think the main distinction is around "interactivity" and how long it takes from typing a query to getting results out. Once you stand up a Flink dataflow, it
by frankmcsherry 6y ago
I think the main distinction is around "interactivity" and how long it takes from typing a query to getting results out. Once you stand up a Flink dataflow, it should move along a brisk clip. But standing up a new dataflow is relatively heavy-weight for them; typically you have to re-flow all of the data that feeds the query.
Materialize has a different architecture that allows more state-sharing between operators, and allows many queries to spin up in milliseconds. Primarily, this is when your query depends on existing relational data in pre-indexed form (e.g. joining by primary and foreign keys).
You can read a bit more in an overview blog post [0] and in more detail in a VLDB paper [1]. I'm sure there are a number of other quirks distinguishing Flink and Materialize, probably several in their favor, but this is the high-order bit for me.
[0]: https://materialize.com/materialize-under-the-hood/ https://materialize.com/materialize-under-the-hood/
[1]: http://www.vldb.org/pvldb/vol13/p1793-mcsherry.pdf http://www.vldb.org/pvldb/vol13/p1793-mcsherry.pdf