4 ms·

In the streaming SQL space, I feel like at this point it's essential to start off with saying what you bring to the table compared with Materialize. No matter

by posnet 3y ago

In the streaming SQL space, I feel like at this point it's essential to start off with saying what you bring to the table compared with Materialize. No matter what you think about how the Materialize, the company, has executed, differential dataflow should still be the standard that all streaming SQl hold themselves to. If you can't do recursive CTE's streaming, then you are worse than Materialize and you need to show that you add something above and beyond. Note, I don't work for, or have any affiliation with the company, I just enjoy and wish more people have read, 'Scalability! But at what COST?'.

ramraj07 3y ago
I was hyped about materialize in the beginning but now looks like they’re just cloud focused and it’s unclear who is supposed to be their customer at all. Also I swear I saw their original docs said they can do window functions but not anymore.
- higeorge13 3y ago
  https://materialize.com/docs/transform-data/patterns/window-functions/ https://materialize.com/docs/transform-data/patterns/window-... Are you sure they don’t?
ergl 3y ago
If you click "Get access" on their main page it says: > It seamlessly integrates with your existing database, saving you from complicated setup procedures. So maybe their advantage over Materialize is that this can work on top of whatever database you already have?
amath 3y ago
That paper is too good not to link - https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://www.usenix.org/system/files/conference/hotos15/hotos... I hope more companies can be built around timely-dataflow.
- necubi 3y ago
  And yet today Materialize is distributed ;)
maor10 3y ago
Hi, author here :) I think there are two critical differences, one on the tech side and one on the UX side- On the UX side, we're seated behind your database, and our goal is for you to only ever interact with your original database, and never have to query us directly. This has a lot of implications when it comes to migrations on your database (you call epsio.create_view in your original DB which is transaction safe), and not having the backend interact with two databases. On the tech side, we're built from ground zero to be above storage, whereas differential dataflow (the package Materialize is built upon) is strongly built to be above memory (we actually played with the idea of using differential with spillover to disk- was too slow). This means we actually lose when it comes to speed (e.g. we may take ~50 milliseconds more which can be a big deal in realtime applications), but have huge gains in cost, since we're trading compute for storage. Check out our FAQ- we have a nice little table for comparison :)

by posnet 3y ago

ramraj07 3y ago

I was hyped about materialize in the beginning but now looks like they’re just cloud focused and it’s unclear who is supposed to be their customer at all. Also I swear I saw their original docs said they can do window functions but not anymore.

higeorge13 3y ago
https://materialize.com/docs/transform-data/patterns/window-functions/ https://materialize.com/docs/transform-data/patterns/window-... Are you sure they don’t?

ergl 3y ago

If you click "Get access" on their main page it says: > It seamlessly integrates with your existing database, saving you from complicated setup procedures. So maybe their advantage over Materialize is that this can work on top of whatever database you already have?

amath 3y ago

That paper is too good not to link - https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://www.usenix.org/system/files/conference/hotos15/hotos... I hope more companies can be built around timely-dataflow.

necubi 3y ago
And yet today Materialize is distributed ;)

maor10 3y ago

Hi, author here :) I think there are two critical differences, one on the tech side and one on the UX side- On the UX side, we're seated behind your database, and our goal is for you to only ever interact with your original database, and never have to query us directly. This has a lot of implications when it comes to migrations on your database (you call epsio.create_view in your original DB which is transaction safe), and not having the backend interact with two databases. On the tech side, we're built from ground zero to be above storage, whereas differential dataflow (the package Materialize is built upon) is strongly built to be above memory (we actually played with the idea of using differential with spillover to disk- was too slow). This means we actually lose when it comes to speed (e.g. we may take ~50 milliseconds more which can be a big deal in realtime applications), but have huge gains in cost, since we're trading compute for storage. Check out our FAQ- we have a nice little table for comparison :)