2 ms·
One thing that I think would be awesome is if there was a Postgres extension for streaming queries, where you could join tables with millions of rows and when a
by madjam002 3y ago
One thing that I think would be awesome is if there was a Postgres extension for streaming queries, where you could join tables with millions of rows and when any of the upstream data is changed you’ll be notified of the delta changes.
I tried Materialize but it seemed incredibly wasteful to duplicate terabytes of data and was also extremely slow compared to the same query running in Postgres.
For now I’ve rolled my own using Debezium CDC and simply reevaluating entire queries with joins when any data changes, but having something more smart would be ideal.
- tedheath123 3y agoRethinkDB attempts to do something like this, but I agree, an efficient SQL version of this would be nice.
- chuckhend 3y agoCheck out what the folks at https://docs.epsio.io/ https://docs.epsio.io/ built. I'm not aware of any other projects doing this, but I think it's a great idea.
- madjam002 3y agoThis looks pretty cool, thanks for the pointer! It would be nice if there was a way to create a stream using this where the materialized data doesn't actually get stored on disk in a materialized table, instead it just streams changes to e.g kafka.