4 ms·
What I would really love is a dead simple way to: 1) connect to my transactional Postgres db 2) define my materialized views 3) have these views update in realt
by cocoflunchy 2y ago
What I would really love is a dead simple way to:
1) connect to my transactional Postgres db
2) define my materialized views
3) have these views update in realtime
4) query these views with a fast engine
And ideally have the whole thing open source and be able to run it in CI
We tried peerdb + clickhouse but Clickhouse materialized views are not refreshed when joining tables.
Right now we’re back to standard materialized views inside Postgres refreshed once a day but the full refreshes are pretty slow… the operational side is great though, a single db to manage.
- benpacker 2y agoYou can do this with Materialize or Feldera. The keyword to look for is “incremental view maintenance”
- capkutay 2y agothat's been supported in striim since 2016 https://dl.acm.org/doi/10.1145/3129292.3129294 https://dl.acm.org/doi/10.1145/3129292.3129294
- lsuresh 2y agoYeah you definitely need something like Feldera: https://github.com/feldera/feldera https://github.com/feldera/feldera
- deleted 2y ago[deleted]
- make_it_sure 2y agowe have the exact same problem. We tried Clickhouse but their materialized view limitations stopped us.