4 ms·
Good stuff, although I’m not quite sure about the fast OLAP use case. If you’re already sharding by tenant for other reasons, OK… But I see CDC to a true OLAP
by drchaim 4mo ago
Good stuff, although I’m not quite sure about the fast OLAP use case.
If you’re already sharding by tenant for other reasons, OK…
But I see CDC to a true OLAP system as more scalable.
PostgreSQL still needs real columnar tables in the core, hopefully one day
- levkk 4mo agoOLAP means different things to different people. For us, it's just making sure your admin dashboard keeps working basically: SELECT tenant_id, COUNT(clicks) FROM users GROUP BY tenant_id ORDER BY 2 DESC LIMIT 25; Performance is a side effect - definitely needed and we'll do everything we can, but we are not competing with ClickHouse or Snowflake - just trying to make sharded Postgres work with your app.
- christoff12 4mo agoRe OLAP: It's probably ~good enough~ for a lean team that's trying to keep the tech stack standard and/or doesn't have a dedicated data person to take advantage of a columnar store.
- vira28 4mo agoTomas Vondra, a major Postgres contributor recently revived a thread on using Bloom filters - https://www.postgresql.org/message-id/flat/5cd8c20c-14b5-4b0d-bedc-69bf714e87eb%40vondra.me https://www.postgresql.org/message-id/flat/5cd8c20c-14b5-4b0... So there is more core work happening on support OLAP but I do think it will take some time. In the meantime, I think we have all the pieces (storage, query engine, table format) to set up a true OLAP. For instance, I created https://github.com/viggy28/streambed https://github.com/viggy28/streambed to pressure test this idea.