4 ms·

Hoping for more easy columnar support in databases, which is one of the things that can lead you to storing json in database columns (if your data is truly colu

Hoping for more easy columnar support in databases, which is one of the things that can lead you to storing json in database columns (if your data is truly columnar). Currently the vendor lock-in or requirements for installing plugins make it hard to do with cloud sql providers. Especially hard since by the time it's a problem you're probably at enough scale to make switching db/vendors hard or impossible.

hobs 1y ago
How does columnar = json? json isn't colunar at all... If you just want to have a schema in json instead of sql, use a no-sql db, postgres nosql features are strong, but the db features are actually much stronger.
- NegativeLatency 1y ago
  json isn't necessarily columnar, but it is a natrual fit for stuff that is columnar that's otherwise harder to model in a traditional relational db here's my usecase: - we have a bunch of attributes (all different names by customer, and many different values for each record that a customer stores) - it's a fairly natural fit for a json value with only one level of key: value mapping - we use mysql on GCP (no columnar plugins, too hard to switch to postgres) Someone could go back in time and correctly model it as columns and not json but that ship has sorta sailed. While it's not impossible to change, it would be pretty hard, time will tell if that ends up happening. I would love to be able to tell mysql "put this column in a collumnar engine and use that when I query on it" (AlloyDB is this for postgres on GCP)
  - hobs 1y ago
    Too bad mysql doesn't even have materialized views - its not a good combination with a json only situation, but you clearly are suffering through that, good luck :) probably etl it to clickhouse and move on. Could always try and dump to GCP/Parquet/json and use duckdb/bq to just query it all.
moonikakiss 1y ago
great point. with pg_mooncake v0.2 (launching in ~couple weeks), you'll be able to get a columnar copy of your Postgres that's always synced (<s freshness). Keep your write path unchanged, and keep your Postgres where it is. Deploy Mooncake as a replica for the columnar queries.
- NegativeLatency 1y ago
  I wish, someone unfortunately picked mysql

hobs 1y ago

How does columnar = json? json isn't colunar at all... If you just want to have a schema in json instead of sql, use a no-sql db, postgres nosql features are strong, but the db features are actually much stronger.

NegativeLatency 1y ago
json isn't necessarily columnar, but it is a natrual fit for stuff that is columnar that's otherwise harder to model in a traditional relational db here's my usecase: - we have a bunch of attributes (all different names by customer, and many different values for each record that a customer stores) - it's a fairly natural fit for a json value with only one level of key: value mapping - we use mysql on GCP (no columnar plugins, too hard to switch to postgres) Someone could go back in time and correctly model it as columns and not json but that ship has sorta sailed. While it's not impossible to change, it would be pretty hard, time will tell if that ends up happening. I would love to be able to tell mysql "put this column in a collumnar engine and use that when I query on it" (AlloyDB is this for postgres on GCP)
- hobs 1y ago
  Too bad mysql doesn't even have materialized views - its not a good combination with a json only situation, but you clearly are suffering through that, good luck :) probably etl it to clickhouse and move on. Could always try and dump to GCP/Parquet/json and use duckdb/bq to just query it all.

moonikakiss 1y ago

great point. with pg_mooncake v0.2 (launching in ~couple weeks), you'll be able to get a columnar copy of your Postgres that's always synced (<s freshness). Keep your write path unchanged, and keep your Postgres where it is. Deploy Mooncake as a replica for the columnar queries.

NegativeLatency 1y ago
I wish, someone unfortunately picked mysql