3 ms·

Are you running the "OLAP" TimescaleDB on the same instance as your regular OLTP Postgres? This is the only reason I would entertain TimescaleDB, if I had a str

Are you running the "OLAP" TimescaleDB on the same instance as your regular OLTP Postgres? This is the only reason I would entertain TimescaleDB, if I had a strict "1 server" requirement. I briefly deployed and looked into it and there were a lot of footguns like with compression. If not, I would suggest looking at a proper OLAP DB. VictoriaMetrics has been great and was easy to set up.

phamilton 3y ago
A reason I would still bias towards postgres is the maturity of managed solutions (including decoupled compute/storage solutions like Aurora and AlloyDB). Are managed "proper OLAP DB" solutions competitive with managed RDBMS from a price and ease of use standpoint?
- PaulWaldman 3y ago
  Using TimescaleDB from a managed provider is limited, unless of course that provider is Timescale. Other managed providers are only permitted to use the TimescaleDB Apache 2 Edition. This link has a comparison of features[1]. [1] https://docs.timescale.com/about/latest/timescaledb-editions/ https://docs.timescale.com/about/latest/timescaledb-editions...
tetha 3y ago
We're much rather looking at reducing the number of technologies we have and exchanging one specialized one-use database for another one doesn't sound great. And sure, TimescaleDB is a hefty extension and will require some work to understand it, but things like HA, backups and overall management of Postgres are pretty much solved for us. And beyond that, TimescaleDB works with a few things we have already. We could migrate Zabbix to use TimescaleDB for a large performance boost. Also 1-2 teams are building reporting solutions for the product platform, and they are generating some significant timeseries data in a Postgres database as well.
- pphysch 3y ago
  That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is designed to achieve great compression while also allowing flexibility of series. Hopefully Timescale will/has fixed that without adding another drastic perf tradeoff, but given how Postgres is architected for OLTP I would be surprised.
  - hnarn 3y ago
    You threw out a database because it didn’t offer compression in your specific use case? That’s it? Just solve compression on the block level, why are you so specific about it happening in the database? It’s probably one of the least interesting feature comparisons when betting on which database to trust.
    - pphysch 3y ago
      Not all compression is created equally. Timeseries data has well-defined characteristics that generic block level compression doesn't understand. It's a great example where application-level compression is worthwhile. The proof is in the pudding.
  - PaulWaldman 3y ago
    >My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. What is the "dynamic labels" feature? Is it a part of Postgres or Timescale?
    - pphysch 3y ago
      It's a Timescale feature that allows you to "just insert" metrics without first doing a schema migration to make sure a possible new label schema is supported. I.e. a feature that is taken for granted in native timeseries databases, which don't have to work around RDBMS schemas. I assume it's doing an automatic ALTER TABLE when necessary, which modifies each row and somehow breaks compression across the sharded tables. Or at least an automatic re-compression would cause massive latency on insert that they wanted to avoid.

phamilton 3y ago

A reason I would still bias towards postgres is the maturity of managed solutions (including decoupled compute/storage solutions like Aurora and AlloyDB). Are managed "proper OLAP DB" solutions competitive with managed RDBMS from a price and ease of use standpoint?

PaulWaldman 3y ago
Using TimescaleDB from a managed provider is limited, unless of course that provider is Timescale. Other managed providers are only permitted to use the TimescaleDB Apache 2 Edition. This link has a comparison of features[1]. [1] https://docs.timescale.com/about/latest/timescaledb-editions/ https://docs.timescale.com/about/latest/timescaledb-editions...

tetha 3y ago

We're much rather looking at reducing the number of technologies we have and exchanging one specialized one-use database for another one doesn't sound great. And sure, TimescaleDB is a hefty extension and will require some work to understand it, but things like HA, backups and overall management of Postgres are pretty much solved for us. And beyond that, TimescaleDB works with a few things we have already. We could migrate Zabbix to use TimescaleDB for a large performance boost. Also 1-2 teams are building reporting solutions for the product platform, and they are generating some significant timeseries data in a Postgres database as well.

pphysch 3y ago
That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is designed to achieve great compression while also allowing flexibility of series. Hopefully Timescale will/has fixed that without adding another drastic perf tradeoff, but given how Postgres is architected for OLTP I would be surprised.
- hnarn 3y ago
  You threw out a database because it didn’t offer compression in your specific use case? That’s it? Just solve compression on the block level, why are you so specific about it happening in the database? It’s probably one of the least interesting feature comparisons when betting on which database to trust.
  - pphysch 3y ago
    Not all compression is created equally. Timeseries data has well-defined characteristics that generic block level compression doesn't understand. It's a great example where application-level compression is worthwhile. The proof is in the pudding.
- PaulWaldman 3y ago
  >My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. What is the "dynamic labels" feature? Is it a part of Postgres or Timescale?
  - pphysch 3y ago
    It's a Timescale feature that allows you to "just insert" metrics without first doing a schema migration to make sure a possible new label schema is supported. I.e. a feature that is taken for granted in native timeseries databases, which don't have to work around RDBMS schemas. I assume it's doing an automatic ALTER TABLE when necessary, which modifies each row and somehow breaks compression across the sharded tables. Or at least an automatic re-compression would cause massive latency on insert that they wanted to avoid.