7 ms·

We're migrating off of InfluxDB due to that rollercoaster, honestly. It's hard enough to find time to maintain the monitoring stack at work. Casually dropping "

by tetha 3y ago

We're migrating off of InfluxDB due to that rollercoaster, honestly. It's hard enough to find time to maintain the monitoring stack at work. Casually dropping "Oh, and now you get to rebuild the entire grafana to change the query language" on that doesn't help. And apparently, version 3 does the same thing, except backwards. Sorry, but at that point, we've decided to rebuild the entire metric visualization once on TimescaleDB, since we're running postgres a lot anyhow.

pphysch 3y ago
Are you running the "OLAP" TimescaleDB on the same instance as your regular OLTP Postgres? This is the only reason I would entertain TimescaleDB, if I had a strict "1 server" requirement. I briefly deployed and looked into it and there were a lot of footguns like with compression. If not, I would suggest looking at a proper OLAP DB. VictoriaMetrics has been great and was easy to set up.
- phamilton 3y ago
  A reason I would still bias towards postgres is the maturity of managed solutions (including decoupled compute/storage solutions like Aurora and AlloyDB). Are managed "proper OLAP DB" solutions competitive with managed RDBMS from a price and ease of use standpoint?
  - PaulWaldman 3y ago
    Using TimescaleDB from a managed provider is limited, unless of course that provider is Timescale. Other managed providers are only permitted to use the TimescaleDB Apache 2 Edition. This link has a comparison of features[1]. [1] https://docs.timescale.com/about/latest/timescaledb-editions/ https://docs.timescale.com/about/latest/timescaledb-editions...
- tetha 3y ago
  We're much rather looking at reducing the number of technologies we have and exchanging one specialized one-use database for another one doesn't sound great. And sure, TimescaleDB is a hefty extension and will require some work to understand it, but things like HA, backups and overall management of Postgres are pretty much solved for us. And beyond that, TimescaleDB works with a few things we have already. We could migrate Zabbix to use TimescaleDB for a large performance boost. Also 1-2 teams are building reporting solutions for the product platform, and they are generating some significant timeseries data in a Postgres database as well.
  - pphysch 3y ago
    That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is designed to achieve great compression while also allowing flexibility of series. Hopefully Timescale will/has fixed that without adding another drastic perf tradeoff, but given how Postgres is architected for OLTP I would be surprised.
    - hnarn 3y ago
      You threw out a database because it didn’t offer compression in your specific use case? That’s it? Just solve compression on the block level, why are you so specific about it happening in the database? It’s probably one of the least interesting feature comparisons when betting on which database to trust.
      - pphysch 3y ago
        Not all compression is created equally. Timeseries data has well-defined characteristics that generic block level compression doesn't understand. It's a great example where application-level compression is worthwhile. The proof is in the pudding.
    - PaulWaldman 3y ago
      >My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. What is the "dynamic labels" feature? Is it a part of Postgres or Timescale?
      - pphysch 3y ago
        It's a Timescale feature that allows you to "just insert" metrics without first doing a schema migration to make sure a possible new label schema is supported. I.e. a feature that is taken for granted in native timeseries databases, which don't have to work around RDBMS schemas. I assume it's doing an automatic ALTER TABLE when necessary, which modifies each row and somehow breaks compression across the sharded tables. Or at least an automatic re-compression would cause massive latency on insert that they wanted to avoid.
the_duke 3y ago
Fair warning, I had serious scaling issues with Timescale. Solutions like Grafana Mimir, Victoria Metrics, Clickhouse, or yes, the new Influx implementation, are much more scalable and will give you much fewer headaches. ClickhouseDB is realy brilliant, btw, it's a powerhouse. Especially with the fairly recent additions that enable hybrid local + S3 option, pushing older metrics to S3 for cheap long-term storage.
- PaulWaldman 3y ago
  Can you elaborate on what scaling issues you had with Timescale?
  - efxhoy 3y ago
    I ran into issues with TS too. Main issue I recall now was maintaining a grand total count of events that were already rolled up into daily counts was not fast since it always looked back at all the data. There was no way in the ts patterns to efficiently express it without handrolling something. The issue was that a grand total count can’t be expressed in terms of a hypertable because there’s no time column. It’s fantastic for workloads that neatly fit in the hypertable pattern though.
    - lispisok 3y ago
      You tried to use a timeseries database without a time column?
      - efxhoy 3y ago
        > Don't be snarky. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Are you expecting a real answer of how I was hoping timescale's internal watermark system would help me roll up a total count or are you just implying I'm an idiot?
        mardifoufs 3y ago
efxhoy 3y ago
Another alternative to timescale could be hydra. Haven’t tried it myself but the promise of columnar tables seems wildly useful.
x86a 3y ago
We are influxdb enterprise customers and looking to do the same thing. They've kept their enterprise offering on 1.x, which has kept us mostly happy, but seeing what's going on in their OSS stuff is horrifying and we're looking to avoid the crash and burn at the end of the tunnel.
- pauldix 3y ago
  We announced the availability of the v3 successor to Enterprise v1. It supports the v1 API. We're still building data migration tooling, but if you're interested in testing it out just email support or your sales rep.
  - roygbiv2 3y ago
    How does one upgrade from v2 beta to the latest v2? The docs for doing that seem to no longer exist https://github.com/influxdata/influxdb/issues/24393 https://github.com/influxdata/influxdb/issues/24393
    - pauldix 3y ago
      To be honest I'm not sure. Upgrading individual releases on the way should take you there, but the v2 beta was quite a while ago.
kungito 3y ago
Same here. I joined current company 3 years ago when Influx v2 was coming out. I was supposed to build some analytics on top of it. It was very painful. Flux compiler was often giving internal errors, docs were unclear and it was hard to write any a bit more complicated code. The dash is subpar to graphana but graphana had just raw support. There was no query builder for flux so I tried building dashboards in influxv2 but the whole experience was excrutiating. I still have an issue open where they have an internal function incorrectly written in their own flux code and I provided the fix and what was the issue but it was never addressed. Often times I had a feeling that I found bugs in situations that were so basic that it felt like I was the only person on the planet writing Flux code
- mmcnl 3y ago
  I'm running InfluxDB 1 and 2 in parallel for a personal project, waiting for v2 to get mature and stable enough to replace v1. It's never happening I guess. v1 still works great for me.
za3faran 3y ago
What are you migrating to?

by tetha 3y ago

pphysch 3y ago

Are you running the "OLAP" TimescaleDB on the same instance as your regular OLTP Postgres? This is the only reason I would entertain TimescaleDB, if I had a strict "1 server" requirement. I briefly deployed and looked into it and there were a lot of footguns like with compression. If not, I would suggest looking at a proper OLAP DB. VictoriaMetrics has been great and was easy to set up.

phamilton 3y ago
A reason I would still bias towards postgres is the maturity of managed solutions (including decoupled compute/storage solutions like Aurora and AlloyDB). Are managed "proper OLAP DB" solutions competitive with managed RDBMS from a price and ease of use standpoint?
- PaulWaldman 3y ago
  Using TimescaleDB from a managed provider is limited, unless of course that provider is Timescale. Other managed providers are only permitted to use the TimescaleDB Apache 2 Edition. This link has a comparison of features[1]. [1] https://docs.timescale.com/about/latest/timescaledb-editions/ https://docs.timescale.com/about/latest/timescaledb-editions...
tetha 3y ago
We're much rather looking at reducing the number of technologies we have and exchanging one specialized one-use database for another one doesn't sound great. And sure, TimescaleDB is a hefty extension and will require some work to understand it, but things like HA, backups and overall management of Postgres are pretty much solved for us. And beyond that, TimescaleDB works with a few things we have already. We could migrate Zabbix to use TimescaleDB for a large performance boost. Also 1-2 teams are building reporting solutions for the product platform, and they are generating some significant timeseries data in a Postgres database as well.
- pphysch 3y ago
  That's a fair point, but it's worth appreciating the fundamental differences between OLTP RDBMS and OLAP timeseries. I'm not saying deploy N different DBs, I'm saying pick a good OLTP solution (Postgres) and a good OLAP solution. My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. A proper timeseries DB is designed to achieve great compression while also allowing flexibility of series. Hopefully Timescale will/has fixed that without adding another drastic perf tradeoff, but given how Postgres is architected for OLTP I would be surprised.
  - hnarn 3y ago
    You threw out a database because it didn’t offer compression in your specific use case? That’s it? Just solve compression on the block level, why are you so specific about it happening in the database? It’s probably one of the least interesting feature comparisons when betting on which database to trust.
    - pphysch 3y ago
      Not all compression is created equally. Timeseries data has well-defined characteristics that generic block level compression doesn't understand. It's a great example where application-level compression is worthwhile. The proof is in the pudding.
  - PaulWaldman 3y ago
    >My bad experience with TimescaleDB 3 years ago was that enabling compression required disabling the "dynamic labels" feature, which was a total nonstarter for us. What is the "dynamic labels" feature? Is it a part of Postgres or Timescale?
    - pphysch 3y ago
      It's a Timescale feature that allows you to "just insert" metrics without first doing a schema migration to make sure a possible new label schema is supported. I.e. a feature that is taken for granted in native timeseries databases, which don't have to work around RDBMS schemas. I assume it's doing an automatic ALTER TABLE when necessary, which modifies each row and somehow breaks compression across the sharded tables. Or at least an automatic re-compression would cause massive latency on insert that they wanted to avoid.

the_duke 3y ago

Fair warning, I had serious scaling issues with Timescale. Solutions like Grafana Mimir, Victoria Metrics, Clickhouse, or yes, the new Influx implementation, are much more scalable and will give you much fewer headaches. ClickhouseDB is realy brilliant, btw, it's a powerhouse. Especially with the fairly recent additions that enable hybrid local + S3 option, pushing older metrics to S3 for cheap long-term storage.

PaulWaldman 3y ago
Can you elaborate on what scaling issues you had with Timescale?
- efxhoy 3y ago
  I ran into issues with TS too. Main issue I recall now was maintaining a grand total count of events that were already rolled up into daily counts was not fast since it always looked back at all the data. There was no way in the ts patterns to efficiently express it without handrolling something. The issue was that a grand total count can’t be expressed in terms of a hypertable because there’s no time column. It’s fantastic for workloads that neatly fit in the hypertable pattern though.
  - lispisok 3y ago
    You tried to use a timeseries database without a time column?
    - efxhoy 3y ago
      > Don't be snarky. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Are you expecting a real answer of how I was hoping timescale's internal watermark system would help me roll up a total count or are you just implying I'm an idiot?
      - mardifoufs 3y ago

efxhoy 3y ago

Another alternative to timescale could be hydra. Haven’t tried it myself but the promise of columnar tables seems wildly useful.

x86a 3y ago

We are influxdb enterprise customers and looking to do the same thing. They've kept their enterprise offering on 1.x, which has kept us mostly happy, but seeing what's going on in their OSS stuff is horrifying and we're looking to avoid the crash and burn at the end of the tunnel.

pauldix 3y ago
We announced the availability of the v3 successor to Enterprise v1. It supports the v1 API. We're still building data migration tooling, but if you're interested in testing it out just email support or your sales rep.
- roygbiv2 3y ago
  How does one upgrade from v2 beta to the latest v2? The docs for doing that seem to no longer exist https://github.com/influxdata/influxdb/issues/24393 https://github.com/influxdata/influxdb/issues/24393
  - pauldix 3y ago
    To be honest I'm not sure. Upgrading individual releases on the way should take you there, but the v2 beta was quite a while ago.

kungito 3y ago

Same here. I joined current company 3 years ago when Influx v2 was coming out. I was supposed to build some analytics on top of it. It was very painful. Flux compiler was often giving internal errors, docs were unclear and it was hard to write any a bit more complicated code. The dash is subpar to graphana but graphana had just raw support. There was no query builder for flux so I tried building dashboards in influxv2 but the whole experience was excrutiating. I still have an issue open where they have an internal function incorrectly written in their own flux code and I provided the fix and what was the issue but it was never addressed. Often times I had a feeling that I found bugs in situations that were so basic that it felt like I was the only person on the planet writing Flux code

mmcnl 3y ago
I'm running InfluxDB 1 and 2 in parallel for a personal project, waiting for v2 to get mature and stable enough to replace v1. It's never happening I guess. v1 still works great for me.

za3faran 3y ago

What are you migrating to?