10 ms·

Whenever I see these posts from TimescaleDB, I always want to ask them how it compares in performance to alternative extensions that implement the same features

Whenever I see these posts from TimescaleDB, I always want to ask them how it compares in performance to alternative extensions that implement the same features, rather than just comparing TimescaleDB to vanilla PostgreSQL. For example, they mention their automated data retention and how it's achieved with one SQL command, and how DELETEing records is a very costly operation, and how "even if you were using Postgres declarative partitioning you’d still need to automate the process yourself, wasting precious developer time, adding additional requirements, and implementing bespoke code that needs to be supported moving forward". There's zero mention anywhere of pg_partman, which does all of these things for you equally as simply, and is a fully OSS free alternative [0]. I get that it's a PG extension that competes with their product. I know that TimescaleDB does a few other things that pg_partman does not. But I can't help but find its (seemingly) purposeful omission in these, otherwise very thorough blog posts, misleading. [0] https://github.com/pgpartman/pg_partman/blob/master/doc/pg_partman.md#retention https://github.com/pgpartman/pg_partman/blob/master/doc/pg_p...

fzliu 4y ago
Seconded. I understand that Timescale extends Postgres, but I'd still like to see performance comparisons relative to other time-series databases such as InfluxDB or even TiDB. Shameless self-plug: we're also building database technology, but for embedding vectors (https://milvus.io https://milvus.io) rather than time-series data - when doing query performance comparisons, we get several orders of magnitude of performance improvement over traditional databases. It's an unfair comparison, so we generally avoid it.
- ebiester 4y ago
  I think it's fair to mention it, but with the specific caveat of "If your workload has these characteristics, we are a particularly good fit."
- akulkarni 4y ago
  [Timescale co-founder] We have published many, many benchmarks versus other database systems. IIRC all of them also made the front page of HackerNews. Here are some of them, for your reading pleasure :-) TimescaleDB vs. InfluxDB: https://www.timescale.com/blog/timescaledb-vs-influxdb-for-time-series-data-timescale-influx-sql-nosql-36489299877/ https://www.timescale.com/blog/timescaledb-vs-influxdb-for-t... TimescaleDB vs. ClickHouse: https://www.timescale.com/blog/what-is-clickhouse-how-does-it-compare-to-postgresql-and-timescaledb-and-how-does-it-perform-for-time-series-data/ https://www.timescale.com/blog/what-is-clickhouse-how-does-i... TimescaleDB vs. Timestream: https://www.timescale.com/blog/timescaledb-vs-amazon-timestream-6000x-higher-inserts-175x-faster-queries-220x-cheaper/ https://www.timescale.com/blog/timescaledb-vs-amazon-timestr...
  - djbusby 4y ago
    Cool. Now do VS PG+extensions like asked about up-thread.
- mekster 4y ago
  Once I've read this, I never had to think twice on what to use. https://valyala.medium.com/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b https://valyala.medium.com/high-cardinality-tsdb-benchmarks-...
brightball 4y ago
Same. I've been experimenting with the recently fully open sourced Citus 11 and it's pretty incredible. To the point that I have a hard time imagining NOT using it going forward. Maybe I need to do my own comparison at some point.
akulkarni 4y ago
(Timescale co-founder) That's a fair question. We find that most developers storing time-series data on Postgres are doing so without pg_partman. So we first wanted to provide a benchmark that would be useful to most developers. This benchmark was also the result of months of dedicated work. So the team did spend a lot of time on this. Unfortunately, they ran out of time to cover pg_partman. But that comparison is on our list of things to do soon. Thanks!
- avar 4y ago
  Most people storing time-series are also doing that outside of TimescaleDB. So the relevant question isn't what a typical PostgreSQL user is doing, but whether someone wanting to optimize their storage should look at a PostgreSQL extension, or an upstart database like TimescaleDB.
  - djk447 4y ago
    NB: I work at Timescale. TimescaleDB is a PostgreSQL extension, just to be clear.
djk447 4y ago
Disclosure, I work at Timescale. Though I didn't write this post, I'd imagine at least part of it is that it's already nearly 4000 words and a 15 minute read and we just didn't want to add another set of things to it, to be perfectly honest. `pg_partman` is cool! I haven't used it in a while, and because it uses declarative partitioning, it has some locking issues that we address with our partitioning scheme, but implying that it is OSS and we're not in terms of things like data retention features is a bit misleading as well. The `drop_chunks` command used for data retention is in the Apache 2 licensed portion of Timescale.
- ensignavenger 4y ago
  But almost all of your posts and benchmarks are based on the closed source version of Timescale. Everywhere I have seen it is always recommend to use the closed source version to get decent performance out of it.
  - akulkarni 4y ago
    (Timescale co-founder) Just to clarify: Nothing on Timescale is closed-source. It is all source available, all on Github. Some of it is Apache2 licensed, some of it is Timescale Licensed. And it is all free.
    - ensignavenger 4y ago
      Since the Timescale License is not an open source license, it is a closed source license. You are right, it is source available, but source available is also closed source. It is closed because it is not open. And it might be free as in beer, but it is not free as in freedom.
      - skeletal88 4y ago
        The source code is available, when someone says something is closed source then it usually means that the source code is not publicly available. Do you want amazing things? Everything can't be "free as in beer" wtf does that even mean, i don't get free beer from anywhere.
bbirk 4y ago
Sometimes when benchmarks are posted on HN that compares your own product (A) with a competitor's product (B), someone working for the competitor refutes the claims made in the blog and states that B were not given the same requirements as A. This particular thread comes to mind [1]. Maybe this is due to the blog writers misunderstanding B or maybe the competitor misunderstands A, or maybe there is a better way to do what was benchmarked using B. I think a better way would be to use some standard benchmark, and publish the code used for benchmarking them. [1] https://news.ycombinator.com/item?id=31767858 https://news.ycombinator.com/item?id=31767858
jamesgresql 4y ago
Hello! I run Developer Advocacy at Timescale. Easy fix - we can do a benchmark comparing TimescaleDB to pg_partman. Longer reply: pg_partman does address many of the same developer experience items we do, but it doesn't offer things like compression or modified plans. It will perform roughly the same as declarative partitioning (because that's what it uses), and I'm guessing we will see results similar to the last large table (in the Declarative Partitioning section).

fzliu 4y ago

Seconded. I understand that Timescale extends Postgres, but I'd still like to see performance comparisons relative to other time-series databases such as InfluxDB or even TiDB. Shameless self-plug: we're also building database technology, but for embedding vectors (https://milvus.io https://milvus.io) rather than time-series data - when doing query performance comparisons, we get several orders of magnitude of performance improvement over traditional databases. It's an unfair comparison, so we generally avoid it.

ebiester 4y ago
I think it's fair to mention it, but with the specific caveat of "If your workload has these characteristics, we are a particularly good fit."
akulkarni 4y ago
[Timescale co-founder] We have published many, many benchmarks versus other database systems. IIRC all of them also made the front page of HackerNews. Here are some of them, for your reading pleasure :-) TimescaleDB vs. InfluxDB: https://www.timescale.com/blog/timescaledb-vs-influxdb-for-time-series-data-timescale-influx-sql-nosql-36489299877/ https://www.timescale.com/blog/timescaledb-vs-influxdb-for-t... TimescaleDB vs. ClickHouse: https://www.timescale.com/blog/what-is-clickhouse-how-does-it-compare-to-postgresql-and-timescaledb-and-how-does-it-perform-for-time-series-data/ https://www.timescale.com/blog/what-is-clickhouse-how-does-i... TimescaleDB vs. Timestream: https://www.timescale.com/blog/timescaledb-vs-amazon-timestream-6000x-higher-inserts-175x-faster-queries-220x-cheaper/ https://www.timescale.com/blog/timescaledb-vs-amazon-timestr...
- djbusby 4y ago
  Cool. Now do VS PG+extensions like asked about up-thread.
mekster 4y ago
Once I've read this, I never had to think twice on what to use. https://valyala.medium.com/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b https://valyala.medium.com/high-cardinality-tsdb-benchmarks-...

brightball 4y ago

Same. I've been experimenting with the recently fully open sourced Citus 11 and it's pretty incredible. To the point that I have a hard time imagining NOT using it going forward. Maybe I need to do my own comparison at some point.

akulkarni 4y ago

(Timescale co-founder) That's a fair question. We find that most developers storing time-series data on Postgres are doing so without pg_partman. So we first wanted to provide a benchmark that would be useful to most developers. This benchmark was also the result of months of dedicated work. So the team did spend a lot of time on this. Unfortunately, they ran out of time to cover pg_partman. But that comparison is on our list of things to do soon. Thanks!

avar 4y ago
Most people storing time-series are also doing that outside of TimescaleDB. So the relevant question isn't what a typical PostgreSQL user is doing, but whether someone wanting to optimize their storage should look at a PostgreSQL extension, or an upstart database like TimescaleDB.
- djk447 4y ago
  NB: I work at Timescale. TimescaleDB is a PostgreSQL extension, just to be clear.

djk447 4y ago

Disclosure, I work at Timescale. Though I didn't write this post, I'd imagine at least part of it is that it's already nearly 4000 words and a 15 minute read and we just didn't want to add another set of things to it, to be perfectly honest. `pg_partman` is cool! I haven't used it in a while, and because it uses declarative partitioning, it has some locking issues that we address with our partitioning scheme, but implying that it is OSS and we're not in terms of things like data retention features is a bit misleading as well. The `drop_chunks` command used for data retention is in the Apache 2 licensed portion of Timescale.

ensignavenger 4y ago
But almost all of your posts and benchmarks are based on the closed source version of Timescale. Everywhere I have seen it is always recommend to use the closed source version to get decent performance out of it.
- akulkarni 4y ago
  (Timescale co-founder) Just to clarify: Nothing on Timescale is closed-source. It is all source available, all on Github. Some of it is Apache2 licensed, some of it is Timescale Licensed. And it is all free.
  - ensignavenger 4y ago
    Since the Timescale License is not an open source license, it is a closed source license. You are right, it is source available, but source available is also closed source. It is closed because it is not open. And it might be free as in beer, but it is not free as in freedom.
    - skeletal88 4y ago
      The source code is available, when someone says something is closed source then it usually means that the source code is not publicly available. Do you want amazing things? Everything can't be "free as in beer" wtf does that even mean, i don't get free beer from anywhere.

bbirk 4y ago

Sometimes when benchmarks are posted on HN that compares your own product (A) with a competitor's product (B), someone working for the competitor refutes the claims made in the blog and states that B were not given the same requirements as A. This particular thread comes to mind [1]. Maybe this is due to the blog writers misunderstanding B or maybe the competitor misunderstands A, or maybe there is a better way to do what was benchmarked using B. I think a better way would be to use some standard benchmark, and publish the code used for benchmarking them. [1] https://news.ycombinator.com/item?id=31767858 https://news.ycombinator.com/item?id=31767858

jamesgresql 4y ago

Hello! I run Developer Advocacy at Timescale. Easy fix - we can do a benchmark comparing TimescaleDB to pg_partman. Longer reply: pg_partman does address many of the same developer experience items we do, but it doesn't offer things like compression or modified plans. It will perform roughly the same as declarative partitioning (because that's what it uses), and I'm guessing we will see results similar to the last large table (in the Declarative Partitioning section).