16 ms·

I'm very towards Postgres and resultingly Timescale for being one of it's biggest shining stars. Combine that with a cynical view that this is essentially S-tie

I'm very towards Postgres and resultingly Timescale for being one of it's biggest shining stars. Combine that with a cynical view that this is essentially S-tier content marketing (in the tech world), and the numbers are still bonkers. Might as well add on a bit here -- if you're into this sort of thing you might enjoy Timescale thrashing other purpose-built databases (which have since also improved so YMMV): - Timescale vs Influx[0] - Timescale vs Mongo[1] - Timescale vs Cassandra[2] These blog posts are all from the timescale side, but the numbers are undeniable and their methodology is open source -- not sure there's more you can ask. Cassandra probably scales better at FB/Twitter scale, but with the recent work done to improve Timescale's scaling features[3] and the permissive license (unless you're an Amazon) I'm not sure even that is a real benefit. I'd love to see a response from Amazon -- if HN is so lucky to have a dev who worked on AWS Timestream that would be awesome. [0]: https://blog.timescale.com/blog/timescaledb-vs-influxdb-for-time-series-data-timescale-influx-sql-nosql-36489299877/ https://blog.timescale.com/blog/timescaledb-vs-influxdb-for-... [1]: https://blog.timescale.com/blog/how-to-store-time-series-data-mongodb-vs-timescaledb-postgresql-a73939734016/ https://blog.timescale.com/blog/how-to-store-time-series-dat... [2]: https://blog.timescale.com/blog/time-series-data-cassandra-vs-timescaledb-postgresql-7c2cc50a89ce/ https://blog.timescale.com/blog/time-series-data-cassandra-v... [3]: https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node-petabyte-scale-completely-free-relational-database-for-time-series/ https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node...

ryanbooz 6y ago
We've made every effort to be open about all of the testing we do and many people outside of Timescale contribute to TSBS. As an aside, we tried to solicit help/feedback on both Twitter and Redit groups. One AWS employee reached out and offered to pass it to the team (I was upfront about what we were doing) - but we never heard back.
ants_a 6y ago
I did some time series data benchmarking recently. Most large data is "time series" data, but I will not digress on terminology right now. For the usecase I was looking at my results for InfluxDB did not qualitatively disagree the above blogpost. Timescale got better compression efficiency, faster query results, more constrained memory usage, but lower ingest speed at high concurrencies blocked by WAL insert locking and single threaded compression. If InfluxDBs measurement oriented data model and query interface is a good fit for your use case, and you don't have high cardinality data, then it might be convenient to use it, but it's a terrible choice for anything outside its niche. However none of the databases tested above is anywhere close to the efficiency of a column store database that can do vectorized execution over batches of rows. ClickHouse is a good example of one such database. For queries that have to shift through large amounts of data, either filtering or aggregating it, the performance difference is easily >10x. I was seeing aggregation performance above 2B rows/s and that was I/O throughput bound.
- hardwaresofton 6y ago
  I know the first thing that shocked me was how they could get so close to something that was purpose built for time series. Even being within spitting distance is really great in my opinion for an off the shelf, general tool. > However none of the databases tested above is anywhere close to the efficiency of a column store database that can do vectorized execution over batches of rows. ClickHouse is a good example of one such database. For queries that have to shift through large amounts of data, either filtering or aggregating it, the performance difference is easily >10x. I was seeing aggregation performance above 2B rows/s and that was I/O throughput bound. Agreed -- OLTP (in the case of Timescale) and purpose built timeseries-focused (but not necessarily analytics focused) DBs hold nothing to a proper OLAP database. Did you write about this anywhere? would love to read it. I've never had a real need for the kind of stuff that Clickhouse does, but it looks to be the best in class for F/OSS OLAP DBs. Have you ever tried Druid?
  - arminiusreturns 6y ago
    I'm out of my depth here (not a db guy but curious), so forgive the interruption, but have either of you messed with either SciDB or VoltDB? I have found them to be quite interesting especially given Stonebraker himself being involved in the design. Druid and timescale have also been on my radar too, and I think these comments have finally given me the push to play around with them in my sandbox.
    - hardwaresofton 6y ago
      I've never even heard of SciDB, thanks for the pointer. I have heard of VoltDB and the only thing I remembered was that it was in-memory... I'm still not 100% sure what SciDB is for but it looks like none of the databases we're mentioning are directly comparable: - SciDB for scientific computing (??) - VoltDB for in memory + SQL - Druid for OLAP queries - TimescaleDB for row-based timeseries storage Seems like all apples to oranges to me. I personally like TimescaleDB because it runs on Postgres and I can usually find a way to do all those other things in postgres relatively efficiently (memory requires some squinting while using UNLOGGED tables).
pnathan 6y ago
I'm also super curious about AWS's response. Because these numbers - and the aspect of backups - mean that, barring Timescale just not doing the benchmarking right, Timestream isn't going to be an option.
valyala 6y ago
See also VictoriaMetrics vs TimescaleDB vs InfluxDB benchmarks: - https://valyala.medium.com/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae https://valyala.medium.com/measuring-vertical-scalability-fo... - https://valyala.medium.com/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4 https://valyala.medium.com/when-size-matters-benchmarking-vi... - https://valyala.medium.com/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b https://valyala.medium.com/high-cardinality-tsdb-benchmarks-...
- hagen1778 6y ago
  I think those benchmarks vs TimescaleDB require an update - a lot of things changed since the last test. However, even then VM ingestion rate was 11 million/s on 32CPU instance and this topic says TimescaleDB ingestion rate is 3.1 million/s.

ryanbooz 6y ago

We've made every effort to be open about all of the testing we do and many people outside of Timescale contribute to TSBS. As an aside, we tried to solicit help/feedback on both Twitter and Redit groups. One AWS employee reached out and offered to pass it to the team (I was upfront about what we were doing) - but we never heard back.

ants_a 6y ago

I did some time series data benchmarking recently. Most large data is "time series" data, but I will not digress on terminology right now. For the usecase I was looking at my results for InfluxDB did not qualitatively disagree the above blogpost. Timescale got better compression efficiency, faster query results, more constrained memory usage, but lower ingest speed at high concurrencies blocked by WAL insert locking and single threaded compression. If InfluxDBs measurement oriented data model and query interface is a good fit for your use case, and you don't have high cardinality data, then it might be convenient to use it, but it's a terrible choice for anything outside its niche. However none of the databases tested above is anywhere close to the efficiency of a column store database that can do vectorized execution over batches of rows. ClickHouse is a good example of one such database. For queries that have to shift through large amounts of data, either filtering or aggregating it, the performance difference is easily >10x. I was seeing aggregation performance above 2B rows/s and that was I/O throughput bound.

hardwaresofton 6y ago
I know the first thing that shocked me was how they could get so close to something that was purpose built for time series. Even being within spitting distance is really great in my opinion for an off the shelf, general tool. > However none of the databases tested above is anywhere close to the efficiency of a column store database that can do vectorized execution over batches of rows. ClickHouse is a good example of one such database. For queries that have to shift through large amounts of data, either filtering or aggregating it, the performance difference is easily >10x. I was seeing aggregation performance above 2B rows/s and that was I/O throughput bound. Agreed -- OLTP (in the case of Timescale) and purpose built timeseries-focused (but not necessarily analytics focused) DBs hold nothing to a proper OLAP database. Did you write about this anywhere? would love to read it. I've never had a real need for the kind of stuff that Clickhouse does, but it looks to be the best in class for F/OSS OLAP DBs. Have you ever tried Druid?
- arminiusreturns 6y ago
  I'm out of my depth here (not a db guy but curious), so forgive the interruption, but have either of you messed with either SciDB or VoltDB? I have found them to be quite interesting especially given Stonebraker himself being involved in the design. Druid and timescale have also been on my radar too, and I think these comments have finally given me the push to play around with them in my sandbox.
  - hardwaresofton 6y ago
    I've never even heard of SciDB, thanks for the pointer. I have heard of VoltDB and the only thing I remembered was that it was in-memory... I'm still not 100% sure what SciDB is for but it looks like none of the databases we're mentioning are directly comparable: - SciDB for scientific computing (??) - VoltDB for in memory + SQL - Druid for OLAP queries - TimescaleDB for row-based timeseries storage Seems like all apples to oranges to me. I personally like TimescaleDB because it runs on Postgres and I can usually find a way to do all those other things in postgres relatively efficiently (memory requires some squinting while using UNLOGGED tables).

pnathan 6y ago

I'm also super curious about AWS's response. Because these numbers - and the aspect of backups - mean that, barring Timescale just not doing the benchmarking right, Timestream isn't going to be an option.

valyala 6y ago

See also VictoriaMetrics vs TimescaleDB vs InfluxDB benchmarks: - https://valyala.medium.com/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae https://valyala.medium.com/measuring-vertical-scalability-fo... - https://valyala.medium.com/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4 https://valyala.medium.com/when-size-matters-benchmarking-vi... - https://valyala.medium.com/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b https://valyala.medium.com/high-cardinality-tsdb-benchmarks-...

hagen1778 6y ago
I think those benchmarks vs TimescaleDB require an update - a lot of things changed since the last test. However, even then VM ingestion rate was 11 million/s on 32CPU instance and this topic says TimescaleDB ingestion rate is 3.1 million/s.