4 ms·
Thanks for the detailed comments. Let me try to answer a couple of them. - We don't bypass the Postgres I/O in general, but have rewritten a lot of the insert
by mfreed 9y ago
Thanks for the detailed comments. Let me try to answer a couple of them.
- We don't bypass the Postgres I/O in general, but have rewritten a lot of the insert path as it relates to managing inserts across many chunks (child tables in Postgres). You can see a detailed explanation and performance comparison of PG10 declarative partitioning vs. TimescaleDB here: https://blog.timescale.com/time-series-data-postgresql-10-vs-timescaledb-816ee808bac5 https://blog.timescale.com/time-series-data-postgresql-10-vs...
- What we find is that in many sensor settings, at least when deployed at centralized collection points, you'll actually place your DB/storage infrastructure behind a front-end anyway (web interface, MQTT, SSL termination point, etc.), and then batch data through a data pipeline like Kafka. This means that when the data hits TimescaleDB, you can perform larger batch inserts (even 10K rows / batch), which is how a single-node Timescale can support 100,000s of rows/sec (millions of metrics/sec). More benchmarks: https://blog.timescale.com/timescaledb-vs-6a696248104e https://blog.timescale.com/timescaledb-vs-6a696248104e
- So while this isn't the native I/O speed of Linux, it's not just about raw write rate; you also get those other things that are critical for a robust data product: reliability, flexible and optimized query performance, etc.
- Scaling out OLTP workloads has in general been challenging, but there are some nice characteristics of time-series workloads that make this better. We're working on scale-out clustering right now, and hope to be able to announce some things later this year.
- We fully integrate with PostGIS and other Postgres extensions. Some of our users customers are very much employing TimescaleDB + PostGIS for temporal+spatial analysis: doing things like asset tracking or logistics over time.