mfreed

27 ms·

211.
by mfreed 9y ago
Yes, we're working on a distributed version of Timescale as you describe. But two clarifications: 1. It can aggregate better than you might think. We've had people run single-node Timescale with 20+ disks, then couple that with q
212.
by mfreed 9y ago
Yep, it can be used for either "irregular" events or "regular" time-series like monitoring data. For event sourcing, just make sure you index on the proper user/session/thing (Docs or Slack for more info).
213.
by mfreed 9y ago
Write performance is a much simpler metric than query performance, which is HIGHLY dependent on the actual query being performed. Plus, in many time-series settings, you actually need to support high-write rates, which vanilla RDBMS tables
214.
by mfreed 9y ago
To my understanding, Tgres is really more of a "middleware" layer that collects metrics and performs aggregations on them that are stored back into Postgres (e.g., generates aggregate rates for evenly spaced time intervals a la RR
215.
by mfreed 9y ago
We've had users at that scale at least on the real-time side (e.g., 100M/5min, 500B rows), although requires some care. Might be easiest to discuss more on Slack ( https://slack-login.timescale.com/ ) or email (mike
216.
by mfreed 9y ago
We've been talking with Amazon, and you can help the process along: https://github.com/timescale/timescaledb/issues/65
217.
by mfreed 9y ago
Generally speaking, a TimescaleDB database just looks like Postgres to admin tools. So this looks interesting, and we are already internal Kubernetes users. Will have to look at more.
218.
by mfreed 9y ago
The delete performance will probably be similar, but standard partitions in postgres have a bunch of current limitations. For example, the insert pipeline is still quite a bit slower, partition creation is still manual, can't do as goo
219.
by mfreed 9y ago
Not sure about parent's use case, but we've seen scenarios where users want to synchronize data from downstream (say, they are collecting data on a hub in an IoT setting, and even using Timescale both on the hub and in the cloud).
220.
by mfreed 9y ago
And actually, if you want to run the benchmarks yourself: https://github.com/timescale/benchmark-postgres
221.
by mfreed 9y ago
Customer requests help the process with RDS along :) https://github.com/timescale/timescaledb/issues/65
222.
by mfreed 9y ago
In a funny bit of coincidence -- we didn't post this link to HN :) -- we just published a blog post today comparing Timescale vs. native Postgres: https://blog.timescale.com/timescaledb-vs-6a696248104e tl;dr: 20x highe
223.
by mfreed 9y ago
We're talking with them...and you can help: https://github.com/timescale/timescaledb/issues/65
224.
by mfreed 9y ago
Hi Tostino, please let us know how things work. Regarding compression: While we haven't yet built in any native compression, we regularly run on ZFS and typically get 3-4x compression using that. (Plus with ZFS, insert rates are actu
225.
by mfreed 9y ago
Hi Craig, Timescale person here. noir-york is correct. TimescaleDB was always an extension, never a fork. So all installations are just a `CREATE EXTENSION` and upgrades between versions just via `ALTER EXTENSION` commands. But you're
226.
by mfreed 9y ago
There's a link at the bottom of the post for downloading both the dataset and TimescaleDB: https://blog.timescale.com/analyzing-ethereum-bitcoin-and-12... Enjoy!
227.
by mfreed 10y ago
Keith Winstein & collaborators have good work about using ML to train TCP's congestion control in different scenarios: http://web.mit.edu/remy/
228.
by mfreed 10y ago
You can query by arbitrary dimension (what we would think of as time aggregates). In the future, you'd be able to materialize to arbitrary dimensions with continuous queries.
229.
by mfreed 10y ago
Don’t remember enough specifics of VoltDB’s partitioning/parallelization details (or its academic H-Store predecessor), but that wouldn’t surprise me. VoltBD’s focus on all in-memory processing and general OLTP workloads (so-called ”Ne
230.
by mfreed 10y ago
Don't think we're going to disagree with this! And because Timescale is an extension in Postgres, it can actually sit inside (and coexist) with your standard Postgres database if you want. Or, it can be deployed as a separate i
231.
by mfreed 10y ago
Thanks for the great suggestions. We've certainly discussed running on ZFS internally, but haven't evaluated yet. We're a bit uncomfortable dictating file system requirements to users, so looking to ultimately provide many of
232.
by mfreed 10y ago
We've been using this framework to avoid the appearance of crafting our own that cherrypicks only our strong suits. Don't disagree with comments that you should account for JSON serialization cost in your benchmarks, but that&#x2
233.
by mfreed 10y ago
At a high-level, most distributed architectures take a similar approach: Spread partitions over multiple nodes, insert/query to any node, route (sub)requests to the appropriate nodes. The unique parts of Timescale is that, unlike Pos
234.
by mfreed 10y ago
And a clarification just in case this wasn't clear: We use constraint exclusion analysis to determine which of the "child tables" of the parent hypertable to query. So, when performing query parallelization, you aren't
235.
by mfreed 10y ago
And just to follow-on previous comment: Yes, we plan to offer such continuous aggregations, where data of lower granularity can then just be automatically written into a different hypertable. This also plays nicely with configurable data r
236.
by mfreed 10y ago
Not sure if this is what you're asking, but some monitoring focused systems have you associate each individual time-series metric with a bunch of metadata/labels (either encoding into a single string ("dc1.rack15.server46.por
237.
by mfreed 10y ago
Sorry the tone wasn't to your liking. For a more academic voice (and deeper technical dive), you might enjoy our LaTeX-typeset paper instead: http://www.timescale.com/papers/timescaledb.pdf
238.
by mfreed 12y ago
Nope, still running...