4 ms·

Looks like a good concept, I've wondered why there isn't better time series databases that are open source

dlsspy 14y ago
I think there are some good ones. I wrote this because I could get it running faster than I could get the data I've got adapted to existing ones. That doesn't mean they're bad as much as it means I don't understand the data I've got. :) The way I like to think about document-oriented databases is that you store what you have when you have it, and worry about what it was later when you need to get things back out of it. e.g. the big bag of stuff I mentioned in the blog post contains a few things I know I don't need, a few things I think I probably need, and a lot of stuff I just don't want to think about (I might need it later, maybe after some manipulation, etc...). Lob it all in. The downside of a system like seriesly vs. a system like rrd (or any modern equivalent) is the same as the downside of any nosql database vs. a sql database. By planning up front, I can keep the size down and get more performance by incrementally computing stuff from the beginning. In the meantime, I'll just buy more disk. :) I'm reasonably happy with the performance, though. There's a good number of visitors on the page right now and this is what they're seeing: 2012/09/11 12:28:50 Completed query processing in 82.54ms, 6,266 keys, 1,280 chunks That means that for that query, it scanned through 6,266 keys in the on-disk b-tree, grouped them into 1,280 separate result "rows" to be reduced and did the necessary computation to emit all of them in under a tenth of a second while lots of other queries were in flight. My "extreme" cases right now are taking under 3 seconds on over half a million keys. I consider that acceptable for two weeks of side-project.
- rz2k 14y ago
  What are some of the others? The last time I looked, MonetDB and LucidDB seemed to be the most popular column-store open source projects, but they seem to have been mostly subsumed by proprietary products.
  - dlsspy 14y ago
    Whisper (backend for graphite), cube, ganglia... possibly more. I used to build things directly on top of rrdtool, but the schema definition can be a pain when you've got a lot of dynamic data.
dude_abides 14y ago
I don't know about a database but I use the zoo and xts packages in R quite heavily and they are pretty good.
cnlwsu 14y ago
theres http://opentsdb.net/ http://opentsdb.net/, most the time I think people just write a custom thing on top of cassandra or equivalent though.
ajcronk 14y ago
We make a time series database service: http://tempo-db.com http://tempo-db.com Although not open source, many people who've struggled with open source tools end up using us.
- benbjohnson 14y ago
  TempoDB is a great service. Andrew and his team are some smart guys.
  - justindelay 14y ago
    Thanks Ben!

I think there are some good ones. I wrote this because I could get it running faster than I could get the data I've got adapted to existing ones. That doesn't mean they're bad as much as it means I don't understand the data I've got. :) The way I like to think about document-oriented databases is that you store what you have when you have it, and worry about what it was later when you need to get things back out of it. e.g. the big bag of stuff I mentioned in the blog post contains a few things I know I don't need, a few things I think I probably need, and a lot of stuff I just don't want to think about (I might need it later, maybe after some manipulation, etc...). Lob it all in. The downside of a system like seriesly vs. a system like rrd (or any modern equivalent) is the same as the downside of any nosql database vs. a sql database. By planning up front, I can keep the size down and get more performance by incrementally computing stuff from the beginning. In the meantime, I'll just buy more disk. :) I'm reasonably happy with the performance, though. There's a good number of visitors on the page right now and this is what they're seeing: 2012/09/11 12:28:50 Completed query processing in 82.54ms, 6,266 keys, 1,280 chunks That means that for that query, it scanned through 6,266 keys in the on-disk b-tree, grouped them into 1,280 separate result "rows" to be reduced and did the necessary computation to emit all of them in under a tenth of a second while lots of other queries were in flight. My "extreme" cases right now are taking under 3 seconds on over half a million keys. I consider that acceptable for two weeks of side-project.

rz2k 14y ago
What are some of the others? The last time I looked, MonetDB and LucidDB seemed to be the most popular column-store open source projects, but they seem to have been mostly subsumed by proprietary products.
- dlsspy 14y ago
  Whisper (backend for graphite), cube, ganglia... possibly more. I used to build things directly on top of rrdtool, but the schema definition can be a pain when you've got a lot of dynamic data.

dude_abides 14y ago

I don't know about a database but I use the zoo and xts packages in R quite heavily and they are pretty good.

cnlwsu 14y ago

theres http://opentsdb.net/ http://opentsdb.net/, most the time I think people just write a custom thing on top of cassandra or equivalent though.

ajcronk 14y ago

We make a time series database service: http://tempo-db.com http://tempo-db.com Although not open source, many people who've struggled with open source tools end up using us.

benbjohnson 14y ago
TempoDB is a great service. Andrew and his team are some smart guys.
- justindelay 14y ago
  Thanks Ben!