4 ms·
I'd recommend OpenTSDB. Using a 11 node hadoop cluster on m1.xlarge nodes in Amazon, (2 name, 9 data), I can ingest a sustained rate of ~75,000 time series data
by oppositelock 12y ago
I'd recommend OpenTSDB. Using a 11 node hadoop cluster on m1.xlarge nodes in Amazon, (2 name, 9 data), I can ingest a sustained rate of ~75,000 time series datapoints per second in an HBase table.
The upside is that OpenTSDB scales really well with hadoop cluster size, so you can just scale it up to handle more load.
The downsides are that their data schema and query format are optimized for data efficiency, not speed or flexibility. It's really easy to refine a search for a particular metric by filtering on tags, but it's really hard to do any sort of analysis across metrics, so you have to write your own glue on top of that which fetches the datapoints for the metrics you care about, and does its own aggregation.
- linuxhansl 12y agoThat's what we're doing. Uses HBase underneath, scales well.
- bra-ket 12y agothis, or Hbase + Phoenix: http://phoenix.apache.org/ http://phoenix.apache.org/