4 ms·

Prometheus InfluxDB opentsdb ... and on and on It seems it's pretty crowded

• willempienaar 11y ago
  It's crowded because there isn't a clear leader.
  • bbrazil 11y ago
    Speaking as a Prometheus developer, each is suited to different things. For example if you want to store timeseries data long term and already use HBase then OpenTSDB is a good choice. On the other hand if you want to do monitoring that's simple and dependable in an emergency with querying, graphing and alerting over short/medium-term data, then Prometheus would a good choice.
    • pmahoney 11y ago
      > short/medium-term data This in particular, but your entire post, would be a great addition to the Prometheus front page, or top of the documentation section. This wasn't clear to me initially, speaking as someone who evaluated prometheus a few months ago.
      • bbrazil 11y ago
        We plan on seamlessly integrating with long-term storage (https://prometheus.io/docs/introduction/roadmap/#long-term-storage https://prometheus.io/docs/introduction/roadmap/#long-term-s...) and OpenTSDB is one option for that for us. Ignoring that as it's planned work, the typical considerations are more around availability vs. consistency and that we're a metrics system focused on operations rather than a event store. Most of that's already covered in our docs. See https://prometheus.io/docs/introduction/overview/#when-does-it-fit https://prometheus.io/docs/introduction/overview/#when-does-... and https://prometheus.io/docs/introduction/faq/ https://prometheus.io/docs/introduction/faq/
• infinotize 11y ago
  I wouldn't include opentsdb under "lightweight" as it requires HBase.
  • Cerium 11y ago
    Last time I checked opentsdb only supported millisecond resolution timestamps. That's not enough precision for scientific data. I'm using influxdb because of the nanosecond timestamps, though microseconds would be enough.
• manigandham 11y ago
  Just submitted this: https://news.ycombinator.com/item?id=11654402 https://news.ycombinator.com/item?id=11654402 > It turns out you can accomplish quite a lot with 4,709 lines of Go code! How about a full time-series database implementation, robust enough to be run in production for a year where it stored 2.1 trillion data points, and supporting 119M queries per second (53M inserts per second) in a four-node cluster? Statistical queries over the data complete in 100-250 ms while summarizing up to 4 billion points. It’s pretty space-efficient too, with a 2.9x compression ratio. At the heart of these impressive results, is a data structure supporting a novel abstraction for time-series data: a time partitioning, copy-on-write, version-annotated, k-ary tree.
• salex89 11y ago
  It's not at all crowded since not all of them actually work well and they are optimized for different use cases and different setups. One use case I believe is not covered well, and I use it often, and evaluated it, is actually storing the complete time series and reading it as a whole, without any aggregation. This is important for different signal processing and so on, where sampling does not cut it. At the and I chosen KairosDB (with Cassandra backend). Not that it is perfect, but it if CRUD is most important, and you offload processing to other engines, it works well. That is why I am always looking forward to new things such as this or BTrDB. There is still a lot of room for improvement everywhere.