4 ms·

What do you use for push-oriented time-series metrics? We are thinking of using LogStash (it has a go client). Unfortunately, as I understand, it's not time-s

What do you use for push-oriented time-series metrics? We are thinking of using LogStash (it has a go client). Unfortunately, as I understand, it's not time-series oriented.

Rapzid 12y ago
I push all my logs into logstash->elasticsearch. For metrics I go Riemann->Graphite. Riemann comes with a graphite compatible server so I push straight to that for processing and alerting. I also send from Riemann to logstash and logstash to Riemann where it makes sense. For my metrics dashboard I use Grafana which is really awesome. I make use of its templating pretty heavily as I run a lot of individual IIS clusters. I can create cluster overview and drill down dashboards and template it so I can just change the cluster number to see its stats. You can also link directly into a dashboard passing variables as a query string parameter. Pretty excellent.
bbrazil 12y ago
https://github.com/rcrowley/go-metrics https://github.com/rcrowley/go-metrics is one I've came across while researching potential Prometheus integrations, it seems to support a number of backends including Graphite and InfluxDB. One challenge with that model of metrics is that it assumes that the monitoring system doesn't have much ability to work with samples, so it makes up for this by calculating rates and quantiles in the client. This isn't as accurate as doing it in the server, and some instrumentation systems don't allow you to extract out the raw data to allow you to do it in the server. For example say you've a 1-minute rate exported/scraped every minute. If you miss a push/scrape you lose all information about that minute. Similarly if you're delayed by a second, you'll miss a spike just after the previous push/scrape. If instead you expose a monotonically increasing counter and calculate a rate over that in the server, you don't lose that data.

What do you use for push-oriented time-series metrics? We are thinking of using LogStash (it has a go client). Unfortunately, as I understand, it's not time-series oriented.

Rapzid 12y ago

I push all my logs into logstash->elasticsearch. For metrics I go Riemann->Graphite. Riemann comes with a graphite compatible server so I push straight to that for processing and alerting. I also send from Riemann to logstash and logstash to Riemann where it makes sense. For my metrics dashboard I use Grafana which is really awesome. I make use of its templating pretty heavily as I run a lot of individual IIS clusters. I can create cluster overview and drill down dashboards and template it so I can just change the cluster number to see its stats. You can also link directly into a dashboard passing variables as a query string parameter. Pretty excellent.

bbrazil 12y ago

https://github.com/rcrowley/go-metrics https://github.com/rcrowley/go-metrics is one I've came across while researching potential Prometheus integrations, it seems to support a number of backends including Graphite and InfluxDB. One challenge with that model of metrics is that it assumes that the monitoring system doesn't have much ability to work with samples, so it makes up for this by calculating rates and quantiles in the client. This isn't as accurate as doing it in the server, and some instrumentation systems don't allow you to extract out the raw data to allow you to do it in the server. For example say you've a 1-minute rate exported/scraped every minute. If you miss a push/scrape you lose all information about that minute. Similarly if you're delayed by a second, you'll miss a spike just after the previous push/scrape. If instead you expose a monotonically increasing counter and calculate a rate over that in the server, you don't lose that data.