3 ms·

How does that compare to the ELK (ElasticSearch/LogStash/Kibana) stack? (Is it even right to compare?) I'm currently evaluating ElasticSearch + Kibana to handl

How does that compare to the ELK (ElasticSearch/LogStash/Kibana) stack? (Is it even right to compare?) I'm currently evaluating ElasticSearch + Kibana to handle metrics/errors/application logs in our web app, it's impressive but Kibana is lacking a lot of features to be complete IMO. Good point for ELK in my case is that it runs on windows which is not the case of InfluxDB it seems (running on windows is a requirement in my company unfortunately).

phildougherty 12y ago
It's a similar web interface (Grafana is based on Kibana), but for different things. ELK is best for deciphering large amounts of data from logs IMHO. Yes logstash has tons of plugins for third party monitoring systems, but generally speaking you're going to dump your web server access/error logs and syslogs into your ELK setup. Grafana makes graphite/influxdb more useful and is by far and away the best option available. Usually you're going to be tracking things like cpu/memory/diskio/network traffic, but also anything from StatsD in your Grafana setup. Combining the two systems into one could be neat, but they work great on their own and have their own benefits.
- eloycoto 12y ago
  I'm using Grafana + Influxdb + statsd and I'm quite happy: - In our django/python apps I'm using statsd client - For server monitoring I'm using diamond - For dashboards I'm using grafana. I have in a few clicks all company information. I used ELK service, but only for logs but queries were difficult for me. But, ELK is great for PFsense routers[http://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/ http://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-el...], or you can use http://packetbeat.com/ http://packetbeat.com/ if you want to monitor all your servers. Last week I spoke in a local community about my stack. Here you have my slides + examples: http://acalustra.com/statsd-talk-at-python-vigo-meetup.html http://acalustra.com/statsd-talk-at-python-vigo-meetup.html Regards
- Guillaume86 12y ago
  Thanks for the response. I should probably give a shot to Kibana 3.x because I'm on 4.x and I suspect it's still pretty new and lacking in features because of it. I could already track system metrics on windows with WMI (well I know CPU and memory is possible at least) and send them to ES but I will need to hand code it to avoid LogStash (too big for this simple job in my case).
torkelo 12y ago
ELK is about log analytics, Grafana is about Time series & metric analytics and monitoring. The difference might not seem that big but currently it quite a big difference. Time series are usually more about being able to collect huge amount of metrics. Metrics that can then be combined, averaged, filtered, put through a processing pipeline (analytical functions), summarized by different intervals. All in order to visualize (usually through graphs) recent live trends or long term trends and statistics. Grafana is all about maximizing the power and ease of use of the underlying time series store so the user can focus om on building informative and nice looking dashboards. It is also about letting users define generic dashboards through variables that can be used in metric queries, this allows users to reuse the same dashboard for different servers, apps or experiments as long as the metric naming follows a consistent pattern. Grafana also uses Elasticsearch but not for log analytics, but for annotating graphs with event/log information. At some point in in the coming 1-3 years log analytics and metric analytics & visualization is going to converge and be solved/addressed by the same piece of software. But that is tricky right now without sacrificing either domain.
Florin_Andrei 12y ago
Monitoring includes: - fault detection - metrics - logs Grafana + Graphite cover #2. Graphite may also help with #1 if you have something like Sensu watching the metric thresholds. ELK covers #3.

phildougherty 12y ago

It's a similar web interface (Grafana is based on Kibana), but for different things. ELK is best for deciphering large amounts of data from logs IMHO. Yes logstash has tons of plugins for third party monitoring systems, but generally speaking you're going to dump your web server access/error logs and syslogs into your ELK setup. Grafana makes graphite/influxdb more useful and is by far and away the best option available. Usually you're going to be tracking things like cpu/memory/diskio/network traffic, but also anything from StatsD in your Grafana setup. Combining the two systems into one could be neat, but they work great on their own and have their own benefits.

eloycoto 12y ago
I'm using Grafana + Influxdb + statsd and I'm quite happy: - In our django/python apps I'm using statsd client - For server monitoring I'm using diamond - For dashboards I'm using grafana. I have in a few clicks all company information. I used ELK service, but only for logs but queries were difficult for me. But, ELK is great for PFsense routers[http://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/ http://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-el...], or you can use http://packetbeat.com/ http://packetbeat.com/ if you want to monitor all your servers. Last week I spoke in a local community about my stack. Here you have my slides + examples: http://acalustra.com/statsd-talk-at-python-vigo-meetup.html http://acalustra.com/statsd-talk-at-python-vigo-meetup.html Regards
Guillaume86 12y ago
Thanks for the response. I should probably give a shot to Kibana 3.x because I'm on 4.x and I suspect it's still pretty new and lacking in features because of it. I could already track system metrics on windows with WMI (well I know CPU and memory is possible at least) and send them to ES but I will need to hand code it to avoid LogStash (too big for this simple job in my case).

torkelo 12y ago

ELK is about log analytics, Grafana is about Time series & metric analytics and monitoring. The difference might not seem that big but currently it quite a big difference. Time series are usually more about being able to collect huge amount of metrics. Metrics that can then be combined, averaged, filtered, put through a processing pipeline (analytical functions), summarized by different intervals. All in order to visualize (usually through graphs) recent live trends or long term trends and statistics. Grafana is all about maximizing the power and ease of use of the underlying time series store so the user can focus om on building informative and nice looking dashboards. It is also about letting users define generic dashboards through variables that can be used in metric queries, this allows users to reuse the same dashboard for different servers, apps or experiments as long as the metric naming follows a consistent pattern. Grafana also uses Elasticsearch but not for log analytics, but for annotating graphs with event/log information. At some point in in the coming 1-3 years log analytics and metric analytics & visualization is going to converge and be solved/addressed by the same piece of software. But that is tricky right now without sacrificing either domain.

Florin_Andrei 12y ago

Monitoring includes: - fault detection - metrics - logs Grafana + Graphite cover #2. Graphite may also help with #1 if you have something like Sensu watching the metric thresholds. ELK covers #3.