3 ms·

I did an experiment with logstash, elasticsearch, and Kibana for our HAProxy logs. The default with logstash was to store each field, and then the whole text so

I did an experiment with logstash, elasticsearch, and Kibana for our HAProxy logs. The default with logstash was to store each field, and then the whole text so stuff go quite large for our web logs. Also the Kibana interface was pretty buggy. Parsing our logs (~2k entries) a second doesn't work well with Regex so in our version we do a bunch of substring stuff. I'm excited about Kibana/ES for the rest of our logs over with their recent hire. When I looked at StatsD and Graphite last time I didn't really see an API. I really like the model of data being queryable and returning nice serialized format like json (like OpenTSDB does). I'm also not that fond of the "many files" model and the automatic data summarization as it ages (it does save space, but makes forecasting difficult as it can skew data).

zwily 13y ago
We're parsing 2k lines/sec with logstash using regexes. We scaled it out to 6 logstash processes across 2 nodes. They pull off a shared redis queue and then insert the results directly into elasticsearch. (That said, I'd like to configure our apache logs to just output the json logstash expects.) Graphite has a very simple and powerful JSON API [1]. Any graph URL can include &format=json and you'll get back the raw JSON values for the datapoints. I haven't used OpenTSDB yet - I'm curious how its API is better. You get to choose the levels of summarization. If you want to keep 1 second intervals for a year, you're welcome to do so. [1] http://graphite.readthedocs.org/en/0.9.12/render_api.html http://graphite.readthedocs.org/en/0.9.12/render_api.html