3 ms·
Does anyone know of any good solutions for feature extraction from logs, for the purpose of making graphs out of these features? Has anyone integrated a graphi
by pmoriarty 12y ago
Does anyone know of any good solutions for feature extraction from logs, for the purpose of making graphs out of these features?
Has anyone integrated a graphing system like this in to Graylog/Logstash and care to share some lessons or advice on how you did it?
Finally, monitoring suites like Nagios an Zabbix: they have their own data-collection and graphing features. When tools like Graphite are used in conjunction with these, would you bypass them entirely? Or would you leverage their data-collection features to first grabb the data and then somehow funnel that data in to Graphite?
If it's the former, what do you use instaed of all thosebuilt-in data collection tools? If it's the latter, how do you do it?
- dothebart 12y agoI've used collectd's tail module for this. See https://collectd.org/wiki/index.php/Tail https://collectd.org/wiki/index.php/Tail for details.
- grosskur 12y agoRyan Smith's "Building Metrics From Log Data" is an interesting talk about this: http://vimeo.com/68183624 http://vimeo.com/68183624 Also check out Heroku's Lumbermill project, which handles extracting router metrics on their platform: https://github.com/heroku/lumbermill https://github.com/heroku/lumbermill If you're on AWS, CloudWatch is capable of ingesting log data and extracting metrics via pattern-matching: http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/WhatIsCloudWatchLogs.html http://docs.aws.amazon.com/AmazonCloudWatch/latest/Developer... When I used Graphite with Nagios, I bypassed all the Nagios data-collection and graphing features. Instead, I funneled all the data into Graphite and used check-graphite to alert on it: https://github.com/pyr/check-graphite https://github.com/pyr/check-graphite