4 ms·

> Looking at a sample of the logs it’s obvious that there is both repetition and structure in this data – similarly structured messages repeat with different pa

> Looking at a sample of the logs it’s obvious that there is both repetition and structure in this data – similarly structured messages repeat with different parameters, dates, numbers, file names, etc. This seems like a great candidate for some automation, let’s see what we can do! While "uno" is, perhaps, "cooler", logcheck [0] has been around for ages and does just this -- go through your log files, weed out all the stuff that you don't care about, and mails you anything else (which, by definition, is all the stuff you do care about). Logcheck doesn't use "fuzziness" or uniqueness or any intelligence or anything like that -- just regexes -- but I end up with one single mail per day that only takes me a minute or two to sift through. Additionally, logcheck is open source. [0]: http://logcheck.org http://logcheck.org

CupOfJava 8y ago
Do you have to setup the regexes ahead of time or does logcheck automatically compute the regexes?
- michaelmior 8y ago
  > Normal entries are entries which match one of the many included regular expression files contain in the database. I haven't used logcheck, but I assume you can add your own regexes, but looks like you have something to start with.
  - jlgaddis 8y ago
    It ships with a ton of "rules" [0]. You can choose to use just those, just your own, or some or all of both. [0]: https://salsa.debian.org/debian/logcheck/tree/master/rulefiles/linux https://salsa.debian.org/debian/logcheck/tree/master/rulefil...
justaj 8y ago
I'm not really sure it's still an active project: └─$ git clone git://git.debian.org/git/logcheck/logcheck.git Cloning into 'logcheck'... fatal: unable to connect to git.debian.org: moszumanska.debian.org[0: 5.153.231.21]: errno=No route to host moszumanska.debian.org[1: 2001:41c8:1000:21::21:21]: errno=Network is unreachable Also none of their mailing lists seem to exist anymore: http://logcheck.org/lists.html http://logcheck.org/lists.html
- jlgaddis 8y ago
  Nowadays it lives on Debian's self-hosted GitLab instance [0]. [0]: https://salsa.debian.org/debian/logcheck https://salsa.debian.org/debian/logcheck

CupOfJava 8y ago

Do you have to setup the regexes ahead of time or does logcheck automatically compute the regexes?

michaelmior 8y ago
> Normal entries are entries which match one of the many included regular expression files contain in the database. I haven't used logcheck, but I assume you can add your own regexes, but looks like you have something to start with.
- jlgaddis 8y ago
  It ships with a ton of "rules" [0]. You can choose to use just those, just your own, or some or all of both. [0]: https://salsa.debian.org/debian/logcheck/tree/master/rulefiles/linux https://salsa.debian.org/debian/logcheck/tree/master/rulefil...

justaj 8y ago

I'm not really sure it's still an active project: └─$ git clone git://git.debian.org/git/logcheck/logcheck.git Cloning into 'logcheck'... fatal: unable to connect to git.debian.org: moszumanska.debian.org[0: 5.153.231.21]: errno=No route to host moszumanska.debian.org[1: 2001:41c8:1000:21::21:21]: errno=Network is unreachable Also none of their mailing lists seem to exist anymore: http://logcheck.org/lists.html http://logcheck.org/lists.html

jlgaddis 8y ago
Nowadays it lives on Debian's self-hosted GitLab instance [0]. [0]: https://salsa.debian.org/debian/logcheck https://salsa.debian.org/debian/logcheck