4 ms·

And if you are building anything other than toy systems then please, for the love of the FSM, use structured logging! If I could get my hands on the developers

by gbuk2013 6y ago

And if you are building anything other than toy systems then please, for the love of the FSM, use structured logging! If I could get my hands on the developers who wrote all the snowflake log formats I had to deal with on a daily basis back when I was doing support and ops …

oftenwrong 6y ago
Came here to say this as well. If you're not using structured logging in 2020, get on it! It's usually trivial to set up. Most of the work will be converting your unstructured log messages to correctly separate out the application-level fields. I cannot emphasise enough how much better it is for searching the logs and running any kind of automation off of the logs (analytics, monitoring, alerting). "Noisy" logs are often cited as a major problem, but well-structured logging allows you to filter out the noise very easily, and focus on only the log events that are relevant to your task.
mamcx 6y ago
I tried this, but, the only way to "use" them is to pay absurd amounts of money for cloud tools or setup complex grafana + ... things. Using them from the command line? Reading them just here? Don't exist anything alike "nano server.log".
- rob-olmos 6y ago
  If the logs are stored in JSON, the jq tool can be used. I understand it's more complicated that grep for certain structures, but also having a library of jq filters for the app may be useful and can be shared.
  - mamcx 6y ago
    But that requiere to learn extra stuff that is not the normal flow: Locate that part of the log or scan it quickly.
- gbuk2013 6y ago
  Not at all - grep piped to jq works like a charm - I have made reports using that in the CLI wrapped in a Makefile that were exec friendly. And there are other tools like gron for example. And if you are building something serious and with some sort of scale then you will need to have something like ELK stack anyway to aggregate and display data - there is no getting away from that. Working with unstructured logs however is a mess. Perl / Awk helps of course but it is a lot of extra effort. And in the end, at scale you will just be writing those scripts to turn that mess into structured data to feed into that same ELK stack anyway.
- wsargent 6y ago
  Off the top of my head: https://github.com/koenbollen/jl https://github.com/koenbollen/jl https://github.com/brocode/fblog https://github.com/brocode/fblog https://github.com/gistia/json-log-viewer https://github.com/gistia/json-log-viewer https://github.com/otros-systems/otroslogviewer https://github.com/otros-systems/otroslogviewer

oftenwrong 6y ago

Came here to say this as well. If you're not using structured logging in 2020, get on it! It's usually trivial to set up. Most of the work will be converting your unstructured log messages to correctly separate out the application-level fields. I cannot emphasise enough how much better it is for searching the logs and running any kind of automation off of the logs (analytics, monitoring, alerting). "Noisy" logs are often cited as a major problem, but well-structured logging allows you to filter out the noise very easily, and focus on only the log events that are relevant to your task.

mamcx 6y ago

I tried this, but, the only way to "use" them is to pay absurd amounts of money for cloud tools or setup complex grafana + ... things. Using them from the command line? Reading them just here? Don't exist anything alike "nano server.log".

rob-olmos 6y ago
If the logs are stored in JSON, the jq tool can be used. I understand it's more complicated that grep for certain structures, but also having a library of jq filters for the app may be useful and can be shared.
- mamcx 6y ago
  But that requiere to learn extra stuff that is not the normal flow: Locate that part of the log or scan it quickly.
gbuk2013 6y ago
Not at all - grep piped to jq works like a charm - I have made reports using that in the CLI wrapped in a Makefile that were exec friendly. And there are other tools like gron for example. And if you are building something serious and with some sort of scale then you will need to have something like ELK stack anyway to aggregate and display data - there is no getting away from that. Working with unstructured logs however is a mess. Perl / Awk helps of course but it is a lot of extra effort. And in the end, at scale you will just be writing those scripts to turn that mess into structured data to feed into that same ELK stack anyway.
wsargent 6y ago
Off the top of my head: https://github.com/koenbollen/jl https://github.com/koenbollen/jl https://github.com/brocode/fblog https://github.com/brocode/fblog https://github.com/gistia/json-log-viewer https://github.com/gistia/json-log-viewer https://github.com/otros-systems/otroslogviewer https://github.com/otros-systems/otroslogviewer