3 ms·

In what format should that filter output? Which fields should it show? Should it be specific for the application or should it be generic? What parameters should

by CrLf 15y ago

In what format should that filter output? Which fields should it show? Should it be specific for the application or should it be generic? What parameters should it accept in this case? Do you see where I'm going with this? We have been using text-based logs and text-based pipes on unix for decades, even though it has always been inefficient to parse (especially on ancient machines). The thing is: it's much more efficient for humans. I understand the kind of cognitive dissonance this causes in programmers, I feel it myself. However, I've also been a sysadmin for a few years now and understand just how having plain text makes my life (much) easier.

sneak 15y ago
Fine, then: tail -f logfile | json2line '%date %name %level %shortmsg' | grep expr Machine-readable logs are a huge win. It's trivial to make them backward-compatible with the entire universe of existing-format log processing tools (including your mind). To put up make-believe barriers in the way is simply fear of beneficial change. (To answer your question directly: The filter should be generic, and should allow you to specify simple format strings to be yanked out of top-level json fields.)
- CrLf 15y ago
  I often find myself troubleshooting applications for which I know little about, or that I don't have to troubleshoot often enough (fortunately). I don't want to have to look what fields are available in the log to prepare the proper format string with the fields that I want because... I do not know what I want. I want to go to "/var/log/application" or whatever place the application stores its logs, and cat/tail whatever it is there and seems remotely related to the problem at hand. This is one of the reasons it is painful to troubleshoot problems in Windows using PowerShell, where what you suggested is more or less what needs to be done to parse the event log in a more intelligent way that filtering stuff using the Event Viewer. This shows in the speed and efficiency in solving issues with unix vs. windows systems. But you can say that's just my anecdotal evidence.

by CrLf 15y ago

sneak 15y ago

Fine, then: tail -f logfile | json2line '%date %name %level %shortmsg' | grep expr Machine-readable logs are a huge win. It's trivial to make them backward-compatible with the entire universe of existing-format log processing tools (including your mind). To put up make-believe barriers in the way is simply fear of beneficial change. (To answer your question directly: The filter should be generic, and should allow you to specify simple format strings to be yanked out of top-level json fields.)

CrLf 15y ago
I often find myself troubleshooting applications for which I know little about, or that I don't have to troubleshoot often enough (fortunately). I don't want to have to look what fields are available in the log to prepare the proper format string with the fields that I want because... I do not know what I want. I want to go to "/var/log/application" or whatever place the application stores its logs, and cat/tail whatever it is there and seems remotely related to the problem at hand. This is one of the reasons it is painful to troubleshoot problems in Windows using PowerShell, where what you suggested is more or less what needs to be done to parse the event log in a more intelligent way that filtering stuff using the Event Viewer. This shows in the speed and efficiency in solving issues with unix vs. windows systems. But you can say that's just my anecdotal evidence.