3 ms·

Hi, usually grep saves the day. I will check your tool, but what I need is for a terminal command that can recognize the meta fields from a log record and put

Hi, usually grep saves the day. I will check your tool, but what I need is for a terminal command that can recognize the meta fields from a log record and put them on a line separated from the main message. Also, it must be installed everywhere I work, which is not so easy. Putting logs in a table with filtering capabilities might be best, but this means web access to the location of the logs which is again tricky.

ryanianian 5y ago
> what I need is for a terminal command that can recognize the meta fields from a log record and put them on a line separated from the main message Isn't this the exact use-case of structured logging? Log events have {timestamp, log level, log category, string message, ...arbitrary key/value pairs} Usually serializing each message as a single json line in a file. Since it's all on one line you can still use grep, but then since it's machine-readable you can pipe the grep to anything that can parse json. Vanilla python3 works and tends to be a part of most ops toolkits. Such tooling can split out the fields onto other lines etc or in a more reader-friendly format.
- gostsamo 5y ago
  Yes, this has been my idea in many cases, but it is not always that I have a say over the logging format.
akavel 5y ago
The idea of my tool is not really to help in some specific way of processing logs, but rather to make it easier to fiddle with grep and other Linux CLI filtering tools, by shortening the feedback loop vs. normal shell REPL. I'm not sure if that sounds in any way clear; also, it might sound strange that shell REPL is IMO too slow, or that it is in any way important, but I found it enough so that I invented a way to speed it up, and it seemed to hit a nerve of quite many people, seeing from the reception the tool got. I can try to explain more if you are interested and/or don't really understand what I might be even talking about. I tried to explain it in the readme, but for many reasons I have no slightest freaking idea to what extent it might sound understandable to you - not the least due to the fact, that for fully-sighted people one important way I tried to convey the idea is through an animated gif of a terminal window showing how the tool is used. As someone said, with really innovative ideas, it's often necessary to push them down people's throats to make them understand those; this gif is part of that effort, and fortunately seems quite effective, but even it is not enough for some people, and I'm by default assuming that it's for obvious reasons completely inaccessible to you. I was pondering just now whether trying to copy this animation to asciinema could make it in any way more accessible to you, but as of now I have high doubts if that would work at all (including that I have no idea whether the asciinema site is at all accessible to you, as well as that I'm highly suspicious the terminal ANSI sequences generated by a library I'm using are "tiniest diffs", so although the result might look indistinguishable to fully-sighted people, a screen reader might [or might not???] take them at face value and read them as a mess of random jumps and single-character changes on the screen). That said, I'm more than 100% happy if you managed to understand what the tool is doing, and just don't think it could be useful to you, whether as is or with some accessibility improvement attempts. Or if you don't understand but don't feel like wanting to dive deeper either.

ryanianian 5y ago

> what I need is for a terminal command that can recognize the meta fields from a log record and put them on a line separated from the main message Isn't this the exact use-case of structured logging? Log events have {timestamp, log level, log category, string message, ...arbitrary key/value pairs} Usually serializing each message as a single json line in a file. Since it's all on one line you can still use grep, but then since it's machine-readable you can pipe the grep to anything that can parse json. Vanilla python3 works and tends to be a part of most ops toolkits. Such tooling can split out the fields onto other lines etc or in a more reader-friendly format.

gostsamo 5y ago
Yes, this has been my idea in many cases, but it is not always that I have a say over the logging format.

akavel 5y ago

The idea of my tool is not really to help in some specific way of processing logs, but rather to make it easier to fiddle with grep and other Linux CLI filtering tools, by shortening the feedback loop vs. normal shell REPL. I'm not sure if that sounds in any way clear; also, it might sound strange that shell REPL is IMO too slow, or that it is in any way important, but I found it enough so that I invented a way to speed it up, and it seemed to hit a nerve of quite many people, seeing from the reception the tool got. I can try to explain more if you are interested and/or don't really understand what I might be even talking about. I tried to explain it in the readme, but for many reasons I have no slightest freaking idea to what extent it might sound understandable to you - not the least due to the fact, that for fully-sighted people one important way I tried to convey the idea is through an animated gif of a terminal window showing how the tool is used. As someone said, with really innovative ideas, it's often necessary to push them down people's throats to make them understand those; this gif is part of that effort, and fortunately seems quite effective, but even it is not enough for some people, and I'm by default assuming that it's for obvious reasons completely inaccessible to you. I was pondering just now whether trying to copy this animation to asciinema could make it in any way more accessible to you, but as of now I have high doubts if that would work at all (including that I have no idea whether the asciinema site is at all accessible to you, as well as that I'm highly suspicious the terminal ANSI sequences generated by a library I'm using are "tiniest diffs", so although the result might look indistinguishable to fully-sighted people, a screen reader might [or might not???] take them at face value and read them as a mess of random jumps and single-character changes on the screen). That said, I'm more than 100% happy if you managed to understand what the tool is doing, and just don't think it could be useful to you, whether as is or with some accessibility improvement attempts. Or if you don't understand but don't feel like wanting to dive deeper either.