3 ms·

Here's why I often have command chains that start with cat: head -n999 $LOG | grep $WANT | cut $BYTESORFIELDS | sed $CHANGES I build that iteratively, a p

Here's why I often have command chains that start with cat: head -n999 $LOG | grep $WANT | cut $BYTESORFIELDS | sed $CHANGES I build that iteratively, a pipe at a time, until I'm getting the right output. Then I replace "head -n999" with "cat". That saves me a lot of time, as the log file might be hundreds of MB or even GB in size, and working on the whole file when a small representative chunk will do is stupid. Less places to change, and all at the very front, is less likely to make a mistake and put a param in the wrong spot. In that case, cat isn't useless, it was the minimal edit to go from test case to production case, and minimizing differences between testing and production is good engineering.

couchand 6y ago
A reasonable workflow. I suppose the counter argument is that you could replace "head -n999 $LOG |" with "<$LOG" instead.
- kbenson 6y ago
  I could, but them I'm replacing characters before $LOG and a character (or two, depending on spacing) after $LOG as well. It's both conceptually and work-wise easier to consider it as "I take the first X lines of " and "I take all of". and just replace those bits. It's a very small bit of complexity, but when chaining things together, those small bits of complexity build on each other as well, so reducing them where you can reduces the complexity (and problems caused by it) overall. It's the same reason I don't use grep -c usually, but instead always pipe to wc -l. There's less cognitive overhead in having one way to do it that always works, rather than recognizing when I'm ending some command or chain of commands with a grep and I can use a flag on grep to count instead. The "| wc -l" that are always at the end are much easier to notice and correctly remove than a -c flag to grep I might or might not use depending on the last command. Just like they say the best diet is the one you can keep doing, the best/most useful way to formulate your shell commands are the ways that provide you the most benefit. Don't let anyone change your workflow "because you're doing it wrong". At the same time, you should examine suggestions and if you try it out to see if you find it beneficial. It's a fine line to walk. I had a friends that recommended screen for years when I didn't use anything of the sort. I use tmux (an equivalent) religiously now, and I can't imagine going back to life without it, just from the ability to resume work on servers from lost shell connections or from different computers.
oftenwrong 6y ago
One could just as easily say this is AT LEAST a useless use of head and grep because one could replace both with a single POSIX command: awk "NR < 999 && /$WANT/" $LOG | ... ...but such pedantry should only be unleashed on someone pointing out a UUOC.

couchand 6y ago

A reasonable workflow. I suppose the counter argument is that you could replace "head -n999 $LOG |" with "<$LOG" instead.

kbenson 6y ago
I could, but them I'm replacing characters before $LOG and a character (or two, depending on spacing) after $LOG as well. It's both conceptually and work-wise easier to consider it as "I take the first X lines of " and "I take all of". and just replace those bits. It's a very small bit of complexity, but when chaining things together, those small bits of complexity build on each other as well, so reducing them where you can reduces the complexity (and problems caused by it) overall. It's the same reason I don't use grep -c usually, but instead always pipe to wc -l. There's less cognitive overhead in having one way to do it that always works, rather than recognizing when I'm ending some command or chain of commands with a grep and I can use a flag on grep to count instead. The "| wc -l" that are always at the end are much easier to notice and correctly remove than a -c flag to grep I might or might not use depending on the last command. Just like they say the best diet is the one you can keep doing, the best/most useful way to formulate your shell commands are the ways that provide you the most benefit. Don't let anyone change your workflow "because you're doing it wrong". At the same time, you should examine suggestions and if you try it out to see if you find it beneficial. It's a fine line to walk. I had a friends that recommended screen for years when I didn't use anything of the sort. I use tmux (an equivalent) religiously now, and I can't imagine going back to life without it, just from the ability to resume work on servers from lost shell connections or from different computers.

oftenwrong 6y ago

One could just as easily say this is AT LEAST a useless use of head and grep because one could replace both with a single POSIX command: awk "NR < 999 && /$WANT/" $LOG | ... ...but such pedantry should only be unleashed on someone pointing out a UUOC.