3 ms·

What's really useless are all the "useless use of cat" comments (and shellcheck warnings although I take it for a shell script there could be cases where one le

What's really useless are all the "useless use of cat" comments (and shellcheck warnings although I take it for a shell script there could be cases where one less process may be justified [although really if you're at that point you've got other things to worry than cat, sadly]). I use "cat ... | ... | ... " like in TFA and just like many in this thread because it simply makes sense. It's more intuitive. It's easier to read. It requires less braincycles to remember how this or that command wants its parameter passed, etc. I think the "useless use of cat" movement made its time: it failed. Many of us are never going to give up our use(less|ful) of cat (you decide). So stop wasting your time complaining about it.

thaumaturgy 3y ago
I use shellcheck a lot, and I've found so far that its complaints fall neatly into one of two buckets: "oh, whoops, nice catch" and "shut up, shellcheck". "useless use of cat" goes into the latter bucket. Complaining at me about it does not actually improve the code; it's just a nag about a bad habit that, arguably, isn't even a bad habit.
- dataflow 3y ago
  I think you're mistaken here, and confusing two different usages of cat. "Useless" uses of cat aren't bad habits during interactive usage, for all the reasons people mention here which I won't rehash. For scripts, however, the story is different than for one-off commands. For one thing, it's slower due to the extra forks and copying of data across pipes, so there's at least that. For another, it prevents the command from inspecting the other end of its pipe, which can negatively impact usage in some case. (For example, if the program knows its input is from a terminal, it may flush its output on every newline it sees.) Moreover, a bunch of the arguments for the interaction case (like "it's fewer keystrokes" or whatever) don't even apply to the script case in the first place... The end result here is that you definitely shouldn't assume some habit is just fine with scripting merely because it's fine when you're typing on the terminal, or vice-versa.
  - thaumaturgy 3y ago
    Those are all reasonable points, but: For shell scripts, I would argue quite vehemently that the most important goals should be correctness and readability, with performance being a very distant third concern. I'd even be tempted to argue that performance shouldn't be a consideration at all, except of course that argument would be misinterpreted to support some absurd edge case until I'd have to admit that of course performance is a little bit of a concern. But in any case, I can't recall a single example of a cat pipe being the root cause of an unacceptable performance problem in a shell script. On the readability point, the example that probably irritates me most often is a cat pipe into some commands into a while loop. I much prefer this: cat file.conf | sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do... to this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... or this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do stuff... done < file.conf ...and that's a pretty common pattern where the edge case of reading input from a terminal doesn't apply. So this is the kind of thing that makes me go "shut up shellcheck" instead of "thanks shellcheck!"
    - dataflow 3y ago
      Fork performance is a much more severe problem on Windows (WSL1, MSYS2, etc.) than Linux, so I'm not claiming you'll personally run into it per se, but it can affect users of some scripts. But: performance was just one of the problems I cited. I gave you more than that -- one was a correctness reason (which you do care about) and had nothing to do with performance. And, again, incorrect buffering (which can make the script literally unusable in some cases) was just one example. I've seen needless redirection interfere with Ctrl+C handling too, though I don't recall the exact example. Oh, and there's terminal coloring and ANSI escape processing too, which programs detect differently. Point is, being unable to see the end of the pipe can definitely cause an unnecessary mess in some cases. As for readability - honestly, part of the reason you find it less readable is that you're missing something else. Namely, this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... should really have been: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' -- wrongfile.conf | tr... | while read line; do... which is in fact both more correct (at least when the file name isn't hard-coded, which is the common case in shell scripts) and more readable than your example; you can immediately spot where the file name is. The difference between that and cat "$blah" | sed ... is very minor at that point (and in fact you should be doing cat -- "$blah" as well...); anybody reading a command like sed without a pipe input knows to look for an input argument. The important point regarding readability here is, it's not like the code gets overly tricky if you write it one way vs. another way. It's just a matter of spending 1-2 extra seconds glancing over. So it's very much a minor thing to be prioritizing above everything else. (If the logic became harder to reason about, that'd be a different story, and it'd put more weight on the readability aspect.)

thaumaturgy 3y ago

I use shellcheck a lot, and I've found so far that its complaints fall neatly into one of two buckets: "oh, whoops, nice catch" and "shut up, shellcheck". "useless use of cat" goes into the latter bucket. Complaining at me about it does not actually improve the code; it's just a nag about a bad habit that, arguably, isn't even a bad habit.

dataflow 3y ago
I think you're mistaken here, and confusing two different usages of cat. "Useless" uses of cat aren't bad habits during interactive usage, for all the reasons people mention here which I won't rehash. For scripts, however, the story is different than for one-off commands. For one thing, it's slower due to the extra forks and copying of data across pipes, so there's at least that. For another, it prevents the command from inspecting the other end of its pipe, which can negatively impact usage in some case. (For example, if the program knows its input is from a terminal, it may flush its output on every newline it sees.) Moreover, a bunch of the arguments for the interaction case (like "it's fewer keystrokes" or whatever) don't even apply to the script case in the first place... The end result here is that you definitely shouldn't assume some habit is just fine with scripting merely because it's fine when you're typing on the terminal, or vice-versa.
- thaumaturgy 3y ago
  Those are all reasonable points, but: For shell scripts, I would argue quite vehemently that the most important goals should be correctness and readability, with performance being a very distant third concern. I'd even be tempted to argue that performance shouldn't be a consideration at all, except of course that argument would be misinterpreted to support some absurd edge case until I'd have to admit that of course performance is a little bit of a concern. But in any case, I can't recall a single example of a cat pipe being the root cause of an unacceptable performance problem in a shell script. On the readability point, the example that probably irritates me most often is a cat pipe into some commands into a while loop. I much prefer this: cat file.conf | sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do... to this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... or this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do stuff... done < file.conf ...and that's a pretty common pattern where the edge case of reading input from a terminal doesn't apply. So this is the kind of thing that makes me go "shut up shellcheck" instead of "thanks shellcheck!"
  - dataflow 3y ago
    Fork performance is a much more severe problem on Windows (WSL1, MSYS2, etc.) than Linux, so I'm not claiming you'll personally run into it per se, but it can affect users of some scripts. But: performance was just one of the problems I cited. I gave you more than that -- one was a correctness reason (which you do care about) and had nothing to do with performance. And, again, incorrect buffering (which can make the script literally unusable in some cases) was just one example. I've seen needless redirection interfere with Ctrl+C handling too, though I don't recall the exact example. Oh, and there's terminal coloring and ANSI escape processing too, which programs detect differently. Point is, being unable to see the end of the pipe can definitely cause an unnecessary mess in some cases. As for readability - honestly, part of the reason you find it less readable is that you're missing something else. Namely, this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... should really have been: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' -- wrongfile.conf | tr... | while read line; do... which is in fact both more correct (at least when the file name isn't hard-coded, which is the common case in shell scripts) and more readable than your example; you can immediately spot where the file name is. The difference between that and cat "$blah" | sed ... is very minor at that point (and in fact you should be doing cat -- "$blah" as well...); anybody reading a command like sed without a pipe input knows to look for an input argument. The important point regarding readability here is, it's not like the code gets overly tricky if you write it one way vs. another way. It's just a matter of spending 1-2 extra seconds glancing over. So it's very much a minor thing to be prioritizing above everything else. (If the logic became harder to reason about, that'd be a different story, and it'd put more weight on the readability aspect.)