3 ms·

I think you're mistaken here, and confusing two different usages of cat. "Useless" uses of cat aren't bad habits during interactive usage, for all the reasons

I think you're mistaken here, and confusing two different usages of cat. "Useless" uses of cat aren't bad habits during interactive usage, for all the reasons people mention here which I won't rehash. For scripts, however, the story is different than for one-off commands. For one thing, it's slower due to the extra forks and copying of data across pipes, so there's at least that. For another, it prevents the command from inspecting the other end of its pipe, which can negatively impact usage in some case. (For example, if the program knows its input is from a terminal, it may flush its output on every newline it sees.) Moreover, a bunch of the arguments for the interaction case (like "it's fewer keystrokes" or whatever) don't even apply to the script case in the first place... The end result here is that you definitely shouldn't assume some habit is just fine with scripting merely because it's fine when you're typing on the terminal, or vice-versa.

thaumaturgy 3y ago
Those are all reasonable points, but: For shell scripts, I would argue quite vehemently that the most important goals should be correctness and readability, with performance being a very distant third concern. I'd even be tempted to argue that performance shouldn't be a consideration at all, except of course that argument would be misinterpreted to support some absurd edge case until I'd have to admit that of course performance is a little bit of a concern. But in any case, I can't recall a single example of a cat pipe being the root cause of an unacceptable performance problem in a shell script. On the readability point, the example that probably irritates me most often is a cat pipe into some commands into a while loop. I much prefer this: cat file.conf | sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do... to this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... or this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do stuff... done < file.conf ...and that's a pretty common pattern where the edge case of reading input from a terminal doesn't apply. So this is the kind of thing that makes me go "shut up shellcheck" instead of "thanks shellcheck!"
- dataflow 3y ago
  Fork performance is a much more severe problem on Windows (WSL1, MSYS2, etc.) than Linux, so I'm not claiming you'll personally run into it per se, but it can affect users of some scripts. But: performance was just one of the problems I cited. I gave you more than that -- one was a correctness reason (which you do care about) and had nothing to do with performance. And, again, incorrect buffering (which can make the script literally unusable in some cases) was just one example. I've seen needless redirection interfere with Ctrl+C handling too, though I don't recall the exact example. Oh, and there's terminal coloring and ANSI escape processing too, which programs detect differently. Point is, being unable to see the end of the pipe can definitely cause an unnecessary mess in some cases. As for readability - honestly, part of the reason you find it less readable is that you're missing something else. Namely, this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... should really have been: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' -- wrongfile.conf | tr... | while read line; do... which is in fact both more correct (at least when the file name isn't hard-coded, which is the common case in shell scripts) and more readable than your example; you can immediately spot where the file name is. The difference between that and cat "$blah" | sed ... is very minor at that point (and in fact you should be doing cat -- "$blah" as well...); anybody reading a command like sed without a pipe input knows to look for an input argument. The important point regarding readability here is, it's not like the code gets overly tricky if you write it one way vs. another way. It's just a matter of spending 1-2 extra seconds glancing over. So it's very much a minor thing to be prioritizing above everything else. (If the logic became harder to reason about, that'd be a different story, and it'd put more weight on the readability aspect.)
  - thaumaturgy 3y ago
    It's unclear if you missed the filename being wrongfile.conf. Embedding a -- in the middle of a long series of arguments isn't the magic pixie dust that suddenly makes the filename argument stand out the way that it does when it's the very first argument in the pipe. Yes, I saw your other points, and I chose this example because it is an example drawn from real-world use where there is zero objective reason to wag a finger about "useless use of cat". Those other points are not relevant in this example, and piping a cat into some other commands into a while loop is pretty typical shellcode. Forcing me to move a filename argument into the middle of a long line for stylistic reasons should be obviously wrong. It is one case where shellcheck is over-reaching and being a nuisance rather than helping me catch errors. This has been argued better and to death already: https://stackoverflow.com/a/16619430 https://stackoverflow.com/a/16619430, http://oletange.blogspot.com/2013/10/useless-use-of-cat.html http://oletange.blogspot.com/2013/10/useless-use-of-cat.html, https://news.ycombinator.com/item?id=23341711 https://news.ycombinator.com/item?id=23341711, https://news.ycombinator.com/item?id=36116208 https://news.ycombinator.com/item?id=36116208, https://news.ycombinator.com/item?id=6367319 https://news.ycombinator.com/item?id=6367319, https://news.ycombinator.com/item?id=1116085 https://news.ycombinator.com/item?id=1116085, etc.

thaumaturgy 3y ago

Those are all reasonable points, but: For shell scripts, I would argue quite vehemently that the most important goals should be correctness and readability, with performance being a very distant third concern. I'd even be tempted to argue that performance shouldn't be a consideration at all, except of course that argument would be misinterpreted to support some absurd edge case until I'd have to admit that of course performance is a little bit of a concern. But in any case, I can't recall a single example of a cat pipe being the root cause of an unacceptable performance problem in a shell script. On the readability point, the example that probably irritates me most often is a cat pipe into some commands into a while loop. I much prefer this: cat file.conf | sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do... to this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... or this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' | tr... | while read line; do stuff... done < file.conf ...and that's a pretty common pattern where the edge case of reading input from a terminal doesn't apply. So this is the kind of thing that makes me go "shut up shellcheck" instead of "thanks shellcheck!"

dataflow 3y ago
Fork performance is a much more severe problem on Windows (WSL1, MSYS2, etc.) than Linux, so I'm not claiming you'll personally run into it per se, but it can affect users of some scripts. But: performance was just one of the problems I cited. I gave you more than that -- one was a correctness reason (which you do care about) and had nothing to do with performance. And, again, incorrect buffering (which can make the script literally unusable in some cases) was just one example. I've seen needless redirection interfere with Ctrl+C handling too, though I don't recall the exact example. Oh, and there's terminal coloring and ANSI escape processing too, which programs detect differently. Point is, being unable to see the end of the pipe can definitely cause an unnecessary mess in some cases. As for readability - honestly, part of the reason you find it less readable is that you're missing something else. Namely, this: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' wrongfile.conf | tr... | while read line; do... should really have been: sed -e 's/pattern/replacement/g' -e 's/reallybigolhonkinpattern/other-replacement/g' -- wrongfile.conf | tr... | while read line; do... which is in fact both more correct (at least when the file name isn't hard-coded, which is the common case in shell scripts) and more readable than your example; you can immediately spot where the file name is. The difference between that and cat "$blah" | sed ... is very minor at that point (and in fact you should be doing cat -- "$blah" as well...); anybody reading a command like sed without a pipe input knows to look for an input argument. The important point regarding readability here is, it's not like the code gets overly tricky if you write it one way vs. another way. It's just a matter of spending 1-2 extra seconds glancing over. So it's very much a minor thing to be prioritizing above everything else. (If the logic became harder to reason about, that'd be a different story, and it'd put more weight on the readability aspect.)
- thaumaturgy 3y ago
  It's unclear if you missed the filename being wrongfile.conf. Embedding a -- in the middle of a long series of arguments isn't the magic pixie dust that suddenly makes the filename argument stand out the way that it does when it's the very first argument in the pipe. Yes, I saw your other points, and I chose this example because it is an example drawn from real-world use where there is zero objective reason to wag a finger about "useless use of cat". Those other points are not relevant in this example, and piping a cat into some other commands into a while loop is pretty typical shellcode. Forcing me to move a filename argument into the middle of a long line for stylistic reasons should be obviously wrong. It is one case where shellcheck is over-reaching and being a nuisance rather than helping me catch errors. This has been argued better and to death already: https://stackoverflow.com/a/16619430 https://stackoverflow.com/a/16619430, http://oletange.blogspot.com/2013/10/useless-use-of-cat.html http://oletange.blogspot.com/2013/10/useless-use-of-cat.html, https://news.ycombinator.com/item?id=23341711 https://news.ycombinator.com/item?id=23341711, https://news.ycombinator.com/item?id=36116208 https://news.ycombinator.com/item?id=36116208, https://news.ycombinator.com/item?id=6367319 https://news.ycombinator.com/item?id=6367319, https://news.ycombinator.com/item?id=1116085 https://news.ycombinator.com/item?id=1116085, etc.