4 ms·

In what way do you see those alternatives as superior?

by nicky0 3y ago

paulddraper 3y ago
They avoid an unnecessary invocation of the cat executable. Instead, they open a file descriptor and pass that. Tiny difference but there you go.
- latexr 3y ago
  To add, searching for “useless use of cat” will yield several results for those interested in learning more. Other examples include “useless use of echo” and “useless use of ls *”.
  - vram22 3y ago
    Yes. Even the Wikipedia page on cat has a section about that, titled eponymously. https://en.m.wikipedia.org/wiki/Cat_(Unix) https://en.m.wikipedia.org/wiki/Cat_(Unix)
- adrianmonk 3y ago
  Not just that, but also all the bytes have to go through an extra pipe. Presumably they're copied an extra time because of this. When you run "cmd < file", the command reads from stdin, which pulls directly from the file. When you do "cat file | cmd", "cat" opens the file, reads from there, and writes to a pipe. Then "cmd" reads from its stdin, which is a pipe.
  - chlorion 3y ago
    GNU cat will use the copy_file_range syscall when possible! copy_file_range allows a user land program to copy data between two files without doing any user space work. Instead of reading data into a buffer and writing it back out to the destination, the kernel will somehow manage to move the data for you. I think this will prevent any extra copies from occurring in situations where it can be used. https://man.archlinux.org/man/copy_file_range.2 https://man.archlinux.org/man/copy_file_range.2 https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat.c#n507 https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat...
- omginternets 3y ago
  >They avoid an unnecessary invocation of the cat executable. And ... ?
- wpm 3y ago
  I teach shell scripting. Cat invocations are cheap and help learners understand and keep clear where input is coming from, and where it is going. There are no awards or benefits to reducing the number of lines, commands invoked, or finding the shortest possible way to perform a task in a script. There are plenty of detriments to reading and understanding though when we try to obfuscate this to save 1ms of execution time on a script that is going to execute near instantaneously anyways. In short, I straight up don't care.
  - revscat 3y ago
    I 100% agree with you. My only defense of OP is that `<` is something tends to be forgotten. Like everyone else in this thread I go to `cat` first for things like this. But sometimes I forget that even `<` exists, and the callout is a nice reminder.
derefr 3y ago
If the command is meant to stream through something really fast by using a large buffer size, then prepending a cat(1) will limit the incoming buffer size to ~4k.
- vram22 3y ago
  Interesting. Maybe use dd with one of its blocksize options, then? Not at a terminal, can't check.
gdavisson 3y ago
It usually doesn't matter much, but there are some situations where it can matter a lot. For one thing, you can't use seek() on a pipe, so e.g. `cat bigfile | tail` has to read through the entire file to find the end, but `tail bigfile` will read the file backward from the end, completely skipping the irrelevant beginning and middle. With `pv bigfile | whatever`, pv (which is basically a pipeline progress indicator) can tell how big file is and tell you how for through you are as a percentage; with `cat bigfile | pv | whatever`, it has no idea (unless you add a flag to tell it). Also, `cat bigfile | head` will end up killing cat with a SIGPIPE signal after head exits; if you're using something like "Unofficial bash strict mode" [1], this will cause your script to exit prematurely. Another sometimes-important difference is that if there are multiple input files, `somecommand file1 file2 file3` can tell what data is coming from which file; with `cat file1 file2 file3 | somecommand` they're all mashed together, and the program has no idea what's coming from where. In general, though, I think it's mostly a matter of people's expertise level in using the shell. If you're a beginner, it makes sense to learn one very general way to do things (`cat |`), and use it everywhere. But as you gain expertise, you learn other ways of doing it, and will choose the best method for each specific situation. While `cat |` is usually an ok method to read from a file, it's almost never the best method, so expert shell users will almost never use it. [1] http://redsymbol.net/articles/unofficial-bash-strict-mode/ http://redsymbol.net/articles/unofficial-bash-strict-mode/

In what way do you see those alternatives as superior?

by nicky0 3y ago

In what way do you see those alternatives as superior?

They avoid an unnecessary invocation of the cat executable. Instead, they open a file descriptor and pass that. Tiny difference but there you go.

latexr 3y ago
To add, searching for “useless use of cat” will yield several results for those interested in learning more. Other examples include “useless use of echo” and “useless use of ls *”.
- vram22 3y ago
  Yes. Even the Wikipedia page on cat has a section about that, titled eponymously. https://en.m.wikipedia.org/wiki/Cat_(Unix) https://en.m.wikipedia.org/wiki/Cat_(Unix)
adrianmonk 3y ago
Not just that, but also all the bytes have to go through an extra pipe. Presumably they're copied an extra time because of this. When you run "cmd < file", the command reads from stdin, which pulls directly from the file. When you do "cat file | cmd", "cat" opens the file, reads from there, and writes to a pipe. Then "cmd" reads from its stdin, which is a pipe.
- chlorion 3y ago
  GNU cat will use the copy_file_range syscall when possible! copy_file_range allows a user land program to copy data between two files without doing any user space work. Instead of reading data into a buffer and writing it back out to the destination, the kernel will somehow manage to move the data for you. I think this will prevent any extra copies from occurring in situations where it can be used. https://man.archlinux.org/man/copy_file_range.2 https://man.archlinux.org/man/copy_file_range.2 https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat.c#n507 https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat...
omginternets 3y ago
>They avoid an unnecessary invocation of the cat executable. And ... ?
wpm 3y ago
I teach shell scripting. Cat invocations are cheap and help learners understand and keep clear where input is coming from, and where it is going. There are no awards or benefits to reducing the number of lines, commands invoked, or finding the shortest possible way to perform a task in a script. There are plenty of detriments to reading and understanding though when we try to obfuscate this to save 1ms of execution time on a script that is going to execute near instantaneously anyways. In short, I straight up don't care.
- revscat 3y ago
  I 100% agree with you. My only defense of OP is that `<` is something tends to be forgotten. Like everyone else in this thread I go to `cat` first for things like this. But sometimes I forget that even `<` exists, and the callout is a nice reminder.

derefr 3y ago

If the command is meant to stream through something really fast by using a large buffer size, then prepending a cat(1) will limit the incoming buffer size to ~4k.

vram22 3y ago
Interesting. Maybe use dd with one of its blocksize options, then? Not at a terminal, can't check.

gdavisson 3y ago

It usually doesn't matter much, but there are some situations where it can matter a lot. For one thing, you can't use seek() on a pipe, so e.g. `cat bigfile | tail` has to read through the entire file to find the end, but `tail bigfile` will read the file backward from the end, completely skipping the irrelevant beginning and middle. With `pv bigfile | whatever`, pv (which is basically a pipeline progress indicator) can tell how big file is and tell you how for through you are as a percentage; with `cat bigfile | pv | whatever`, it has no idea (unless you add a flag to tell it). Also, `cat bigfile | head` will end up killing cat with a SIGPIPE signal after head exits; if you're using something like "Unofficial bash strict mode" [1], this will cause your script to exit prematurely. Another sometimes-important difference is that if there are multiple input files, `somecommand file1 file2 file3` can tell what data is coming from which file; with `cat file1 file2 file3 | somecommand` they're all mashed together, and the program has no idea what's coming from where. In general, though, I think it's mostly a matter of people's expertise level in using the shell. If you're a beginner, it makes sense to learn one very general way to do things (`cat |`), and use it everywhere. But as you gain expertise, you learn other ways of doing it, and will choose the best method for each specific situation. While `cat |` is usually an ok method to read from a file, it's almost never the best method, so expert shell users will almost never use it. [1] http://redsymbol.net/articles/unofficial-bash-strict-mode/ http://redsymbol.net/articles/unofficial-bash-strict-mode/