4 ms·

The infamous "useless use of cat"

adrusi 7y ago
That's relevant in some scripts, where unnecessary forks might actually add up to some meaningful resource usage. Not relevant when typed at a tty.
nsajko 7y ago
To clarify, it would have been prettier (conceptually less clunky and shorter) to run this: time grep tt9184994 < title_complete_json.txt >> result
- catucia 7y ago
  grep takes a file directly, so this could simply be: time grep tt9184994 title_complete_json.txt >> result
hyper_reality 7y ago
It annoys me whenever somebody brings this up. People do this because it's easier to modify commands at the end of your prompt than near the beginning. Especially in this case, where the grandparent wants to quickly switch between benchmarking grep, ripgrep etc. Not useless at all.
- naniwaduni 7y ago
  Though a neat thing to know is that redirections can appear anywhere on the line in Bourneish shells: time < title_complete_json.txt grep tt9184994 As it happens, when benchmarking grep-like tools on large files, this is actually somewhat likely to make a significant difference since the "useless" use of cat actually forces the file to be read sequentially as a stream (because it's piped through), while a process receiving a file on standard input can treat it as a file and e.g. mmap it.
  - hyper_reality 7y ago
    I found a benchmark that suggests the performance gap between a pipe and input redirection is insignificant for most workloads, but there would indeed start to be a difference for the exact case discussed here (grepping a large file): http://oletange.blogspot.com/2013/10/useless-use-of-cat.html http://oletange.blogspot.com/2013/10/useless-use-of-cat.html Thanks for your tip, it's clearly the best of both worlds for this purpose.
    - burntsushi 7y ago
      Incidentally, ripgrep will actually be faster in this case generally (on Linux at least) when given an explicit file path, since that lets it use memory maps. For example: $ time rg QZQZQZQZ < subtitles.en.txt real 1.842 user 0.552 sys 1.287 maxmem 9 MB faults 0 $ time rg QZQZQZQZ subtitles.en.txt real 1.210 user 0.776 sys 0.433 maxmem 9510 MB faults 0 (See my other comments in this thread for where to get subtitles.en.txt.)
mnbvkhgvmj 7y ago
Surely using cat like this is more in line with the unix philosophy?

That's relevant in some scripts, where unnecessary forks might actually add up to some meaningful resource usage. Not relevant when typed at a tty.

nsajko 7y ago

To clarify, it would have been prettier (conceptually less clunky and shorter) to run this: time grep tt9184994 < title_complete_json.txt >> result

catucia 7y ago
grep takes a file directly, so this could simply be: time grep tt9184994 title_complete_json.txt >> result

hyper_reality 7y ago

It annoys me whenever somebody brings this up. People do this because it's easier to modify commands at the end of your prompt than near the beginning. Especially in this case, where the grandparent wants to quickly switch between benchmarking grep, ripgrep etc. Not useless at all.

naniwaduni 7y ago
Though a neat thing to know is that redirections can appear anywhere on the line in Bourneish shells: time < title_complete_json.txt grep tt9184994 As it happens, when benchmarking grep-like tools on large files, this is actually somewhat likely to make a significant difference since the "useless" use of cat actually forces the file to be read sequentially as a stream (because it's piped through), while a process receiving a file on standard input can treat it as a file and e.g. mmap it.
- hyper_reality 7y ago
  I found a benchmark that suggests the performance gap between a pipe and input redirection is insignificant for most workloads, but there would indeed start to be a difference for the exact case discussed here (grepping a large file): http://oletange.blogspot.com/2013/10/useless-use-of-cat.html http://oletange.blogspot.com/2013/10/useless-use-of-cat.html Thanks for your tip, it's clearly the best of both worlds for this purpose.
  - burntsushi 7y ago
    Incidentally, ripgrep will actually be faster in this case generally (on Linux at least) when given an explicit file path, since that lets it use memory maps. For example: $ time rg QZQZQZQZ < subtitles.en.txt real 1.842 user 0.552 sys 1.287 maxmem 9 MB faults 0 $ time rg QZQZQZQZ subtitles.en.txt real 1.210 user 0.776 sys 0.433 maxmem 9510 MB faults 0 (See my other comments in this thread for where to get subtitles.en.txt.)

mnbvkhgvmj 7y ago

Surely using cat like this is more in line with the unix philosophy?