3 ms·

cat /usr/share/dict/words | grep -e '^a..$' | wc -l can be shortened to grep -cx 'a..' /usr/share/dict/words but yeah, author probably just wanted to sho

by asicsp 4y ago

cat /usr/share/dict/words | grep -e '^a..$' | wc -l can be shortened to grep -cx 'a..' /usr/share/dict/words but yeah, author probably just wanted to show pipe example. I usually show `du -sh -- * | sort -h` for such cases.

googlryas 4y ago
Performing 100 of the cat versions on M1 MBP with the default zsh consistently takes between 6.1s and 6.2s, whereas 100 of the grep -cx version consistently takes between 9.1s and 9.3s grep -cx version: for i in {1..100} do grep -cx 'a..' /usr/share/dict/words done cat version: for i in {1..100} do cat /usr/share/dict/words | grep -e "^a..$" | wc -l done (adding a | tr -d " " to the cat version to make the output equivalent only adds about .1s to the total run).
- asicsp 4y ago
  6 and 9 seconds!! Something seems off. Here's what I get (Linux, Intel Gold G6400 CPU @ 4.00GHz): $ time for i in {1..100}; do grep -cx 'a..' /usr/share/dict/words; done > f1 real 0m0.249s user 0m0.207s sys 0m0.046s $ time for i in {1..100}; do cat /usr/share/dict/words | grep -e '^a..$' | wc -l; done > f2 real 0m0.306s user 0m0.386s sys 0m0.141s Edit: probably because your word file is similar to the one in the post? I've got only 39 matching lines from a 102401 line word file.
  - googlryas 4y ago
    235886 lines with 90 matches on whatever is shipped with macos - shasum /usr/share/dict/words = a62edf8685920f7d5a95113020631cdebd18a185 I also put each version in a separate file, so my time commands were time ~/a and time ~/b, and I let the output go to stdout Also, weirdly, the commands you provided don't print the results from the time command - I'm not sure why (I never use zsh, it's just on the computer I'm using at the moment).
    - asicsp 4y ago
      I just tried with 662349 line file having 177 matches ( got the word file from http://app.aspell.net/create http://app.aspell.net/create). I got `0m1.327s` for just grep command and `0m1.412s` for cat+grep+wc version. I use bash, so not sure about zsh. 6+ seconds really seems off.
      - planede 4y ago
        AFAIK GNU grep is a different beast than other grep implementations.
- eddyg 4y ago
  This is an unfair comparison because you aren't using the same `grep` options. What your test shows is that `-cx` is not as efficient. On an x86 MBP: $ time (for i in {1..100}; {grep -cx 'a..' /usr/share/dict/words}) >/dev/null user=9.22s system=0.34s cpu=94% total=10.097 $ time (for i in {1..100}; {grep -e '^a..$' /usr/share/dict/words | wc -l}) >/dev/null user=6.14s system=0.52s cpu=95% total=6.948 $ time (for i in {1..100}; {cat /usr/share/dict/words | grep -e '^a..$' | wc -l}) >/dev/null user=6.29s system=0.80s cpu=100% total=7.024
  - googlryas 4y ago
    I just used the grep command that OP mentioned in their cat abuse post.
    - taskforcegemini 4y ago
      seems that OPs suggestion was only meant to make the command more concise

by asicsp 4y ago

googlryas 4y ago

Performing 100 of the cat versions on M1 MBP with the default zsh consistently takes between 6.1s and 6.2s, whereas 100 of the grep -cx version consistently takes between 9.1s and 9.3s grep -cx version: for i in {1..100} do grep -cx 'a..' /usr/share/dict/words done cat version: for i in {1..100} do cat /usr/share/dict/words | grep -e "^a..$" | wc -l done (adding a | tr -d " " to the cat version to make the output equivalent only adds about .1s to the total run).

asicsp 4y ago
6 and 9 seconds!! Something seems off. Here's what I get (Linux, Intel Gold G6400 CPU @ 4.00GHz): $ time for i in {1..100}; do grep -cx 'a..' /usr/share/dict/words; done > f1 real 0m0.249s user 0m0.207s sys 0m0.046s $ time for i in {1..100}; do cat /usr/share/dict/words | grep -e '^a..$' | wc -l; done > f2 real 0m0.306s user 0m0.386s sys 0m0.141s Edit: probably because your word file is similar to the one in the post? I've got only 39 matching lines from a 102401 line word file.
- googlryas 4y ago
  235886 lines with 90 matches on whatever is shipped with macos - shasum /usr/share/dict/words = a62edf8685920f7d5a95113020631cdebd18a185 I also put each version in a separate file, so my time commands were time ~/a and time ~/b, and I let the output go to stdout Also, weirdly, the commands you provided don't print the results from the time command - I'm not sure why (I never use zsh, it's just on the computer I'm using at the moment).
  - asicsp 4y ago
    I just tried with 662349 line file having 177 matches ( got the word file from http://app.aspell.net/create http://app.aspell.net/create). I got `0m1.327s` for just grep command and `0m1.412s` for cat+grep+wc version. I use bash, so not sure about zsh. 6+ seconds really seems off.
    - planede 4y ago
      AFAIK GNU grep is a different beast than other grep implementations.
eddyg 4y ago
This is an unfair comparison because you aren't using the same `grep` options. What your test shows is that `-cx` is not as efficient. On an x86 MBP: $ time (for i in {1..100}; {grep -cx 'a..' /usr/share/dict/words}) >/dev/null user=9.22s system=0.34s cpu=94% total=10.097 $ time (for i in {1..100}; {grep -e '^a..$' /usr/share/dict/words | wc -l}) >/dev/null user=6.14s system=0.52s cpu=95% total=6.948 $ time (for i in {1..100}; {cat /usr/share/dict/words | grep -e '^a..$' | wc -l}) >/dev/null user=6.29s system=0.80s cpu=100% total=7.024
- googlryas 4y ago
  I just used the grep command that OP mentioned in their cat abuse post.
  - taskforcegemini 4y ago
    seems that OPs suggestion was only meant to make the command more concise