5 ms·

Very nice. Not only fast, but feels modern. Tried it out on a 3.5GB JSON file: # rg rg erzg4 k.json > /dev/null 1.80s user 2.54s system 53% cpu 8.053 tot

Very nice. Not only fast, but feels modern. Tried it out on a 3.5GB JSON file: # rg rg erzg4 k.json > /dev/null 1.80s user 2.54s system 53% cpu 8.053 total # rg with 4 threads rg -j4 erzg4 k.json > /dev/null 1.76s user 1.29s system 99% cpu 3.059 total # OS X grep grep erzg4 k.json > /dev/null 60.62s user 0.96s system 99% cpu 1:01.75 total # GNU Grep ggrep erzg4 k.json > /dev/null 1.96s user 1.43s system 88% cpu 2.691 total GNU Grep wins, but it's pretty crusty, especially with regards to its output (even with colourization).

burntsushi 10y ago
My guess is that since you ran `rg` first, the file wasn't in memory, and you ended up benchmarking disk IO. (Notice the sys time decrease from your first run to the second run.) Subsequent commands then run faster with the file already in memory. This is one of many reasons why assembling the benchmarks in my blog post was so difficult. For example, on every command I benchmarked, I ran them 3 times for "warmup" and didn't record any measurements. I then ran them another 10 times in which I recorded them. You can see the raw output here: https://github.com/BurntSushi/ripgrep/blob/master/benchsuite/runs/2016-09-20-ubuntu1604-ec2/raw.csv https://github.com/BurntSushi/ripgrep/blob/master/benchsuite... In any case, on my underpowered Mac, here are some results on a 1.2 GB file (notice how much the time fluctuates until its fully in cache): mac:~ andrew$ ggrep --version ggrep (GNU grep) 2.25 Packaged by Homebrew mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m5.447s user 0m0.600s sys 0m0.350s mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m1.247s user 0m0.549s sys 0m0.264s mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.803s user 0m0.542s sys 0m0.259s mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.805s user 0m0.544s sys 0m0.260s And now for rg: mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null real 0m1.062s user 0m0.339s sys 0m0.333s mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.640s user 0m0.337s sys 0m0.302s mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.637s user 0m0.336s sys 0m0.300s Oh! And check this out, on a Mac, not using a memory map for single files is faster. My goodness---memory map performance is all over the place. mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl --no-mmap > /dev/null real 0m0.445s user 0m0.170s sys 0m0.274s If I do this on my Linux machine on the same file, I get timings of 0.275s for rg, 0.398s for rg with no memory maps (opposite direction for Mac) and 0.708s for GNU grep (v 2.25). Benchmarks are fun, eh?
- atombender 10y ago
  I ran each test four times and picked the best result — not my first rodeo — but for the first result I picked the wrong time from my output, which obviously didn't make use of the cache. Here's it again, complete result, added --no-mmap: # ggrep zerogravitas$ for x in {1..4}; do (time ggrep erzg4 k.json >/dev/null); done ggrep erzg4 k.json > /dev/null 1.96s user 0.67s system 99% cpu 2.641 total ggrep erzg4 k.json > /dev/null 1.95s user 0.68s system 99% cpu 2.660 total ggrep erzg4 k.json > /dev/null 2.00s user 0.66s system 99% cpu 2.672 total ggrep erzg4 k.json > /dev/null 1.96s user 0.67s system 98% cpu 2.662 total # rg zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json >/dev/null); done rg erzg4 k.json > /dev/null 1.76s user 1.40s system 99% cpu 3.180 total rg erzg4 k.json > /dev/null 1.77s user 1.31s system 99% cpu 3.088 total rg erzg4 k.json > /dev/null 1.74s user 1.36s system 99% cpu 3.128 total rg erzg4 k.json > /dev/null 1.76s user 1.41s system 97% cpu 3.265 total # rg --no-mmap zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json --no-mmap >/dev/null); done rg erzg4 k.json --no-mmap > /dev/null 0.98s user 0.75s system 99% cpu 1.743 total rg erzg4 k.json --no-mmap > /dev/null 0.99s user 0.75s system 99% cpu 1.740 total rg erzg4 k.json --no-mmap > /dev/null 1.01s user 0.76s system 99% cpu 1.772 total rg erzg4 k.json --no-mmap > /dev/null 0.99s user 0.75s system 99% cpu 1.754 total # rg -j4 zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json -j4 >/dev/null); done rg erzg4 k.json -j4 > /dev/null 1.75s user 1.35s system 98% cpu 3.134 total rg erzg4 k.json -j4 > /dev/null 1.75s user 1.44s system 98% cpu 3.224 total rg erzg4 k.json -j4 > /dev/null 1.80s user 1.38s system 99% cpu 3.204 total rg erzg4 k.json -j4 > /dev/null 1.80s user 1.35s system 99% cpu 3.164 total # rg -j4 --no-mmap zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json -j4 --no-mmap >/dev/null); done rg erzg4 k.json -j4 --no-mmap > /dev/null 0.98s user 0.75s system 99% cpu 1.740 total rg erzg4 k.json -j4 --no-mmap > /dev/null 0.97s user 0.74s system 99% cpu 1.721 total rg erzg4 k.json -j4 --no-mmap > /dev/null 0.99s user 0.75s system 99% cpu 1.752 total rg erzg4 k.json -j4 --no-mmap > /dev/null 0.98s user 0.76s system 99% cpu 1.748 total Sounds like "alias rg=rg --no-mmap" is a good idea on a Mac.
  - burntsushi 10y ago
    Wow. Those are awesome results, thank you. > Sounds like "alias rg=rg --no-mmap" is a good idea on a Mac. I will fix that in ripgrep proper by making --no-mmap the default on Mac. :-) It should be an easy one to knock off: https://github.com/BurntSushi/ripgrep/issues/36 https://github.com/BurntSushi/ripgrep/issues/36 > not my first rodeo Right, sorry about that. :-) Just had to cover all my bases! (Also, `-j` on a single file won't do anything, and ripgrep should try to use multiple threads by default when searching multiple files.)

burntsushi 10y ago

My guess is that since you ran `rg` first, the file wasn't in memory, and you ended up benchmarking disk IO. (Notice the sys time decrease from your first run to the second run.) Subsequent commands then run faster with the file already in memory. This is one of many reasons why assembling the benchmarks in my blog post was so difficult. For example, on every command I benchmarked, I ran them 3 times for "warmup" and didn't record any measurements. I then ran them another 10 times in which I recorded them. You can see the raw output here: https://github.com/BurntSushi/ripgrep/blob/master/benchsuite/runs/2016-09-20-ubuntu1604-ec2/raw.csv https://github.com/BurntSushi/ripgrep/blob/master/benchsuite... In any case, on my underpowered Mac, here are some results on a 1.2 GB file (notice how much the time fluctuates until its fully in cache): mac:~ andrew$ ggrep --version ggrep (GNU grep) 2.25 Packaged by Homebrew mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m5.447s user 0m0.600s sys 0m0.350s mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m1.247s user 0m0.549s sys 0m0.264s mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.803s user 0m0.542s sys 0m0.259s mac:~ andrew$ time ggrep 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.805s user 0m0.544s sys 0m0.260s And now for rg: mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null real 0m1.062s user 0m0.339s sys 0m0.333s mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.640s user 0m0.337s sys 0m0.302s mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl > /dev/null real 0m0.637s user 0m0.336s sys 0m0.300s Oh! And check this out, on a Mac, not using a memory map for single files is faster. My goodness---memory map performance is all over the place. mac:~ andrew$ time rg 'Bruce Springsteen' foo.jsonl --no-mmap > /dev/null real 0m0.445s user 0m0.170s sys 0m0.274s If I do this on my Linux machine on the same file, I get timings of 0.275s for rg, 0.398s for rg with no memory maps (opposite direction for Mac) and 0.708s for GNU grep (v 2.25). Benchmarks are fun, eh?

atombender 10y ago
I ran each test four times and picked the best result — not my first rodeo — but for the first result I picked the wrong time from my output, which obviously didn't make use of the cache. Here's it again, complete result, added --no-mmap: # ggrep zerogravitas$ for x in {1..4}; do (time ggrep erzg4 k.json >/dev/null); done ggrep erzg4 k.json > /dev/null 1.96s user 0.67s system 99% cpu 2.641 total ggrep erzg4 k.json > /dev/null 1.95s user 0.68s system 99% cpu 2.660 total ggrep erzg4 k.json > /dev/null 2.00s user 0.66s system 99% cpu 2.672 total ggrep erzg4 k.json > /dev/null 1.96s user 0.67s system 98% cpu 2.662 total # rg zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json >/dev/null); done rg erzg4 k.json > /dev/null 1.76s user 1.40s system 99% cpu 3.180 total rg erzg4 k.json > /dev/null 1.77s user 1.31s system 99% cpu 3.088 total rg erzg4 k.json > /dev/null 1.74s user 1.36s system 99% cpu 3.128 total rg erzg4 k.json > /dev/null 1.76s user 1.41s system 97% cpu 3.265 total # rg --no-mmap zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json --no-mmap >/dev/null); done rg erzg4 k.json --no-mmap > /dev/null 0.98s user 0.75s system 99% cpu 1.743 total rg erzg4 k.json --no-mmap > /dev/null 0.99s user 0.75s system 99% cpu 1.740 total rg erzg4 k.json --no-mmap > /dev/null 1.01s user 0.76s system 99% cpu 1.772 total rg erzg4 k.json --no-mmap > /dev/null 0.99s user 0.75s system 99% cpu 1.754 total # rg -j4 zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json -j4 >/dev/null); done rg erzg4 k.json -j4 > /dev/null 1.75s user 1.35s system 98% cpu 3.134 total rg erzg4 k.json -j4 > /dev/null 1.75s user 1.44s system 98% cpu 3.224 total rg erzg4 k.json -j4 > /dev/null 1.80s user 1.38s system 99% cpu 3.204 total rg erzg4 k.json -j4 > /dev/null 1.80s user 1.35s system 99% cpu 3.164 total # rg -j4 --no-mmap zerogravitas$ for x in {1..4}; do (time rg erzg4 k.json -j4 --no-mmap >/dev/null); done rg erzg4 k.json -j4 --no-mmap > /dev/null 0.98s user 0.75s system 99% cpu 1.740 total rg erzg4 k.json -j4 --no-mmap > /dev/null 0.97s user 0.74s system 99% cpu 1.721 total rg erzg4 k.json -j4 --no-mmap > /dev/null 0.99s user 0.75s system 99% cpu 1.752 total rg erzg4 k.json -j4 --no-mmap > /dev/null 0.98s user 0.76s system 99% cpu 1.748 total Sounds like "alias rg=rg --no-mmap" is a good idea on a Mac.
- burntsushi 10y ago
  Wow. Those are awesome results, thank you. > Sounds like "alias rg=rg --no-mmap" is a good idea on a Mac. I will fix that in ripgrep proper by making --no-mmap the default on Mac. :-) It should be an easy one to knock off: https://github.com/BurntSushi/ripgrep/issues/36 https://github.com/BurntSushi/ripgrep/issues/36 > not my first rodeo Right, sorry about that. :-) Just had to cover all my bases! (Also, `-j` on a single file won't do anything, and ripgrep should try to use multiple threads by default when searching multiple files.)