3 ms·
Can you provide a concrete example where that's faster? ripgrep should generally already be approximating `git ls-files` by respecting gitignore. Also, `-uu` t
by burntsushi 1y ago
Can you provide a concrete example where that's faster? ripgrep should generally already be approximating `git ls-files` by respecting gitignore.
Also, `-uu` tells ripgrep to not respect gitignore and to search hidden files. But ripgrep will still skip binary files. You need `-uuu` to also ignore binary files.
I tried playing with your `rgg` function. First problem occurred when I tried it on a checkout the Linux kernel:
$ rgg APM_RESUME
bash: /home/andrew/rust/ripgrep/target/release/rg: Argument list too long
OK, so let's just use `xargs`:
$ git ls-files -z | time xargs -0 rg APM_RESUME
arch/x86/kernel/apm_32.c
473: { APM_RESUME_DISABLED, "Resume timer disabled" },
include/uapi/linux/apm_bios.h
89:#define APM_RESUME_DISABLED 0x0d
real 0.638
user 0.741
sys 1.441
maxmem 29 MB
faults 0
And compared to just `rg APM_RESUME`:
$ time rg APM_RESUME
arch/x86/kernel/apm_32.c
473: { APM_RESUME_DISABLED, "Resume timer disabled" },
include/uapi/linux/apm_bios.h
89:#define APM_RESUME_DISABLED 0x0d
real 0.097
user 0.399
sys 0.588
maxmem 29 MB
faults 0
So do you have an example where `git ls-files -z | xargs -0 rg ...` is faster than just `rg ...`?
- oever 1y agoA checkout of my repository [0] with many pdf and audio files (20GB) is slow with -u. These data files are normally ignored because 1) they are in .gitignore and 2) they are binary. The repository contains CI files in .woodpecker. These are scripts that I'd normally expect to be searching in. Until a week ago I used -uu to do so, but that made rg take over 4 seconds for a search. Using -. brings the search time down to 24ms. git ls-files -z | time xargs -0 rg -w e23 40ms rg -w. e23 24ms rgg -w e23 16ms rg -wuu e23 2754ms To reproduce this with the given repository, fill it with 20GB of binary files. The -. flag makes this point moot though. [0] https://codeberg.org/vandenoever/rehorse https://codeberg.org/vandenoever/rehorse
- burntsushi 1y agoOh I see now. I now understand that you thought you couldn't convince ripgrep to search hidden files without also searching files typically ignored by gitignore. Thus, `git ls-files`. Yes, now it makes sense. And yes, `-./--hidden` makes it moot. Thanks for following up!
- EnPissant 1y agoI don't think this is the same thing as using gitignore. It will only search tracked files. For that it can just use the index. I would expect the index to be faster than looking at the fs for listings.
- burntsushi 1y agoI was extremely careful with my wording. Re-quoting, with added emphasis: > ripgrep should generally already be approximating `git ls-files` by respecting gitignore. See also: https://news.ycombinator.com/item?id=45629515 https://news.ycombinator.com/item?id=45629515
- EnPissant 1y agoI'm just trying to be helpful, not call you out. I've implemented gitignore aware file scanning before, and it was slower than git native operations when you only care about tracked files. It's the speed that is the part I was speaking too, not the semantics.
- burntsushi 1y agoYes, I agree, `git ls-files` can indeed be faster than just `rg --files`. On my Chromium checkout: $ hyperfine --output pipe 'rg --files' 'git ls-files' Benchmark 1: rg --files Time (mean ± σ): 141.2 ms ± 7.1 ms [User: 1134.5 ms, System: 376.3 ms] Range (min … max): 128.7 ms … 154.8 ms 20 runs Benchmark 2: git ls-files Time (mean ± σ): 54.9 ms ± 1.7 ms [User: 41.7 ms, System: 13.1 ms] Range (min … max): 52.2 ms … 62.0 ms 54 runs Summary git ls-files ran 2.57 ± 0.15 times faster than rg --files But the semantics here are important, because ripgrep doesn't only try to approximate `git ls-files`. It also needs to deal with `.rgignore` and `.ignore`. And no git repository state will help there. ripgrep also supports respecting `.gitignore` even when it isn't inside a git repository.