3 ms·

Never managed to switch to ripgrep over ag. Performance difference is not that much of a dealbreaker in practice, and rg is not totally backwards-compatible to

by krick 7y ago

Never managed to switch to ripgrep over ag. Performance difference is not that much of a dealbreaker in practice, and rg is not totally backwards-compatible to ag. I don't exactly remember, what was the problem. I guess it has this annoying practice to ignore everything that's in .gitignore, while for ag I could set up an additional .agignore per-project.

OJFord 7y ago
Rg similarly supports it's own ignore file, and optionally disabling the default of respecting .gitignore.
burntsushi 7y ago
For ripgrep, if you want to use PCRE2 regexes (assuming that's what you meant by backwards compatible), then the -P flag will do that. Otherwise, ripgrep's support for .gitignore should be substantially better than what's in ag. You can use .rgignore just like .agignore. Both ripgrep and ag also support .ignore. These can override whatever is in your .gitignore. See the guide for more details: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#automatic-filtering https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#a... > Performance difference is not that much of a dealbreaker in practice This is true for smallish inputs. But the difference should become larger for bigger inputs. Hell, sometimes ag just refuses to search inputs that are too big. e.g., For a ~9.5GB file: $ time rg '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c 3003 real 1.608 user 1.105 sys 0.496 maxmem 9473 MB faults 0 $ time ag '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c ERR: Skipping OpenSubtitles2016.raw.en: pcre_exec() can't handle files larger than 2147483647 bytes. real 0.016 user 0.004 sys 0.011 maxmem 6 MB faults 0
- krick 7y ago
  Thank you for your answer. What I mean, it doesn't seem to be possible to make it forget about .gitignore files completely. For me, it doesn't make any sense for my grep tool to pay any attention to .gitignore files (let alone info/exclude and such), these are 2 completely separate tools. I might occasionally want for some project to put into .rgignore the same line as I put in .gitignore, but that's an explicit choice. Actually, most of the time I have a few global ignores (which is possible to do with ripgrep) and an occasional .agignore file, while pretty much every git directory has .gitignore of some sorts. This is not something I can get around with, since while .agignore (.rgignore) is totally personal, .gitignore affects everyone who works on the project and serves explicitly to indicate that you ought not commit these files. If I put "--no-ignore" in my .ripgreprc, it skips both .gitignore and .rgignore files, which isn't obviously what I'm trying to achieve. One more thing (even thought it it possible to get around it by writing my own bash function that would invoke rg) is that unlike the ag, rg doesn't seem have "--pager" option, which I always use heavily, by setting half a dozen of parameters for `less` when grepping. So these are what I consider a deal breakers and reasons why I didn't switch to rg: while it's true that in some cases I would appreciate a better performance, it's extremely rare that I grep something over a 9.5GB file. And conveniently search for a substring in a project is something that I'm in need multiple times a day. BTW, noticed one more minor quirk right now: export RIPGREP_CONFIG_PATH="~/.ripgreprc" doesn't work (but RIPGREP_CONFIG_PATH="/home/$USER/.ripgreprc" does, so it's probably almost never an issue).
  - burntsushi 7y ago
    You want --no-ignore-vcs. There are several other --no-ignore-* flags. ripgrep is a lot more flexible than ag there. And its implementation of the gitignore matching rules should work a lot better than ag. (Search ag's issue tracker to see just how many bugs there are. I'd be surprised if you weren't hitting any of them!) As for a pager, that's pretty easy to solve: `rg foo | less -F`. Add `-R` to make colors work. You don't need to search a 9.5GB file to make ag upset. All it takes is ~2GB. A key advantage of ripgrep over ag is that you don't have to just use it for code searching. It actually works just as well any place you'd use standard grep. ag just doesn't have an implementation that's good enough to be that robust.
    - krick 7y ago
      > --no-ignore-vcs Oh, thanks, I've missed it somehow. > rg foo | less -F Yeah, that's what I meant by writing my own wrapper, since I wouldn't want to type that in every time I search for something. And since I use bash, $@ in the middle of an alias is not possible, so: rg() { /usr/bin/rg $@ | less -LFXRqn } It doesn't seem to be working as expected for me without explicitly adding --pretty though (obviously, rg sees it doesn't operate in a terminal and tries to be smart). End even then it's... weird. I don't know if that's `less` issue of `rg` issue, I'm guessing something about it being multi-threaded: it sometimes doesn't display some found items at the top of the terminal, but if I scroll screen down and up again -- it's there! Cannot reproduce with ag. Gonna play with this for some more time later, but it surely doesn't work as solidly with a pager, as ag --pager='whatever' does. EDIT: actually found a bug report that seems to describe exactly the same problem. https://github.com/BurntSushi/ripgrep/issues/513 https://github.com/BurntSushi/ripgrep/issues/513
      - burntsushi 7y ago
        Interesting. I can't reproduce that problem. Piping to less works perfectly fine on both my Linux and macOS machines. If you're willing, I'd definitely appreciate it if you could update #513 with your environment. In particular, by answering these questions: https://github.com/BurntSushi/ripgrep/issues/513#issuecomment-308542044 https://github.com/BurntSushi/ripgrep/issues/513#issuecommen... If it's convenient and you could try it in a different environment (perhaps a different terminal emulator or a different shell), then that would be great. Also, this doesn't sound like a multi-threading issue to me, but if you want to rule that out, then `-j1` will force ripgrep into single threaded mode.

by krick 7y ago

OJFord 7y ago

Rg similarly supports it's own ignore file, and optionally disabling the default of respecting .gitignore.

burntsushi 7y ago

For ripgrep, if you want to use PCRE2 regexes (assuming that's what you meant by backwards compatible), then the -P flag will do that. Otherwise, ripgrep's support for .gitignore should be substantially better than what's in ag. You can use .rgignore just like .agignore. Both ripgrep and ag also support .ignore. These can override whatever is in your .gitignore. See the guide for more details: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#automatic-filtering https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#a... > Performance difference is not that much of a dealbreaker in practice This is true for smallish inputs. But the difference should become larger for bigger inputs. Hell, sometimes ag just refuses to search inputs that are too big. e.g., For a ~9.5GB file: $ time rg '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c 3003 real 1.608 user 1.105 sys 0.496 maxmem 9473 MB faults 0 $ time ag '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c ERR: Skipping OpenSubtitles2016.raw.en: pcre_exec() can't handle files larger than 2147483647 bytes. real 0.016 user 0.004 sys 0.011 maxmem 6 MB faults 0

krick 7y ago
Thank you for your answer. What I mean, it doesn't seem to be possible to make it forget about .gitignore files completely. For me, it doesn't make any sense for my grep tool to pay any attention to .gitignore files (let alone info/exclude and such), these are 2 completely separate tools. I might occasionally want for some project to put into .rgignore the same line as I put in .gitignore, but that's an explicit choice. Actually, most of the time I have a few global ignores (which is possible to do with ripgrep) and an occasional .agignore file, while pretty much every git directory has .gitignore of some sorts. This is not something I can get around with, since while .agignore (.rgignore) is totally personal, .gitignore affects everyone who works on the project and serves explicitly to indicate that you ought not commit these files. If I put "--no-ignore" in my .ripgreprc, it skips both .gitignore and .rgignore files, which isn't obviously what I'm trying to achieve. One more thing (even thought it it possible to get around it by writing my own bash function that would invoke rg) is that unlike the ag, rg doesn't seem have "--pager" option, which I always use heavily, by setting half a dozen of parameters for `less` when grepping. So these are what I consider a deal breakers and reasons why I didn't switch to rg: while it's true that in some cases I would appreciate a better performance, it's extremely rare that I grep something over a 9.5GB file. And conveniently search for a substring in a project is something that I'm in need multiple times a day. BTW, noticed one more minor quirk right now: export RIPGREP_CONFIG_PATH="~/.ripgreprc" doesn't work (but RIPGREP_CONFIG_PATH="/home/$USER/.ripgreprc" does, so it's probably almost never an issue).
- burntsushi 7y ago
  You want --no-ignore-vcs. There are several other --no-ignore-* flags. ripgrep is a lot more flexible than ag there. And its implementation of the gitignore matching rules should work a lot better than ag. (Search ag's issue tracker to see just how many bugs there are. I'd be surprised if you weren't hitting any of them!) As for a pager, that's pretty easy to solve: `rg foo | less -F`. Add `-R` to make colors work. You don't need to search a 9.5GB file to make ag upset. All it takes is ~2GB. A key advantage of ripgrep over ag is that you don't have to just use it for code searching. It actually works just as well any place you'd use standard grep. ag just doesn't have an implementation that's good enough to be that robust.
  - krick 7y ago
    > --no-ignore-vcs Oh, thanks, I've missed it somehow. > rg foo | less -F Yeah, that's what I meant by writing my own wrapper, since I wouldn't want to type that in every time I search for something. And since I use bash, $@ in the middle of an alias is not possible, so: rg() { /usr/bin/rg $@ | less -LFXRqn } It doesn't seem to be working as expected for me without explicitly adding --pretty though (obviously, rg sees it doesn't operate in a terminal and tries to be smart). End even then it's... weird. I don't know if that's `less` issue of `rg` issue, I'm guessing something about it being multi-threaded: it sometimes doesn't display some found items at the top of the terminal, but if I scroll screen down and up again -- it's there! Cannot reproduce with ag. Gonna play with this for some more time later, but it surely doesn't work as solidly with a pager, as ag --pager='whatever' does. EDIT: actually found a bug report that seems to describe exactly the same problem. https://github.com/BurntSushi/ripgrep/issues/513 https://github.com/BurntSushi/ripgrep/issues/513
    - burntsushi 7y ago
      Interesting. I can't reproduce that problem. Piping to less works perfectly fine on both my Linux and macOS machines. If you're willing, I'd definitely appreciate it if you could update #513 with your environment. In particular, by answering these questions: https://github.com/BurntSushi/ripgrep/issues/513#issuecomment-308542044 https://github.com/BurntSushi/ripgrep/issues/513#issuecommen... If it's convenient and you could try it in a different environment (perhaps a different terminal emulator or a different shell), then that would be great. Also, this doesn't sound like a multi-threading issue to me, but if you want to rule that out, then `-j1` will force ripgrep into single threaded mode.