5 ms·

weird right? I don't know what ripgrep has, maybe it's just the name? the fact that you call `ag` but it's called `the_silver_searcher`?

by rscnt 5y ago

pdimitar 5y ago
From my side, I knew about both `ripgrep` and `the_silver_searcher` but I will openly admit I've lost faith in C and C++'s abilities to protect against memory safety errors. Thus, Rust tools get a priority, at least for myself. There are probably other memory-safe languages but I haven't had the chance to give them a spin like I did with Rust. If I found out about them then I'll also prefer tools written in them if there's no Rust port and if the alternatives are C/C++ tools.
- chx 5y ago
  This is what I suspected: Rust zealotry. We are searching for a string. Language matters to the guy writing it but the user? Nah. There's no security hole here.
  - pdimitar 5y ago
    Who's the zealot here? The guy who doesn't want to risk and openly states so, or the guy proclaiming there's no risk, even with ample historical evidence for the opposite? I don't accept your labeling, especially when it's so egregiously misguided. Also, you're clearly seeing what you want to see.
- jhoechtl 5y ago
  Ag is shorthand for argentum which in turn means silver. https://en.wikipedia.org/wiki/Silver https://en.wikipedia.org/wiki/Silver
- saagarjha 5y ago
  Code search on your own codebase is generally not a place where memory safety shines.
  - pdimitar 5y ago
    When it's about security, I try not to pick and choose. Plus a memory error might lead to scary consequences (like a random script getting elevated privileges). Finally, and last I checked, `rg` is quite featureful and I've never felt constrained by it. So for me it's a win all around.
    - saagarjha 5y ago
      This is kind of my point: security requires a threat model, and you don't really have one. Rust has a lot going for it, and it does hold promise in improving the security of a lot of critical software. But in this case, it's not really doing that, so it's kind of misleading to say it is meaningfully doing anything for security.
      - pdimitar 5y ago
        I agree that in the case of grep-ing in the terminal the odds of covering your butt well enough by using a Rust tool are super slim. That being said, there are powerful adversaries of anonymity and the right to personal data out there -- and security in depth is what works best against them. There's no one UltimateSecuritySolution™; there are many small ones that we layer on top of each other so we don't allow even a smidgen of air to pass between the cracks. But yeah, I am paranoid. I am gradually preparing myself to move from macOS to Linux and even though I am not a criminal and never will be, I'll still make a heroic effort to make the odds of any foul play against me practically zero. (And that's why I will start using the userland Rust tools alternatives as well.) I'll concede that in my case the biggest impact would probably come from running Chrome in a jail, and not from using `rg` vs. `ag`. That much is true, yep.
        saagarjha 5y ago
        Yeah, it's kind of a shame in this case. There's tons to talk about in the area of where Rust shines here ("makes concurrency easy", "provides easy access to fast algorithm libraries", etc.) but security is just not really one of those points.
timerol 5y ago
ripgrep is noticeably faster than ag. I'm not sure what features you mention that ripgrep is missing, but I've been plenty happy with it for basic grepping around. I'm sure it's also partly because BurntSushi, the author of ripgrep, is reasonably active here. Some fun benchmarks, and a fair description of how text search works, can be found in this 2016 post https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
- chx 5y ago
  > ripgrep is noticeably faster than ag. Since both are instant on every codebase I care about, not so sure about that. These tools were always chiefly I/O bound , SSDs have torn that down especially now. But for features: there's no lookaround in the regex.
  - aidenn0 5y ago
    These tools are I/o bound, but not disk bound. I often perform multiple searches in a row, and all but the first will hit the disk cache.
  - burntsushi 5y ago
    They are only I/O bound when searching data that isn't in cache. It's often the case that you're searching, for example, a repository of code repeatedly. > Since both are instant on every codebase I care about, not so sure about that. It is very possible that there is no performance difference between ripgrep and ag for your use cases, but that does not mean there isn't a performance difference between ripgrep and ag. For example, in my checkout of the chromium repository: $ time rg -c Openbox testing/xvfb.py:2 tools/metrics/histograms/enums.xml:1 ui/base/x/x11_util.cc:1 real 0.448 user 2.593 sys 2.490 maxmem 77 MB faults 0 $ time ag -c Openbox ui/base/x/x11_util.cc:1 tools/metrics/histograms/enums.xml:1 testing/xvfb.py:2 real 2.302 user 2.996 sys 10.462 maxmem 15 MB faults 0 > But for features: there's no lookaround in the regex. That's not true and hasn't been true for a long time. ripgrep supports PCRE2 with the -P/--pcre2 flag. You can even put `--engine auto` in an alias or ripgreprc file and have ripgrep automatically select the regex engine based on whether you're using "fancy" features or not. In general, I also claim that ripgrep has far fewer bugs than ag. ag doesn't really get gitignore support correct, although if you only have simple gitignores, its support might be good enough.
burntsushi 5y ago
It's not weird because ripgrep has had more features than ag for a long time. Originally ripgrep didn't have multi-line support or support for fancy regex features like look-around, but it has both of those now. It also has support for automatic UTF-16 transcoding, preprocessors for searching non-text files and overall less buggy support for gitignore. (Look at ag's issue tracker.) And then there's also the fact that ag isn't that great as a general purpose grep tool. It really falls over in terms of speed when searching, say, bigger files: $ time rg 'Sherlock Holmes' OpenSubtitles2018.raw.en | wc -l 7673 real 1.475 user 1.115 sys 0.356 maxmem 12511 MB faults 0 $ time ag 'Sherlock Holmes' OpenSubtitles2018.raw.en | wc -l 7673 real 20.276 user 19.850 sys 0.413 maxmem 12508 MB faults 0 A lot of people like to comment and say, "well ag is fast enough for me." Well, OK, that's fine. But if you're wondering about why other people might mention ripgrep more, well, maybe it isn't just about the way you use the tools. For example, if you only ever search tiny repositories of code, then you aren't going to care that ripgrep is faster than ag. Which is perfectly reasonable, but it should also be reasonable to be aware that others might actually search bigger corpora than you.
- chx 5y ago
  ripgrep regexp has lookaround? The linked docs still doesn't say that. https://docs.rs/regex/1.5.4/regex/#syntax https://docs.rs/regex/1.5.4/regex/#syntax > This crate provides a library for parsing, compiling, and executing regular expressions. Its syntax is similar to Perl-style regular expressions, but lacks a few features like look around and backreferences. Edit: I see. It can use pcre2 but it's a build time option which of course Ubuntu has off.
  - burntsushi 5y ago
    > Edit: I see. It can use pcre2 but it's a build time option which of course Ubuntu has off. That's unfortunate. Perhaps a bug report to the packagers is in order? Archlinux enabled it: https://github.com/archlinux/svntogit-community/blob/0dc033f5d0a476671dc0971a02eb9baa51cabc73/trunk/PKGBUILD#L26 https://github.com/archlinux/svntogit-community/blob/0dc033f...

weird right? I don't know what ripgrep has, maybe it's just the name? the fact that you call `ag` but it's called `the_silver_searcher`?

by rscnt 5y ago

weird right? I don't know what ripgrep has, maybe it's just the name? the fact that you call `ag` but it's called `the_silver_searcher`?

pdimitar 5y ago

From my side, I knew about both `ripgrep` and `the_silver_searcher` but I will openly admit I've lost faith in C and C++'s abilities to protect against memory safety errors. Thus, Rust tools get a priority, at least for myself. There are probably other memory-safe languages but I haven't had the chance to give them a spin like I did with Rust. If I found out about them then I'll also prefer tools written in them if there's no Rust port and if the alternatives are C/C++ tools.

chx 5y ago
This is what I suspected: Rust zealotry. We are searching for a string. Language matters to the guy writing it but the user? Nah. There's no security hole here.
- pdimitar 5y ago
  Who's the zealot here? The guy who doesn't want to risk and openly states so, or the guy proclaiming there's no risk, even with ample historical evidence for the opposite? I don't accept your labeling, especially when it's so egregiously misguided. Also, you're clearly seeing what you want to see.
jhoechtl 5y ago
Ag is shorthand for argentum which in turn means silver. https://en.wikipedia.org/wiki/Silver https://en.wikipedia.org/wiki/Silver
saagarjha 5y ago
Code search on your own codebase is generally not a place where memory safety shines.
- pdimitar 5y ago
  When it's about security, I try not to pick and choose. Plus a memory error might lead to scary consequences (like a random script getting elevated privileges). Finally, and last I checked, `rg` is quite featureful and I've never felt constrained by it. So for me it's a win all around.
  - saagarjha 5y ago
    This is kind of my point: security requires a threat model, and you don't really have one. Rust has a lot going for it, and it does hold promise in improving the security of a lot of critical software. But in this case, it's not really doing that, so it's kind of misleading to say it is meaningfully doing anything for security.
    - pdimitar 5y ago
      I agree that in the case of grep-ing in the terminal the odds of covering your butt well enough by using a Rust tool are super slim. That being said, there are powerful adversaries of anonymity and the right to personal data out there -- and security in depth is what works best against them. There's no one UltimateSecuritySolution™; there are many small ones that we layer on top of each other so we don't allow even a smidgen of air to pass between the cracks. But yeah, I am paranoid. I am gradually preparing myself to move from macOS to Linux and even though I am not a criminal and never will be, I'll still make a heroic effort to make the odds of any foul play against me practically zero. (And that's why I will start using the userland Rust tools alternatives as well.) I'll concede that in my case the biggest impact would probably come from running Chrome in a jail, and not from using `rg` vs. `ag`. That much is true, yep.
      - saagarjha 5y ago
        Yeah, it's kind of a shame in this case. There's tons to talk about in the area of where Rust shines here ("makes concurrency easy", "provides easy access to fast algorithm libraries", etc.) but security is just not really one of those points.

timerol 5y ago

ripgrep is noticeably faster than ag. I'm not sure what features you mention that ripgrep is missing, but I've been plenty happy with it for basic grepping around. I'm sure it's also partly because BurntSushi, the author of ripgrep, is reasonably active here. Some fun benchmarks, and a fair description of how text search works, can be found in this 2016 post https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/

chx 5y ago
> ripgrep is noticeably faster than ag. Since both are instant on every codebase I care about, not so sure about that. These tools were always chiefly I/O bound , SSDs have torn that down especially now. But for features: there's no lookaround in the regex.
- aidenn0 5y ago
  These tools are I/o bound, but not disk bound. I often perform multiple searches in a row, and all but the first will hit the disk cache.
- burntsushi 5y ago
  They are only I/O bound when searching data that isn't in cache. It's often the case that you're searching, for example, a repository of code repeatedly. > Since both are instant on every codebase I care about, not so sure about that. It is very possible that there is no performance difference between ripgrep and ag for your use cases, but that does not mean there isn't a performance difference between ripgrep and ag. For example, in my checkout of the chromium repository: $ time rg -c Openbox testing/xvfb.py:2 tools/metrics/histograms/enums.xml:1 ui/base/x/x11_util.cc:1 real 0.448 user 2.593 sys 2.490 maxmem 77 MB faults 0 $ time ag -c Openbox ui/base/x/x11_util.cc:1 tools/metrics/histograms/enums.xml:1 testing/xvfb.py:2 real 2.302 user 2.996 sys 10.462 maxmem 15 MB faults 0 > But for features: there's no lookaround in the regex. That's not true and hasn't been true for a long time. ripgrep supports PCRE2 with the -P/--pcre2 flag. You can even put `--engine auto` in an alias or ripgreprc file and have ripgrep automatically select the regex engine based on whether you're using "fancy" features or not. In general, I also claim that ripgrep has far fewer bugs than ag. ag doesn't really get gitignore support correct, although if you only have simple gitignores, its support might be good enough.

burntsushi 5y ago

It's not weird because ripgrep has had more features than ag for a long time. Originally ripgrep didn't have multi-line support or support for fancy regex features like look-around, but it has both of those now. It also has support for automatic UTF-16 transcoding, preprocessors for searching non-text files and overall less buggy support for gitignore. (Look at ag's issue tracker.) And then there's also the fact that ag isn't that great as a general purpose grep tool. It really falls over in terms of speed when searching, say, bigger files: $ time rg 'Sherlock Holmes' OpenSubtitles2018.raw.en | wc -l 7673 real 1.475 user 1.115 sys 0.356 maxmem 12511 MB faults 0 $ time ag 'Sherlock Holmes' OpenSubtitles2018.raw.en | wc -l 7673 real 20.276 user 19.850 sys 0.413 maxmem 12508 MB faults 0 A lot of people like to comment and say, "well ag is fast enough for me." Well, OK, that's fine. But if you're wondering about why other people might mention ripgrep more, well, maybe it isn't just about the way you use the tools. For example, if you only ever search tiny repositories of code, then you aren't going to care that ripgrep is faster than ag. Which is perfectly reasonable, but it should also be reasonable to be aware that others might actually search bigger corpora than you.

chx 5y ago
ripgrep regexp has lookaround? The linked docs still doesn't say that. https://docs.rs/regex/1.5.4/regex/#syntax https://docs.rs/regex/1.5.4/regex/#syntax > This crate provides a library for parsing, compiling, and executing regular expressions. Its syntax is similar to Perl-style regular expressions, but lacks a few features like look around and backreferences. Edit: I see. It can use pcre2 but it's a build time option which of course Ubuntu has off.
- burntsushi 5y ago
  > Edit: I see. It can use pcre2 but it's a build time option which of course Ubuntu has off. That's unfortunate. Perhaps a bug report to the packagers is in order? Archlinux enabled it: https://github.com/archlinux/svntogit-community/blob/0dc033f5d0a476671dc0971a02eb9baa51cabc73/trunk/PKGBUILD#L26 https://github.com/archlinux/svntogit-community/blob/0dc033f...