7 ms·

Just a small note that I would highgly recommend ripgrep[0] over standard grep. It's another modern tool that has been created by leveraging Rust and it's from

Just a small note that I would highgly recommend ripgrep[0] over standard grep. It's another modern tool that has been created by leveraging Rust and it's from BurntSushi[1] who is excellent. 0: https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep 1. https://github.com/BurntSushi https://github.com/BurntSushi

simias 7y ago
In general I think that's very good advice. In this particular instance however a dumb old grep might be superior because it could catch potential security vulnerabilities that are not explicitly hardcoded in the source code by greping through the compilation artifacts for instance. Sure you'll get a bunch of false positives that way but at least you know that nothing is slipping through the cracks.
- burntsushi 7y ago
  To be clear, you can disable all smart filtering in ripgrep. e.g., `rg -uuu foo` should be equivalent to `grep -r foo`. And if you want to exhaustively search binary files, then you need to add the `-a` flag to both commands.
pepper_sauce 7y ago
Why is it better than grep?
- Canada 7y ago
  It's way faster, which is great when you're working with big repos. It's designed for recursively searching through a lot of files.
  - masklinn 7y ago
    Grep is really fast at that at the actual search (gnu grep at least), the gain there is mostly that "smarter" tools will ignore e.g. VCS data or binary files by default whereas grep will trawl through your PNGs and git packfiles.
    - icebraining 7y ago
      Explanation from the original author on why GNU grep is fast: https://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html https://lists.freebsd.org/pipermail/freebsd-current/2010-Aug... Excerpt: "The result of this is that, in the limit, GNU grep averages fewer than 3 x86 instructions executed for each input byte it actually looks at (and it skips many bytes entirely)."
      - masklinn 7y ago
        There's also this bit: https://ridiculousfish.com/blog/posts/old-age-and-treachery.html https://ridiculousfish.com/blog/posts/old-age-and-treachery.... However note https://news.ycombinator.com/item?id=19522987 https://news.ycombinator.com/item?id=19522987 > It does not. ripgrep does not use Boyer-Moore in most searches. > In particular, the advice in [the freebsd mailing list post] is generally out of date. although the out of date bits are really the Boyer-Moore ones: https://lobste.rs/s/ycydmd https://lobste.rs/s/ycydmd > much of Mike Haertel’s advice in this post is still good. The bits about literal scanning, avoiding searching line-by-line, and paying attention to your input handling are on the money. > But the stuff about Boyer-Moore is outdated.
    - oconnor663 7y ago
      If I remember that giant post of benchmarks correctly, there are some big exceptions, particularly around non-ASCII searches.
  - Annatar 7y ago
    However fast it is, it's going to have a tough time beating /usr/bin/fgrep.
    - adrianN 7y ago
      fgrep -r takes about six times longer than rg on the repository that I'm currently working on.
      - Annatar 7y ago
        Real fgrep does not implement -r because that would be implementing tools within tools, which is against the UNIX®️ philosophy. Try /usr/bin/find . -depth -type f -print | /usr/bin/xargs -i /usr/bin/fgrep string '{}' and run it several times so that the filesystem cache is primed.
        adwn 7y ago
        > Real fgrep does not implement -r That's BS. The fgrep on my system – GNU grep 3.1 – provides recursive search (-r). What now, are you claiming that's not "real fgrep"? [1] > that would be implementing tools within tools, which is against the UNIX®️ philosophy Even more BS. Or are you telling me that "rm -r" is also against the "UNIX philosophy"? > /usr/bin/find . -depth -type f -print | /usr/bin/xargs -i /usr/bin/fgrep string '{}' Terrible, terrible advice. Cumbersome, error-prone, and slow as molasses. A quick test: Searching for 'asdfadsgf' in the Linux kernel repository takes 0.25 s using rg, 12 s using GNU fgrep -f, and 228 s (!) using your command. You know, when your ideology results in the worst results of all, you should really reconsider your ideology. [1] https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman
        robohoe 7y ago
        This is why Unix is great. It gives you enough tools to shoot yourself in the foot.
        7y ago
  - roryrjb 7y ago
    GNU grep is fast as well but by default it doesn't ignore anything. Granted it's handy to have this configured out of the box, but I prefer to know and use the flags and perhaps write a shell script wrapper, and I find that practically it feels just as fast as rg or others. My main point in doing this is to avoid the situation where I'm on a different machine to my laptop and can just get going right away without having to install anything. There's a trade off in all things, I just prefer it this way.
    - dev_dull 7y ago
      Portability is key. There’s a reason these gnu tools have such staying power. It’s not necessarily because they’re the best, but because they’re ubiquitous.
- K0nserv 7y ago
  Like a few other tools — ack, ag, and pt — it's specialized for source code, in addition to that it's really fast. The repo contains detailed comparisons with grep and an FAQ.
- seren 7y ago
  Blog post from the author with pros & cons of rg https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/ (It seems down at the moment you can try the cached version)
- deleted 7y ago
  [deleted]
Annatar 7y ago
This program re-invented find + egrep. It is written in Rust which isn't available on anything other than Linux, OS X and Windows. I'll just stick to /usr/bin/find and /usr/bin/egrep programs. That works just fine without having to port Rust before trying to build a re-invention of find and egrep.
- adrianN 7y ago
  It "reinvented" it and made it dramatically faster. Rust is also available on FreeBSD. I don't think you actually need Rust to run ripgrep. It's not like it's an interpreted language.
  - Annatar 7y ago
    Since when do FreeBSD executables run on the illumos family of operating systems, as far as I know, there is no freebsd-branded zone yet in illumos? Dramatically faster? Is there any scientific evidence to that? If it were me, I wouldn't rush to make assumptions.
    - adrianN 7y ago
      I tried it and it's much faster for me. The author did some benchmarks. I'm not about to publish a paper. I also never said anything about Illumos.
    - dmit 7y ago
      Illumos is also a supported platform (SPARC and x86-64).
      - Annatar 7y ago
        You obviously haven't tried it on either of those. They are "second tier", which means one is completely on one's own. What in your opinion would have to be the size of the source code to warrant jumping through the hoops to get this software running, as opposed to a combination of find + xargs + egrep,fgrep,awk?
_kbh_ 7y ago
rg + fzf really makes for a great toolset for some quick first pass code review.

simias 7y ago

In general I think that's very good advice. In this particular instance however a dumb old grep might be superior because it could catch potential security vulnerabilities that are not explicitly hardcoded in the source code by greping through the compilation artifacts for instance. Sure you'll get a bunch of false positives that way but at least you know that nothing is slipping through the cracks.

burntsushi 7y ago
To be clear, you can disable all smart filtering in ripgrep. e.g., `rg -uuu foo` should be equivalent to `grep -r foo`. And if you want to exhaustively search binary files, then you need to add the `-a` flag to both commands.

pepper_sauce 7y ago

Why is it better than grep?

Canada 7y ago
It's way faster, which is great when you're working with big repos. It's designed for recursively searching through a lot of files.
- masklinn 7y ago
  Grep is really fast at that at the actual search (gnu grep at least), the gain there is mostly that "smarter" tools will ignore e.g. VCS data or binary files by default whereas grep will trawl through your PNGs and git packfiles.
  - icebraining 7y ago
    Explanation from the original author on why GNU grep is fast: https://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html https://lists.freebsd.org/pipermail/freebsd-current/2010-Aug... Excerpt: "The result of this is that, in the limit, GNU grep averages fewer than 3 x86 instructions executed for each input byte it actually looks at (and it skips many bytes entirely)."
    - masklinn 7y ago
      There's also this bit: https://ridiculousfish.com/blog/posts/old-age-and-treachery.html https://ridiculousfish.com/blog/posts/old-age-and-treachery.... However note https://news.ycombinator.com/item?id=19522987 https://news.ycombinator.com/item?id=19522987 > It does not. ripgrep does not use Boyer-Moore in most searches. > In particular, the advice in [the freebsd mailing list post] is generally out of date. although the out of date bits are really the Boyer-Moore ones: https://lobste.rs/s/ycydmd https://lobste.rs/s/ycydmd > much of Mike Haertel’s advice in this post is still good. The bits about literal scanning, avoiding searching line-by-line, and paying attention to your input handling are on the money. > But the stuff about Boyer-Moore is outdated.
  - oconnor663 7y ago
    If I remember that giant post of benchmarks correctly, there are some big exceptions, particularly around non-ASCII searches.
- Annatar 7y ago
  However fast it is, it's going to have a tough time beating /usr/bin/fgrep.
  - adrianN 7y ago
    fgrep -r takes about six times longer than rg on the repository that I'm currently working on.
    - Annatar 7y ago
      Real fgrep does not implement -r because that would be implementing tools within tools, which is against the UNIX®️ philosophy. Try /usr/bin/find . -depth -type f -print | /usr/bin/xargs -i /usr/bin/fgrep string '{}' and run it several times so that the filesystem cache is primed.
      - adwn 7y ago
        > Real fgrep does not implement -r That's BS. The fgrep on my system – GNU grep 3.1 – provides recursive search (-r). What now, are you claiming that's not "real fgrep"? [1] > that would be implementing tools within tools, which is against the UNIX®️ philosophy Even more BS. Or are you telling me that "rm -r" is also against the "UNIX philosophy"? > /usr/bin/find . -depth -type f -print | /usr/bin/xargs -i /usr/bin/fgrep string '{}' Terrible, terrible advice. Cumbersome, error-prone, and slow as molasses. A quick test: Searching for 'asdfadsgf' in the Linux kernel repository takes 0.25 s using rg, 12 s using GNU fgrep -f, and 228 s (!) using your command. You know, when your ideology results in the worst results of all, you should really reconsider your ideology. [1] https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman
        robohoe 7y ago
        This is why Unix is great. It gives you enough tools to shoot yourself in the foot.
        7y ago
- roryrjb 7y ago
  GNU grep is fast as well but by default it doesn't ignore anything. Granted it's handy to have this configured out of the box, but I prefer to know and use the flags and perhaps write a shell script wrapper, and I find that practically it feels just as fast as rg or others. My main point in doing this is to avoid the situation where I'm on a different machine to my laptop and can just get going right away without having to install anything. There's a trade off in all things, I just prefer it this way.
  - dev_dull 7y ago
    Portability is key. There’s a reason these gnu tools have such staying power. It’s not necessarily because they’re the best, but because they’re ubiquitous.
K0nserv 7y ago
Like a few other tools — ack, ag, and pt — it's specialized for source code, in addition to that it's really fast. The repo contains detailed comparisons with grep and an FAQ.
seren 7y ago
Blog post from the author with pros & cons of rg https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/ (It seems down at the moment you can try the cached version)
deleted 7y ago
[deleted]

Annatar 7y ago

This program re-invented find + egrep. It is written in Rust which isn't available on anything other than Linux, OS X and Windows. I'll just stick to /usr/bin/find and /usr/bin/egrep programs. That works just fine without having to port Rust before trying to build a re-invention of find and egrep.

adrianN 7y ago
It "reinvented" it and made it dramatically faster. Rust is also available on FreeBSD. I don't think you actually need Rust to run ripgrep. It's not like it's an interpreted language.
- Annatar 7y ago
  Since when do FreeBSD executables run on the illumos family of operating systems, as far as I know, there is no freebsd-branded zone yet in illumos? Dramatically faster? Is there any scientific evidence to that? If it were me, I wouldn't rush to make assumptions.
  - adrianN 7y ago
    I tried it and it's much faster for me. The author did some benchmarks. I'm not about to publish a paper. I also never said anything about Illumos.
  - dmit 7y ago
    Illumos is also a supported platform (SPARC and x86-64).
    - Annatar 7y ago
      You obviously haven't tried it on either of those. They are "second tier", which means one is completely on one's own. What in your opinion would have to be the size of the source code to warrant jumping through the hoops to get this software running, as opposed to a combination of find + xargs + egrep,fgrep,awk?

_kbh_ 7y ago

rg + fzf really makes for a great toolset for some quick first pass code review.