5 ms·

When I use grep (which is fairly regularly), the bottleneck is nearly always the disk or the network (in case of NFS/SMB volumes). Just out of curiosity, what

• nolemurs 10y ago
  `grep` itself is admirably fast on single files, and if you're just searching a small number of files, the bottleneck will be IO, and `grep` will be about as fast as anything. But if you want to use `grep` recursively on a selected subset of millions of files it suffers a bit. `grep` assumes that if you want to collect some subset of files to search, you'll do that with some other utility, and pass the list of files to `grep`, but that creates a bottleneck. More recent search programs like `ack` and `ag` solve the multifile bottleneck issue, but also perform worse than `grep` on single large files.
  • krylon 10y ago
    Thank you!
  • burntsushi 10y ago
    > and `grep` will be about as fast as anything I'd encourage you to check out the benchmarks in my blog, especially the subtitle benchmarks, because this isn't actually true in the case of ripgrep. :-)
    • nolemurs 10y ago
      I take it back! I'm definitely switching over to using ripgrep for the next bit. Off topic - is there a way to exclude directories? I tried doing things like `rg pattern -g '!my_dir/' but nothing I tried seemed to work.
      • burntsushi 10y ago
        Other than using your .gitignore or .ignore files to ignore it, that is indeed the intended way to do it. Sadly, I have an embarrassing bug. Should be fixed soon: https://github.com/BurntSushi/ripgrep/issues/43 https://github.com/BurntSushi/ripgrep/issues/43
        • nolemurs 10y ago
          Ahh, fantastic. I just tried out the new build. It's a real joy to be able to do things like `rg pattern -g '!{docs,tests}/' -tpy` and have it just work. Honestly, all performance aside, your thoughtful choice of command line options is the big selling point for me. I struggle to find the command line flags I need to make `ag` do just what I want, but `rg` seems to fit my expectations without much training.
          • burntsushi 10y ago
            Thanks for your encouragement. It's really hard to stand my ground because so many people have so many different use cases. The design space is large and it's very challenging to get it even a little right. In any case, your example is quite fortuitous, since the `{docs,tests}` glob syntax was also added in 0.2.0. :-)
• burntsushi 10y ago
  > Just out of curiosity, what kind of use case makes grep and prospective replacements scream? Unicode? Check out the subtitle benchmarks in the blog post. In the best case, grep is a little slower. In the worst case, grep is orders of magnitude slower. ripgrep achieves speed by building UTF-8 decoding straight into its DFA regex engine (well, strictly speaking, this is Rust's regex engine, not ripgrep). The other case where grep users might scream is when you're searching large code repositories. A `grep -r` might catch a large binary file, or search your `.git` or whatever. Both `ag` and `rg` will look at your `.gitignore` so that the results you see have higher relevance. (Of course, this is just a default, you can always "search everything" with ripgrep too!)
  • bpchaps 10y ago
    Maybe I'm unique to this, but that sort of default would drive me batshit insane. Not all of us are programmers by trade who use git. It's just another gotcha to keep track of. I'd seriously recommend removing that as a default. That said, this is an overall awesome project.
    • burntsushi 10y ago
      The default isn't going to change. The Silver Searcher has, IMO, proven that it's a good default. I've spoken with so many people that love it, myself included. If you want to not respect .gitignores, it's easy: `alias rg="rg -u"`. (To be clear, I agree that the default is a trade off. I'm not so gung-ho on this that I think it should never be anything else. But for my project and its goals, I think it's the best fit.)
  • krylon 10y ago
    Thanks for the explanation!