6 ms·

I cannot understand all that amount of excitement. Are we under heavy fanbot attack or am I missing something? If the whole thing is that, is faster than grep a

I cannot understand all that amount of excitement. Are we under heavy fanbot attack or am I missing something? If the whole thing is that, is faster than grep and "in Rust we Trust" then please feel free to ignore my comment. I would use OSX's slow grep every single day to find things and I never had any of the non-problems problems which ripgrep is trying to solve. About recursive, case insensitive and search and file number just use grep with the flags "-Rni", so I don't get in which aspect is superior. I never ever had half a thought about grep's performance at the systems I use it (already faster than my cognitive bottle neck). If I ever would need something better than grep I would try to move to the next level or deal with lots of unstructured data I would perhaps move to a local search engine using indexes. And personally I would consider an antipattern to skip the gitignore contents per default. What kind of machines or use cases are hitting the people in the comments?

blux 8y ago
Use case: grepping (big) repos. Rg is great for that, and grep, because it is much slower, almost not usable.
- nudpiedo 8y ago
  maybe I did not had this problem yet... if I used grep on linux sources I would just skip the build folder and it usually works flawless (cannot even recall when was it too slow unless grepping over binary data). Do you refer maybe to deep interdependent javascript projects in which a small library requires hundreds of MB in dependencies? How many Gb have your projects and how many lines? Please include deps & builds (my current project is a small python one, 1.8Gb with deps, and I would usually grep only over the same folders all the time): $ time find ./ -type f -print|wc -l 140197 real 0m0.626s user 0m0.165s sys 0m0.473s Anyway, if it is about grepping over several gigabytes of unstructured code and there is no chance to have a better tool to index everything that could be a use case.
Shoop 8y ago
I use rg on a very large project (8.5 million LOC, 5.2G of files including the build). Searching with rg takes less than 2 seconds and I don't ever have to think about it. rg automatically handles not searching the build files and autogenerated files which often contain lots of matches but are never what I want to search. Additionally, rg has an excellent --vimgrep feature which makes it easy to integrate with the vim quickfix list. Vanilla grep, meanwhile takes 30 seconds -- too long to search while maintaining flow. For me, the whole thing _is_ that it is faster than grep. AFAIK, a lot of its speed is due to breaking compatibility with grep to allow for more convenient defaults (skip binary files by default, skip .gitignore by default, etc) which also happen to be faster. For me this is a clear win-win. Also, I have an 8 core machine. To be waiting for a 30s search knowing 7 cores are doing nothing makes me sad.
- nudpiedo 8y ago
  maybe you never bothered giving to grep as input parameters the same 2 or 3 folders which are usually scanned? Anyway, I ignored grep is not multicore and wouldn't expect CPU to be the main bottleneck. That's a good use case for a new tool. I might try the next time my greps take longer than 2 seconds. P.D. May I ask which kind of project/tech has this ratio of LOC and size?
  - Shoop 8y ago
    Some builds in this project are out-of-tree (the best practices cmake way with a separate build directory). Some are built in-tree (with json files, libraries, object files, LLVM ir files, and more scattered in the same directories as source code). I can't be bothered to spend the time to learn, write, debug, and maintain the correct grep invocation to do this. rg has saved me the 10 minutes it would take to do this, and 28 seconds * ~1000 grep invocations. I'm a very happy customer. The project is a fork of LLVM. We're implementing compiler support for a new design for threading on x86.
PudgePacket 8y ago
You're not wrong, it's a combination! Yes, faster than grep, yes "ooh shiny rust", + "ooh shiny" in general, used as the search function in the most popular editor (statistically, for users of stackoverflow) (https://insights.stackoverflow.com/survey/2019?utm_source=so-owned&utm_medium=blog&utm_campaign=dev-survey-2019&utm_content=launch-blog#development-environments-and-tools https://insights.stackoverflow.com/survey/2019?utm_source=so...). More "technical" reasons here: https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgrep https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgr.... For me, the performance is best in class, ux is great, and has defaults that make sense. Perhaps this helps add context. Every day there are probably thousands of new people learning to write software and learning and command line search tools. They would compare grep, and perhaps something like ripgrep on how they stack up today. They probably don't want to search their javascript dependencies by default (from gitignore, eg node_modules). They probably don't want to search their build (binary/gitignore). They would like to have an easy way to say "only search javascript files" or "only search go files". They would like the fastest tool for the job, why would you opt for the slower one? None of greps historical clout/muscle memory has an impact on them. They would probably appreciate coloured, formatted for easier human consumption output by default. Sorry for the rant!
ausjke 8y ago
for find/grep to catch up they really should take care of ignoring .git/.cvs/.svn/.repo directories as default these days, I don't want to type --exclude etc. I'm not that sensitive to speed here, but ignoring .git is an enough reason to make me use ripgred and fd daily instead of 'grep/ack' and 'find'.
oconnor663 8y ago
Apart from what other people have mentioned, proper UTF-8 support is a big deal for folks who might be searching non-English text.
rrosen326 8y ago
For me it's not about speed at all. I've used ack and loved it because it automatically ignores everything I want ignored (with a .ackrc file). I just tried ripgrep and like it even more because it automatically follows .gitignore, which is huge. To me, that's the killer feature.
burntsushi 8y ago
> I never had any of the non-problems problems which ripgrep is trying to solve That's OK. Not everyone has the same set of problems, and not every tool that is built must be used by everybody. I document pretty clearly what ripgrep does in the README, so if it doesn't target your use cases, then obviously you just don't need to use it. But also, just as obviously, it should be clear that plenty of other people do fit into the use cases that ripgrep targets. It turns out that a lot of people work on big code bases, and as a result, running the default grep command can be quite slow compared to something that is just a little bit smarter. It gets super annoying to always remember to type out the right `--include` commands to grep. I know, because I did it for ten years before I built ripgrep. I mean, this is the same premise that drives other tools like `git grep`, ack and ag. ripgrep isn't alone here. See also: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos... > And personally I would consider an antipattern to skip the gitignore contents per default. Then use alias rg="rg -uuu" and you'll never have to worry about that specific anti-pattern ever again. ;-)
- leo_santagada 8y ago
  Can I just say thank you ripgrep? In my usecase ripgrep is killer, I use it both to search through something around 10GB of a monorepo, and to search through 400GB of textfiles (custom format, similar to json) every day and not having rg performance would make my life considerably worse. When you have very large sets of data to search through the difference from rp to other solutions is like night and day.
  - burntsushi 8y ago
    Glad it helps. :) Although, with 400GB, I think any tool would probably get similar performance, since it's likely bottlenecked by your disk's read speed.
- nudpiedo 8y ago
  Thanks for answering burntsushi I didn't expect you to reply personally. It's true that there is some good value in having smart sane defaults, and skipping always looking for 2 or 3 advanced flags which are only used from time to time so I ended up giving it a chance (installing it via brew). The difference showed up immediately. At the moment first impressions are good, seems to be killing it at the most common grep use case I am having, so I will continue using it and trying for some weeks until I can have a better formed opinion. The only unknowns for me are are the edgy use cases such as piping to other processes where I might end up falling back to system's grep or maybe learn better the rg possibilities.
linsomniac 8y ago
I use "ack" rather than ripgrep, but the reason I changed was that it automatically ignores a lot of files that I don't care about: VCS files largely. That's the big feature for me, because I'm usually searching in Mercurial repos and every match appears twice (real file and .hg file).

blux 8y ago

Use case: grepping (big) repos. Rg is great for that, and grep, because it is much slower, almost not usable.

nudpiedo 8y ago
maybe I did not had this problem yet... if I used grep on linux sources I would just skip the build folder and it usually works flawless (cannot even recall when was it too slow unless grepping over binary data). Do you refer maybe to deep interdependent javascript projects in which a small library requires hundreds of MB in dependencies? How many Gb have your projects and how many lines? Please include deps & builds (my current project is a small python one, 1.8Gb with deps, and I would usually grep only over the same folders all the time): $ time find ./ -type f -print|wc -l 140197 real 0m0.626s user 0m0.165s sys 0m0.473s Anyway, if it is about grepping over several gigabytes of unstructured code and there is no chance to have a better tool to index everything that could be a use case.

Shoop 8y ago

I use rg on a very large project (8.5 million LOC, 5.2G of files including the build). Searching with rg takes less than 2 seconds and I don't ever have to think about it. rg automatically handles not searching the build files and autogenerated files which often contain lots of matches but are never what I want to search. Additionally, rg has an excellent --vimgrep feature which makes it easy to integrate with the vim quickfix list. Vanilla grep, meanwhile takes 30 seconds -- too long to search while maintaining flow. For me, the whole thing _is_ that it is faster than grep. AFAIK, a lot of its speed is due to breaking compatibility with grep to allow for more convenient defaults (skip binary files by default, skip .gitignore by default, etc) which also happen to be faster. For me this is a clear win-win. Also, I have an 8 core machine. To be waiting for a 30s search knowing 7 cores are doing nothing makes me sad.

nudpiedo 8y ago
maybe you never bothered giving to grep as input parameters the same 2 or 3 folders which are usually scanned? Anyway, I ignored grep is not multicore and wouldn't expect CPU to be the main bottleneck. That's a good use case for a new tool. I might try the next time my greps take longer than 2 seconds. P.D. May I ask which kind of project/tech has this ratio of LOC and size?
- Shoop 8y ago
  Some builds in this project are out-of-tree (the best practices cmake way with a separate build directory). Some are built in-tree (with json files, libraries, object files, LLVM ir files, and more scattered in the same directories as source code). I can't be bothered to spend the time to learn, write, debug, and maintain the correct grep invocation to do this. rg has saved me the 10 minutes it would take to do this, and 28 seconds * ~1000 grep invocations. I'm a very happy customer. The project is a fork of LLVM. We're implementing compiler support for a new design for threading on x86.

PudgePacket 8y ago

You're not wrong, it's a combination! Yes, faster than grep, yes "ooh shiny rust", + "ooh shiny" in general, used as the search function in the most popular editor (statistically, for users of stackoverflow) (https://insights.stackoverflow.com/survey/2019?utm_source=so-owned&utm_medium=blog&utm_campaign=dev-survey-2019&utm_content=launch-blog#development-environments-and-tools https://insights.stackoverflow.com/survey/2019?utm_source=so...). More "technical" reasons here: https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgrep https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgr.... For me, the performance is best in class, ux is great, and has defaults that make sense. Perhaps this helps add context. Every day there are probably thousands of new people learning to write software and learning and command line search tools. They would compare grep, and perhaps something like ripgrep on how they stack up today. They probably don't want to search their javascript dependencies by default (from gitignore, eg node_modules). They probably don't want to search their build (binary/gitignore). They would like to have an easy way to say "only search javascript files" or "only search go files". They would like the fastest tool for the job, why would you opt for the slower one? None of greps historical clout/muscle memory has an impact on them. They would probably appreciate coloured, formatted for easier human consumption output by default. Sorry for the rant!

ausjke 8y ago

for find/grep to catch up they really should take care of ignoring .git/.cvs/.svn/.repo directories as default these days, I don't want to type --exclude etc. I'm not that sensitive to speed here, but ignoring .git is an enough reason to make me use ripgred and fd daily instead of 'grep/ack' and 'find'.

oconnor663 8y ago

Apart from what other people have mentioned, proper UTF-8 support is a big deal for folks who might be searching non-English text.

rrosen326 8y ago

For me it's not about speed at all. I've used ack and loved it because it automatically ignores everything I want ignored (with a .ackrc file). I just tried ripgrep and like it even more because it automatically follows .gitignore, which is huge. To me, that's the killer feature.

burntsushi 8y ago

> I never had any of the non-problems problems which ripgrep is trying to solve That's OK. Not everyone has the same set of problems, and not every tool that is built must be used by everybody. I document pretty clearly what ripgrep does in the README, so if it doesn't target your use cases, then obviously you just don't need to use it. But also, just as obviously, it should be clear that plenty of other people do fit into the use cases that ripgrep targets. It turns out that a lot of people work on big code bases, and as a result, running the default grep command can be quite slow compared to something that is just a little bit smarter. It gets super annoying to always remember to type out the right `--include` commands to grep. I know, because I did it for ten years before I built ripgrep. I mean, this is the same premise that drives other tools like `git grep`, ack and ag. ripgrep isn't alone here. See also: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos... > And personally I would consider an antipattern to skip the gitignore contents per default. Then use alias rg="rg -uuu" and you'll never have to worry about that specific anti-pattern ever again. ;-)

leo_santagada 8y ago
Can I just say thank you ripgrep? In my usecase ripgrep is killer, I use it both to search through something around 10GB of a monorepo, and to search through 400GB of textfiles (custom format, similar to json) every day and not having rg performance would make my life considerably worse. When you have very large sets of data to search through the difference from rp to other solutions is like night and day.
- burntsushi 8y ago
  Glad it helps. :) Although, with 400GB, I think any tool would probably get similar performance, since it's likely bottlenecked by your disk's read speed.
nudpiedo 8y ago
Thanks for answering burntsushi I didn't expect you to reply personally. It's true that there is some good value in having smart sane defaults, and skipping always looking for 2 or 3 advanced flags which are only used from time to time so I ended up giving it a chance (installing it via brew). The difference showed up immediately. At the moment first impressions are good, seems to be killing it at the most common grep use case I am having, so I will continue using it and trying for some weeks until I can have a better formed opinion. The only unknowns for me are are the edgy use cases such as piping to other processes where I might end up falling back to system's grep or maybe learn better the rg possibilities.

linsomniac 8y ago

I use "ack" rather than ripgrep, but the reason I changed was that it automatically ignores a lot of files that I don't care about: VCS files largely. That's the big feature for me, because I'm usually searching in Mercurial repos and every match appears twice (real file and .hg file).