34 ms·

Ripgrep 13.0

547 points by mfrw 5y ago

mfrw 5y ago
Highlights: A new short flag, -., has been added. It is an alias for the --hidden flag, which instructs ripgrep to search hidden files and directories. BREAKING CHANGES: Binary detection output has changed slightly.
- rektide 5y ago
  loving '-.'. I use --hidden a lot. also noteable, an underlying lib got some vectorization speed improvements.
- hawski 5y ago
  Lack of this alias was one of the things that put me off regarding ripgrep. I'm using the silver searcher instead as it is plenty fast and it's CLI makes sense for me and doesn't change, AFAIR it's even mostly the same as ack, which I used before.
flakiness 5y ago
Haven't tried ripgrep before but finally installed. It is noticeably faster than older tools and more importantly the CLI ergonomics is much better! Thanks!
- mfrw 5y ago
  on a tangential note: there is a bunch of tools re-written in rust, a handy non exhaustive list [1]. [1]: https://zaiste.net/posts/shell-commands-rust/ https://zaiste.net/posts/shell-commands-rust/
  - ibraheemdev 5y ago
    Here's a hopefully more exhaustive list that isn't restricted to Rust: https://github.com/ibraheemdev/modern-unix https://github.com/ibraheemdev/modern-unix, meaning it can include other awesome tools like fzf (written in Go).
    - mgradowski 5y ago
      If you like fzf, give skim [1] a go. It's basically faster (anecdotally) fzf, but can also be used as a Rust library. I made a little personal to-do manager using it, together with SQLite for persistence. I also improved my fish shell ergonomics with [2]. [1] https://github.com/lotabout/skim https://github.com/lotabout/skim [2] https://github.com/zenofile/fish-skim https://github.com/zenofile/fish-skim
    - mfrw 5y ago
      Thank you for this compilation. I think the demos are a cherry on top; it becomes an easy sell :) Thank you :)
  - tspiteri 5y ago
    I love Rust, but I love and recommend ripgrep because of its usefulness, reliability and robustness, not because of the language it's written in. I haven't looked into all the tools in that list, but I would not for example recommend exa over ls as it is simply not reliable enough: if a filename ends in a space, you won't see that in exa, and that bug has been reported for years. To me that is a clear blocker, and if it is still there, I simply cannot trust the file listing from exa, no matter how pretty it may look.
    - duped 5y ago
      Static linking goes a long way to making Rust worth considering when choosing tools. Anecdote: I once had to recover a system with a corrupted libpcre.so. This will break almost every standard gnutil. The easiest way to do it without a recovery OS was to use a few alternatives written in Rust, which don't have this problem because they statically link their dependencies (and cargo still worked, so it was easy to install them).
      - saagarjha 5y ago
        Your anecdote is amusing, but I hope you’re not using it as an example to support your claim that Rust’s static linking should be considered when choosing tools.
- mekster 5y ago
  Wasn't ripgrep one of the first inspirations for others to initiate "rewrite existing tools with rust with better usability and performance" wave? Either way, thanks for a quality tool, more, if it inspired others to come up with good rust tools.
  - burntsushi 5y ago
    Maybe in Rust. xsv does predate ripgrep though. Maybe ripgrep was the first one to get really popular though. To be honest, I dunno. With that said, Go tools were doing this before Rust even hit 1.0 as far as I remember. There was 'sift' and 'pt' for "smarter" greps for example, although I don't think either of them ever fully matured.
simonw 5y ago
ripgrep is absolutely fantastic software. If you're not using it yet I guarantee it's worth taking the time to install and learn basic usage. On macOS: "brew install ripgrep" - then use "rg searchterm" to search all nested files in the current directory that contain searchterm. You may be using it without realizing already: the code search feature in VS Code uses ripgrep under the hood.
- cute_boi 5y ago
  actually most of the time you might want to use rg -i searchterm . I don't know why its not default? May be they want to make it similar to grep but with perf?
  - simonw 5y ago
    -i is ignore-case, right? Usually I'm searching code for things like variable names, so actually case-sensitive searches are a better default for me.
  - raziel2p 5y ago
    --smart-case is worth looking into as well. you can also put whatever arguments you want in your ripgreprc file to tweak the "defaults" to your liking.
  - bostonvaulter2 5y ago
    I think that's a bit of a personal choice. I definitely prefer the current default of an exact match including case. Partially because I'm very familiar with the common `-i` flag to ignore case which is used in `grep`, `git grep`, `ack`, `ag` and probably others. But I've never used a flag to make a search case-sensitive. Plus it's easy enough to alias `rg` to `rg -i` if you prefer that as the default.
  - Hamuko 5y ago
    I almost never use -i. I may do rg "[Dd]efault" if I want to search for "default" or "Default", but not "DEFAULT". Usually I just want to search in a very specific case since I'm looking for variables.
jayflux 5y ago
Shoutout to Burntsushi He knows how to keep a good changelog https://github.com/BurntSushi/ripgrep/blob/master/CHANGELOG.md https://github.com/BurntSushi/ripgrep/blob/master/CHANGELOG.... it’s good to read through and see the progress. He also cares about ergonomics as well as performance which I like. I find it much more easier to use than existing Grep tools. Interestingly, I believe Ripgrep started off as a test harness to benchmark the Rust Regex crate (he was the author of that). It was originally not intended to be a consumer cli. I guess he dogfooded it that well it became a successful tool
- burntsushi 5y ago
  Thanks. :-) That's right. IIRC, I wrote it as a simple harness because I wanted to check whether someone could feasibly write a grep tool using the regex crate and get similar performance. Once I got the harness up and running though, I realized it was not only competitive with GNU grep in a larger number of cases than I thought it would be (after some perf bug fixes, IIRC), but was substantially faster than ag. So that's when I decided to polish it up into a tool for others to use.
  - e12e 5y ago
    Do you use any tooling for the Changelog/issue tagging (PERF, BUG) etc?
    - burntsushi 5y ago
      Nope. I just try to keep it updated as I go. But in practice, I usually forget to update it and it just ends up becoming part of the release process.[1] [1] - https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CHECKLIST.md https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CH...
  - vvanpo 5y ago
    I recently switched to `rg` after many years of using `ag`, and the fact that it properly handles `.gitignore` syntax makes my life much better.
  - 5y ago
oweiler 5y ago
The best feature about ripgrep (apart from its great performance): Sane defaults out of the box!
- valtism 5y ago
  This can be an underrated feature of software, although I feel like we're getting better at it. I remember being so frustrated with guides to using software back in the early 00s where part of the guide would be reconfiguring default settings. It always seemed to be a sign of sub-par quality software.
coffeeri 5y ago
Also consider installing ripgrep-all [1], an extension making it possible to search through PDFs, eBooks, ZIPs and other documents. The integration with fzf is also fantastic! [1] https://github.com/phiresky/ripgrep-all https://github.com/phiresky/ripgrep-all
- O_H_E 5y ago
  Aaaaa, yes yes yes. I need that in my life.
- joshspankit 5y ago
  :O
- allenleein 5y ago
  I have been using ripgrep-all since very beginning. Can't live without it!
- nyanpasu64 5y ago
  `cargo install ripgrep-all` fails to compile due to a missing cachedir dependency, and the author won't make a new release out of master due to a possible deadlocking issue. This fails to inspire confidence: https://github.com/phiresky/ripgrep-all/issues/88 https://github.com/phiresky/ripgrep-all/issues/88
wonks 5y ago
How does it compare to the_silver_searcher?
- HyperCrowd 5y ago
  I'd like to know this, too
- __jem 5y ago
  similar ergonomics, better performance
- snth 5y ago
  You might be interested in this blog post: https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
- Ashanmaril 5y ago
  There's a comparison to similar tools in the readme https://github.com/BurntSushi/ripgrep#quick-examples-comparing-tools https://github.com/BurntSushi/ripgrep#quick-examples-compari...
boredprograming 5y ago
How does ripgrep compare to Hyperscan? I always wanted to wrap that library in a grep utility and never got around to it
- killercup 5y ago
  Ripgrep includes one of the most prominent algorithms from Hyperscan internally for some expressions. Longer story: Ripgrep uses Rust's regex library, which uses the Aho-Corasick library. That does not just provide the algorithm it is named after, but also "packed" ones using SIMD, including a Rust rewrite of the [Teddy algorithm][1] from the Hyperscan project. [1]: https://github.com/BurntSushi/aho-corasick/tree/4499d7fdb41c6279ea1367bccf3daca9cb06c36c/src/packed/teddy https://github.com/BurntSushi/aho-corasick/tree/4499d7fdb41c...
  - boredprograming 5y ago
    That's awesome!
  - kevincox 5y ago
    This post by the author is a great introduction to the techniques used in ripgrep https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
- burntsushi 5y ago
  ripgrep's internals are generic over the regex engine, so not only is it possible to plug Hyperscan into ripgrep, but someone has already done it: https://sr.ht/~pierrenn/ripgrep/ https://sr.ht/~pierrenn/ripgrep/ (I should really add a link to that in the README.)
- glangdale 5y ago
  Ripgrep uses one of Hyperscan's string search algorithms. In general, Hyperscan isn't wonderfully suited for a "grep use case", as it's not focused on quick compile times. It was built for cases where the regex sets (and it's usually sets, sometimes large sets) are known in advance and where it's worth doing "heroics" to optimize scanning of these sets on a precompiled bytecode. It's not a direct comparison with Rust's regex crate or with ripgrep, but this article shows the comparison with re2. Notice that on average it takes 140K worth of scanning to "catch up" with RE2::Set with 10 patterns - the situation would be even more marked with 1 pattern. https://www.hyperscan.io/2017/06/20/regex-set-scanning-hyperscan-re2set/ https://www.hyperscan.io/2017/06/20/regex-set-scanning-hyper... Personally, I'm dissatisfied with the approach to regex scanning in Hyperscan (too heavyweight at construction and too complex) but not much more pleased by the Rust regex crate or RE2 (frankly, the whole compile-a-giant-DFA-as-you-go isn't that great either). I feel on the verge of taking another crack at the problem. Lord knows the world needs another regex library...
empiricus 5y ago
For SSD, if the read queue depth is big, the performance improves significantly compared to sequential reads. Does ripgrep has an option to read many files in parallel?
- burntsushi 5y ago
  ripgrep uses parallelism by default. Parallelism applies both to directory traversal, gitignore filtering and search. But that's where its granularity ends. It does not use parallelism at the level of single-file search.
atonse 5y ago
I didn’t know this was the same author as xsv, a tool I find indispensable now. Going to install ripgrep ASAP
- imiric 5y ago
  As a ripgrep user, TIL about xsv. Thanks for mentioning it, definitely one for the toolbelt. Can't believe I wasn't following BurntSushi on GitHub, what a track record.
- InspiredIdiot 5y ago
  Yeah. It almost seems unfair but I'll just use both and be glad they are available.
kisamoto 5y ago
I unconsciously reach for ripgrep on a daily basis. Finding references just is incredibly easy without cluttering it with stuff listed in .gitignore (I'm looking at you node_modules..) This has reminded and motivated me to submit a donation to the project.
- kisamoto 5y ago
  For those also looking I found it in the FAQ[0]: ------------------------------------------ How can I donate to ripgrep or its maintainers? As of now, you can't. While I believe the various efforts that are being undertaken to help fund FOSS are extremely important, they aren't a good fit for me. ripgrep is and I hope will remain a project of love that I develop in my free time. As such, involving money---even in the form of donations given without expectations---would severely change that dynamic for me personally. Instead, I'd recommend donating to something else that is doing work that you find meaningful. If you would like suggestions, then my favorites are: The Internet Archive Rails Girls Wikipedia * [0] https://github.com/BurntSushi/ripgrep/blob/64ac2ebe0f2fe1c8967e7ec550bc32466bf40a07/FAQ.md#how-can-i-donate-to-ripgrep-or-its-maintainers https://github.com/BurntSushi/ripgrep/blob/64ac2ebe0f2fe1c89...
jjice 5y ago
My favorite 'new wave Unix' utility by far. So ergonomic, yet an easy switch from egrep.
- RBerenguel 5y ago
  bat [0] (a cat replacement) and fzf [1] are the other two I would miss dearly, standing above some other "new" tools I use very regularly. [0]: https://github.com/sharkdp/bat [1]: https://github.com/junegunn/fzf
  - kasbah 5y ago
    Mine are fd [0] to replace `find` and tldr [1] to replace `man`. [0]: https://github.com/sharkdp/fd https://github.com/sharkdp/fd [1]: https://tldr.sh/ https://tldr.sh/
    - jjice 5y ago
      fd is fantastic as well, and I can get away with tldr 80% of the time before diving into man.
      - mananaysiempre 5y ago
        Have you looked at the original man pages for 7th ed. (http://man.cat-v.org/unix_7th http://man.cat-v.org/unix_7th) or Plan 9 (http://man.cat-v.org/plan_9_3rd_ed http://man.cat-v.org/plan_9_3rd_ed)? They’re basically tldr with less examples, and exhaustive, except there’s so much less stuff to exhaust. I was both pleasantly surprised by how simple they are and unpleasantly surprised by how far we’ve fallen in terms of complexity (though my recent comment https://news.ycombinator.com/item?id=27498189 https://news.ycombinator.com/item?id=27498189 explains why that’s an oversimplification). The GNU man pages (where they even exist) or the POSIX ones are a usability abomination in comparison. Even BSD’s are still quite bad.
    - papaf 5y ago
      I find repgrep defaults sensible when searching for text but fd misses files that I expect it to find -- this has happened a few times now and I have gone back to find. For instance: 'fd .apk' to search for android builds.
gigatexal 5y ago
I love ripgrep. It’s insanely fast. And very useful to my workflow.
sbergot 5y ago
Fun trivia: vscode uses ripgrep for its search feature
- xxpor 5y ago
  When I first started using VS code, I was googling around for a ripgrep extension for search. It took me a bit to find that oh, that's already the default. Shocking getting used to not having 40 years of legacy coming from emacs :)
  - TacticalCoder 5y ago
    I'm using ivy/avy/counsel in Emacs and there's counsel-rg, which I use all the time. It's fantastic. It's not stock, but it's really easy to configure (and it calls ripgrep under the hood, so you need to install ripgrep).
  - todd8 5y ago
    It's Emacs, so configuring it to use ripgrep instead of native is grep is possible in a few lines of Emacs lisp in your configuration file. See https://stegosaurusdormant.com/emacs-ripgrep/ https://stegosaurusdormant.com/emacs-ripgrep/
    - xxpor 5y ago
      I had done that. The point is with a modern editor, I don't have to. Emacs is unusable in a modern dev environment with its defaults. You have to write lisp.
ufo 5y ago
One time, I almost accidentally deleted my files because I typoed rm instead of rg. Was saved by the -i flag... Since then, I've been using "alias gr=rg"
- toxik 5y ago
  Kinda feel like rm should have been a longer command to type, like delete or unlink.
- moonchild 5y ago
  alias rm='rm -i'
- chmaynard 5y ago
  On a Mac, apparently it's safer to let the Finder handle it. # Move deleted files to macOS user trash (safer) trash () { command mv "$@" ~/.Trash ; }
  - saagarjha 5y ago
    I don’t see how this would have helped in this case.
piinbinary 5y ago
I don't see it listed in the notes, but using the --sort-files flag no longer seems to have a performance penalty.
- burntsushi 5y ago
  It still disables parallelism. No changes there. If you have anneasily reproducible benchmark on a public corpus, I could try to provide an analysis. (Note: if the corpus is small enough, it might not be possible to witness a perf difference. Or it may even be possible that using parallelism is slower due to the overhead of spinning up threads.)
  - e12e 5y ago
    Any idea how this works compared to eg GNU sort --parallel? (or clever tricks with partial sort and merge)? I'm guessing rg can be faster in general - because of less memory allocation/copying by sorting before outputting?
    - burntsushi 5y ago
      I'm not familiar with GNU sort's --parallel flag. That --sort-files disables parallelism is not really a theoretical limitation. See: https://github.com/BurntSushi/ripgrep/issues/152 https://github.com/BurntSushi/ripgrep/issues/152
  - piinbinary 5y ago
    Interesting! Perhaps the difference is then explained by my choice of search term. The term I tried after upgrading must happen to appear early in the sorted corpus. I just now tried it with a very rare term, and it does indeed take longer overall to complete the search.
tekronis 5y ago
If you're on macOS, you can also install ripgrep via MacPorts: "sudo port install ripgrep"
nix23 5y ago
Can someone help me? Why no use "grep -r"? Is it faster or better suited for recursive?
- rakenodiax 5y ago
  Check out this post for an in-depth analysis: https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
  - nix23 5y ago
    Thanks allot! Made it clear, HN is a big shit since 2 years, i might have to say goodby someday, one asks a neutral question an gets downvoted...what a fucking wannabe elite mindset.
- lalaithion 5y ago
  https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgrep https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgr...
- kzrdude 5y ago
  ripgrep is just one in the line of "more ergonomic greps", for example ack and ag were popular before ripgrep
c7DJTLrn 5y ago
ripgrep is one of the most powerful tools in my toolkit. It is amazing being able to scan through huge codebases for strings and patterns in what feels like an instant. It eliminates the need for code intelligence or a full fledged IDE in my experience.
nubb 5y ago
Just to jump on the burntsushi bandwagon, his nfldb project taught me postgres. I'll always be thankful for that.
nn3 5y ago
ripgrep always stroke me like the "industrial strength faberge egg" of Knuth/McIlroy fame [1] Yes it's an impressive achievement to make it that fast, but you could get the same or better performance by just using an index to search. I've been using GNU id-utils [2] for a long time, that is using an index, and it gets comparable performance for a fraction of the source code and brain power, and likely energy use too. [1] http://blobthescientist.blogspot.com/2017/10/knuth.html http://blobthescientist.blogspot.com/2017/10/knuth.html [2] https://www.gnu.org/software/idutils/ https://www.gnu.org/software/idutils/
- burntsushi 5y ago
  The comparison doesn't really make much sense IMO. Searching with and without indexing is targeting two very different use cases. I wanted to try idutils and see if it suffered from pitfalls (like if the indexing scanner didn't grab a token) or how it handled regexes. But I couldn't get it compiled using these commands[1]. It failed with: gcc -DHAVE_CONFIG_H -I. -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -MT fseterr.o -MD -MP -MF .deps/fseterr.Tpo -c -o fseterr.o fseterr.c fseterr.c: In function 'fseterr': fseterr.c:74:3: error: #error "Please port gnulib fseterr.c to your platform! Look at the definitions of ferror and clearerr on your system, then report this to bug-gnulib." 74 | #error "Please port gnulib fseterr.c to your platform! Look at the definitions of ferror and clearerr on your system, then report this to bug-gnulib." | ^~~~~ make[3]: *** [Makefile:1590: fseterr.o] Error 1 So I can't even easily get idutils to try it, but suspicion is that it misses some key use cases. And from what I can tell, its index doesn't auto-update, so you now have to use your brain power to remember to update the index. (Or figure out how to automate it.) ripgrep is commonly used in source code repos, where you might be editing code or checking out different branches all the time. An indexing tool in that scenario without excellent automatic updates is a non-starter. [1] - https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=idutils https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=iduti...
infocollector 5y ago
I've been using ugrep these days (https://github.com/Genivia/ugrep https://github.com/Genivia/ugrep) - and am pretty happy with it.
synergy20 5y ago
I use fd and ripgrep 90% of the time before I need return to find/grep these days.
nafizh 5y ago
I was actually searching for a superior grep and found ripgrep and ugrep. Both are nice but ugrep wins for me for the versatility and stronger fuzzy matching.
SNosTrAnDbLe 5y ago
Thanks so much for ripgrep!! It has become my goto for searching large code bases!!
asicsp 5y ago
I have a chapter on ripgrep (includes lot of examples for Rust regex as well) here: https://learnbyexample.github.io/learn_gnugrep_ripgrep/ripgrep.html https://learnbyexample.github.io/learn_gnugrep_ripgrep/ripgr... The -r option is handy as well for some search and replace problems (faster than GNU sed, literal search with -F, can use PCRE2 when needed, etc): https://learnbyexample.github.io/substitution-with-ripgrep/ https://learnbyexample.github.io/substitution-with-ripgrep/ --- I'm currently checking out another tool written in Rust - frawk (https://github.com/ezrosent/frawk https://github.com/ezrosent/frawk) as an awk alternative
TheNewAndy 5y ago
I like this page for a comparison of ripgrep and similar tools: https://beyondgrep.com/feature-comparison/ https://beyondgrep.com/feature-comparison/
tompazourek 5y ago
I use `git grep` all the time, and I'm always amazed how crazy fast it is on huge repos. And, of course, that also uses gitignore, has many parameters (including path filters after --). I guess ripgrep might be worth trying. Anyone used both of these who can compare? I'd like to know what I'm missing out on.
- BiteCode_dev 5y ago
  rip grep works outside of a git repo
- arunix 5y ago
  I find the output of rg easier to read vs git grep. However the latter has some useful options ("--and", "--or") that rg lacks: https://github.com/BurntSushi/ripgrep/issues/875#issuecomment-378089924 https://github.com/BurntSushi/ripgrep/issues/875#issuecommen...
_emacsomancer_ 5y ago
There are a number of Emacs packages which provide ripgrep integration in emacs. It makes for a nice combination of tools.
BiteCode_dev 5y ago
Little trick if you ripgrep a lot of web projects: alias rg="rg --max-columns 200 --max-columns-preview" So now if you hit a minified css or js file, it will truncate any match longer than 200 characters instead of flooding your screen with a million chars line.
- darkstarsys 5y ago
  This is helpful, but it doesn't (yet) show you the actual matching part of the line, just the first N chars.
cies 5y ago
Out of the many traditional CLI commands that have modern counterparts (in Rust) nowadays, this is the only one that really stuck. I need it, it's THAT much better than grep. Thansk Burntsushi.
joshspankit 5y ago
THANK YOU FOR PROVIDING A “If you have not heard of _____ before, this is what it does” LINE RIGHT IN THE RELEASE NOTES. Seriously: the amount of time I’ve spent trying to figure out what something does after clicking an interesting HN headline just boggles me. I feel like I want to start using ripgrep just because of that line.
- burntsushi 5y ago
  You're welcome. :) Someone suggested it long ago, and adding that blurb is specifically part of my release process: https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CHECKLIST.md https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CH...

547 points by mfrw 5y ago

mfrw 5y ago

Highlights: A new short flag, -., has been added. It is an alias for the --hidden flag, which instructs ripgrep to search hidden files and directories. BREAKING CHANGES: Binary detection output has changed slightly.

rektide 5y ago
loving '-.'. I use --hidden a lot. also noteable, an underlying lib got some vectorization speed improvements.
hawski 5y ago
Lack of this alias was one of the things that put me off regarding ripgrep. I'm using the silver searcher instead as it is plenty fast and it's CLI makes sense for me and doesn't change, AFAIR it's even mostly the same as ack, which I used before.

flakiness 5y ago

Haven't tried ripgrep before but finally installed. It is noticeably faster than older tools and more importantly the CLI ergonomics is much better! Thanks!

mfrw 5y ago
on a tangential note: there is a bunch of tools re-written in rust, a handy non exhaustive list [1]. [1]: https://zaiste.net/posts/shell-commands-rust/ https://zaiste.net/posts/shell-commands-rust/
- ibraheemdev 5y ago
  Here's a hopefully more exhaustive list that isn't restricted to Rust: https://github.com/ibraheemdev/modern-unix https://github.com/ibraheemdev/modern-unix, meaning it can include other awesome tools like fzf (written in Go).
  - mgradowski 5y ago
    If you like fzf, give skim [1] a go. It's basically faster (anecdotally) fzf, but can also be used as a Rust library. I made a little personal to-do manager using it, together with SQLite for persistence. I also improved my fish shell ergonomics with [2]. [1] https://github.com/lotabout/skim https://github.com/lotabout/skim [2] https://github.com/zenofile/fish-skim https://github.com/zenofile/fish-skim
  - mfrw 5y ago
    Thank you for this compilation. I think the demos are a cherry on top; it becomes an easy sell :) Thank you :)
- tspiteri 5y ago
  I love Rust, but I love and recommend ripgrep because of its usefulness, reliability and robustness, not because of the language it's written in. I haven't looked into all the tools in that list, but I would not for example recommend exa over ls as it is simply not reliable enough: if a filename ends in a space, you won't see that in exa, and that bug has been reported for years. To me that is a clear blocker, and if it is still there, I simply cannot trust the file listing from exa, no matter how pretty it may look.
  - duped 5y ago
    Static linking goes a long way to making Rust worth considering when choosing tools. Anecdote: I once had to recover a system with a corrupted libpcre.so. This will break almost every standard gnutil. The easiest way to do it without a recovery OS was to use a few alternatives written in Rust, which don't have this problem because they statically link their dependencies (and cargo still worked, so it was easy to install them).
    - saagarjha 5y ago
      Your anecdote is amusing, but I hope you’re not using it as an example to support your claim that Rust’s static linking should be considered when choosing tools.
mekster 5y ago
Wasn't ripgrep one of the first inspirations for others to initiate "rewrite existing tools with rust with better usability and performance" wave? Either way, thanks for a quality tool, more, if it inspired others to come up with good rust tools.
- burntsushi 5y ago
  Maybe in Rust. xsv does predate ripgrep though. Maybe ripgrep was the first one to get really popular though. To be honest, I dunno. With that said, Go tools were doing this before Rust even hit 1.0 as far as I remember. There was 'sift' and 'pt' for "smarter" greps for example, although I don't think either of them ever fully matured.

simonw 5y ago

ripgrep is absolutely fantastic software. If you're not using it yet I guarantee it's worth taking the time to install and learn basic usage. On macOS: "brew install ripgrep" - then use "rg searchterm" to search all nested files in the current directory that contain searchterm. You may be using it without realizing already: the code search feature in VS Code uses ripgrep under the hood.

cute_boi 5y ago
actually most of the time you might want to use rg -i searchterm . I don't know why its not default? May be they want to make it similar to grep but with perf?
- simonw 5y ago
  -i is ignore-case, right? Usually I'm searching code for things like variable names, so actually case-sensitive searches are a better default for me.
- raziel2p 5y ago
  --smart-case is worth looking into as well. you can also put whatever arguments you want in your ripgreprc file to tweak the "defaults" to your liking.
- bostonvaulter2 5y ago
  I think that's a bit of a personal choice. I definitely prefer the current default of an exact match including case. Partially because I'm very familiar with the common `-i` flag to ignore case which is used in `grep`, `git grep`, `ack`, `ag` and probably others. But I've never used a flag to make a search case-sensitive. Plus it's easy enough to alias `rg` to `rg -i` if you prefer that as the default.
- Hamuko 5y ago
  I almost never use -i. I may do rg "[Dd]efault" if I want to search for "default" or "Default", but not "DEFAULT". Usually I just want to search in a very specific case since I'm looking for variables.

jayflux 5y ago

Shoutout to Burntsushi He knows how to keep a good changelog https://github.com/BurntSushi/ripgrep/blob/master/CHANGELOG.md https://github.com/BurntSushi/ripgrep/blob/master/CHANGELOG.... it’s good to read through and see the progress. He also cares about ergonomics as well as performance which I like. I find it much more easier to use than existing Grep tools. Interestingly, I believe Ripgrep started off as a test harness to benchmark the Rust Regex crate (he was the author of that). It was originally not intended to be a consumer cli. I guess he dogfooded it that well it became a successful tool

burntsushi 5y ago
Thanks. :-) That's right. IIRC, I wrote it as a simple harness because I wanted to check whether someone could feasibly write a grep tool using the regex crate and get similar performance. Once I got the harness up and running though, I realized it was not only competitive with GNU grep in a larger number of cases than I thought it would be (after some perf bug fixes, IIRC), but was substantially faster than ag. So that's when I decided to polish it up into a tool for others to use.
- e12e 5y ago
  Do you use any tooling for the Changelog/issue tagging (PERF, BUG) etc?
  - burntsushi 5y ago
    Nope. I just try to keep it updated as I go. But in practice, I usually forget to update it and it just ends up becoming part of the release process.[1] [1] - https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CHECKLIST.md https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CH...
- vvanpo 5y ago
  I recently switched to `rg` after many years of using `ag`, and the fact that it properly handles `.gitignore` syntax makes my life much better.
- 5y ago

oweiler 5y ago

The best feature about ripgrep (apart from its great performance): Sane defaults out of the box!

valtism 5y ago
This can be an underrated feature of software, although I feel like we're getting better at it. I remember being so frustrated with guides to using software back in the early 00s where part of the guide would be reconfiguring default settings. It always seemed to be a sign of sub-par quality software.

coffeeri 5y ago

Also consider installing ripgrep-all [1], an extension making it possible to search through PDFs, eBooks, ZIPs and other documents. The integration with fzf is also fantastic! [1] https://github.com/phiresky/ripgrep-all https://github.com/phiresky/ripgrep-all

O_H_E 5y ago
Aaaaa, yes yes yes. I need that in my life.
joshspankit 5y ago
:O
allenleein 5y ago
I have been using ripgrep-all since very beginning. Can't live without it!
nyanpasu64 5y ago
`cargo install ripgrep-all` fails to compile due to a missing cachedir dependency, and the author won't make a new release out of master due to a possible deadlocking issue. This fails to inspire confidence: https://github.com/phiresky/ripgrep-all/issues/88 https://github.com/phiresky/ripgrep-all/issues/88

wonks 5y ago

How does it compare to the_silver_searcher?

HyperCrowd 5y ago
I'd like to know this, too
__jem 5y ago
similar ergonomics, better performance
snth 5y ago
You might be interested in this blog post: https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
Ashanmaril 5y ago
There's a comparison to similar tools in the readme https://github.com/BurntSushi/ripgrep#quick-examples-comparing-tools https://github.com/BurntSushi/ripgrep#quick-examples-compari...

boredprograming 5y ago

How does ripgrep compare to Hyperscan? I always wanted to wrap that library in a grep utility and never got around to it

killercup 5y ago
Ripgrep includes one of the most prominent algorithms from Hyperscan internally for some expressions. Longer story: Ripgrep uses Rust's regex library, which uses the Aho-Corasick library. That does not just provide the algorithm it is named after, but also "packed" ones using SIMD, including a Rust rewrite of the [Teddy algorithm][1] from the Hyperscan project. [1]: https://github.com/BurntSushi/aho-corasick/tree/4499d7fdb41c6279ea1367bccf3daca9cb06c36c/src/packed/teddy https://github.com/BurntSushi/aho-corasick/tree/4499d7fdb41c...
- boredprograming 5y ago
  That's awesome!
- kevincox 5y ago
  This post by the author is a great introduction to the techniques used in ripgrep https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
burntsushi 5y ago
ripgrep's internals are generic over the regex engine, so not only is it possible to plug Hyperscan into ripgrep, but someone has already done it: https://sr.ht/~pierrenn/ripgrep/ https://sr.ht/~pierrenn/ripgrep/ (I should really add a link to that in the README.)
glangdale 5y ago
Ripgrep uses one of Hyperscan's string search algorithms. In general, Hyperscan isn't wonderfully suited for a "grep use case", as it's not focused on quick compile times. It was built for cases where the regex sets (and it's usually sets, sometimes large sets) are known in advance and where it's worth doing "heroics" to optimize scanning of these sets on a precompiled bytecode. It's not a direct comparison with Rust's regex crate or with ripgrep, but this article shows the comparison with re2. Notice that on average it takes 140K worth of scanning to "catch up" with RE2::Set with 10 patterns - the situation would be even more marked with 1 pattern. https://www.hyperscan.io/2017/06/20/regex-set-scanning-hyperscan-re2set/ https://www.hyperscan.io/2017/06/20/regex-set-scanning-hyper... Personally, I'm dissatisfied with the approach to regex scanning in Hyperscan (too heavyweight at construction and too complex) but not much more pleased by the Rust regex crate or RE2 (frankly, the whole compile-a-giant-DFA-as-you-go isn't that great either). I feel on the verge of taking another crack at the problem. Lord knows the world needs another regex library...

empiricus 5y ago

For SSD, if the read queue depth is big, the performance improves significantly compared to sequential reads. Does ripgrep has an option to read many files in parallel?

burntsushi 5y ago
ripgrep uses parallelism by default. Parallelism applies both to directory traversal, gitignore filtering and search. But that's where its granularity ends. It does not use parallelism at the level of single-file search.

atonse 5y ago

I didn’t know this was the same author as xsv, a tool I find indispensable now. Going to install ripgrep ASAP

imiric 5y ago
As a ripgrep user, TIL about xsv. Thanks for mentioning it, definitely one for the toolbelt. Can't believe I wasn't following BurntSushi on GitHub, what a track record.
InspiredIdiot 5y ago
Yeah. It almost seems unfair but I'll just use both and be glad they are available.

kisamoto 5y ago

I unconsciously reach for ripgrep on a daily basis. Finding references just is incredibly easy without cluttering it with stuff listed in .gitignore (I'm looking at you node_modules..) This has reminded and motivated me to submit a donation to the project.

kisamoto 5y ago
For those also looking I found it in the FAQ[0]: ------------------------------------------ How can I donate to ripgrep or its maintainers? As of now, you can't. While I believe the various efforts that are being undertaken to help fund FOSS are extremely important, they aren't a good fit for me. ripgrep is and I hope will remain a project of love that I develop in my free time. As such, involving money---even in the form of donations given without expectations---would severely change that dynamic for me personally. Instead, I'd recommend donating to something else that is doing work that you find meaningful. If you would like suggestions, then my favorites are: The Internet Archive Rails Girls Wikipedia * [0] https://github.com/BurntSushi/ripgrep/blob/64ac2ebe0f2fe1c8967e7ec550bc32466bf40a07/FAQ.md#how-can-i-donate-to-ripgrep-or-its-maintainers https://github.com/BurntSushi/ripgrep/blob/64ac2ebe0f2fe1c89...

jjice 5y ago

My favorite 'new wave Unix' utility by far. So ergonomic, yet an easy switch from egrep.

RBerenguel 5y ago
bat [0] (a cat replacement) and fzf [1] are the other two I would miss dearly, standing above some other "new" tools I use very regularly. [0]: https://github.com/sharkdp/bat [1]: https://github.com/junegunn/fzf
- kasbah 5y ago
  Mine are fd [0] to replace `find` and tldr [1] to replace `man`. [0]: https://github.com/sharkdp/fd https://github.com/sharkdp/fd [1]: https://tldr.sh/ https://tldr.sh/
  - jjice 5y ago
    fd is fantastic as well, and I can get away with tldr 80% of the time before diving into man.
    - mananaysiempre 5y ago
      Have you looked at the original man pages for 7th ed. (http://man.cat-v.org/unix_7th http://man.cat-v.org/unix_7th) or Plan 9 (http://man.cat-v.org/plan_9_3rd_ed http://man.cat-v.org/plan_9_3rd_ed)? They’re basically tldr with less examples, and exhaustive, except there’s so much less stuff to exhaust. I was both pleasantly surprised by how simple they are and unpleasantly surprised by how far we’ve fallen in terms of complexity (though my recent comment https://news.ycombinator.com/item?id=27498189 https://news.ycombinator.com/item?id=27498189 explains why that’s an oversimplification). The GNU man pages (where they even exist) or the POSIX ones are a usability abomination in comparison. Even BSD’s are still quite bad.
  - papaf 5y ago
    I find repgrep defaults sensible when searching for text but fd misses files that I expect it to find -- this has happened a few times now and I have gone back to find. For instance: 'fd .apk' to search for android builds.

gigatexal 5y ago

I love ripgrep. It’s insanely fast. And very useful to my workflow.

sbergot 5y ago

Fun trivia: vscode uses ripgrep for its search feature

xxpor 5y ago
When I first started using VS code, I was googling around for a ripgrep extension for search. It took me a bit to find that oh, that's already the default. Shocking getting used to not having 40 years of legacy coming from emacs :)
- TacticalCoder 5y ago
  I'm using ivy/avy/counsel in Emacs and there's counsel-rg, which I use all the time. It's fantastic. It's not stock, but it's really easy to configure (and it calls ripgrep under the hood, so you need to install ripgrep).
- todd8 5y ago
  It's Emacs, so configuring it to use ripgrep instead of native is grep is possible in a few lines of Emacs lisp in your configuration file. See https://stegosaurusdormant.com/emacs-ripgrep/ https://stegosaurusdormant.com/emacs-ripgrep/
  - xxpor 5y ago
    I had done that. The point is with a modern editor, I don't have to. Emacs is unusable in a modern dev environment with its defaults. You have to write lisp.

ufo 5y ago

One time, I almost accidentally deleted my files because I typoed rm instead of rg. Was saved by the -i flag... Since then, I've been using "alias gr=rg"

toxik 5y ago
Kinda feel like rm should have been a longer command to type, like delete or unlink.
moonchild 5y ago
alias rm='rm -i'
chmaynard 5y ago
On a Mac, apparently it's safer to let the Finder handle it. # Move deleted files to macOS user trash (safer) trash () { command mv "$@" ~/.Trash ; }
- saagarjha 5y ago
  I don’t see how this would have helped in this case.

piinbinary 5y ago

I don't see it listed in the notes, but using the --sort-files flag no longer seems to have a performance penalty.

burntsushi 5y ago
It still disables parallelism. No changes there. If you have anneasily reproducible benchmark on a public corpus, I could try to provide an analysis. (Note: if the corpus is small enough, it might not be possible to witness a perf difference. Or it may even be possible that using parallelism is slower due to the overhead of spinning up threads.)
- e12e 5y ago
  Any idea how this works compared to eg GNU sort --parallel? (or clever tricks with partial sort and merge)? I'm guessing rg can be faster in general - because of less memory allocation/copying by sorting before outputting?
  - burntsushi 5y ago
    I'm not familiar with GNU sort's --parallel flag. That --sort-files disables parallelism is not really a theoretical limitation. See: https://github.com/BurntSushi/ripgrep/issues/152 https://github.com/BurntSushi/ripgrep/issues/152
- piinbinary 5y ago
  Interesting! Perhaps the difference is then explained by my choice of search term. The term I tried after upgrading must happen to appear early in the sorted corpus. I just now tried it with a very rare term, and it does indeed take longer overall to complete the search.

tekronis 5y ago

If you're on macOS, you can also install ripgrep via MacPorts: "sudo port install ripgrep"

nix23 5y ago

Can someone help me? Why no use "grep -r"? Is it faster or better suited for recursive?

rakenodiax 5y ago
Check out this post for an in-depth analysis: https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
- nix23 5y ago
  Thanks allot! Made it clear, HN is a big shit since 2 years, i might have to say goodby someday, one asks a neutral question an gets downvoted...what a fucking wannabe elite mindset.
lalaithion 5y ago
https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgrep https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgr...
kzrdude 5y ago
ripgrep is just one in the line of "more ergonomic greps", for example ack and ag were popular before ripgrep

c7DJTLrn 5y ago

ripgrep is one of the most powerful tools in my toolkit. It is amazing being able to scan through huge codebases for strings and patterns in what feels like an instant. It eliminates the need for code intelligence or a full fledged IDE in my experience.

nubb 5y ago

Just to jump on the burntsushi bandwagon, his nfldb project taught me postgres. I'll always be thankful for that.

nn3 5y ago

ripgrep always stroke me like the "industrial strength faberge egg" of Knuth/McIlroy fame [1] Yes it's an impressive achievement to make it that fast, but you could get the same or better performance by just using an index to search. I've been using GNU id-utils [2] for a long time, that is using an index, and it gets comparable performance for a fraction of the source code and brain power, and likely energy use too. [1] http://blobthescientist.blogspot.com/2017/10/knuth.html http://blobthescientist.blogspot.com/2017/10/knuth.html [2] https://www.gnu.org/software/idutils/ https://www.gnu.org/software/idutils/

burntsushi 5y ago
The comparison doesn't really make much sense IMO. Searching with and without indexing is targeting two very different use cases. I wanted to try idutils and see if it suffered from pitfalls (like if the indexing scanner didn't grab a token) or how it handled regexes. But I couldn't get it compiled using these commands[1]. It failed with: gcc -DHAVE_CONFIG_H -I. -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -MT fseterr.o -MD -MP -MF .deps/fseterr.Tpo -c -o fseterr.o fseterr.c fseterr.c: In function 'fseterr': fseterr.c:74:3: error: #error "Please port gnulib fseterr.c to your platform! Look at the definitions of ferror and clearerr on your system, then report this to bug-gnulib." 74 | #error "Please port gnulib fseterr.c to your platform! Look at the definitions of ferror and clearerr on your system, then report this to bug-gnulib." | ^~~~~ make[3]: *** [Makefile:1590: fseterr.o] Error 1 So I can't even easily get idutils to try it, but suspicion is that it misses some key use cases. And from what I can tell, its index doesn't auto-update, so you now have to use your brain power to remember to update the index. (Or figure out how to automate it.) ripgrep is commonly used in source code repos, where you might be editing code or checking out different branches all the time. An indexing tool in that scenario without excellent automatic updates is a non-starter. [1] - https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=idutils https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=iduti...

infocollector 5y ago

I've been using ugrep these days (https://github.com/Genivia/ugrep https://github.com/Genivia/ugrep) - and am pretty happy with it.

synergy20 5y ago

I use fd and ripgrep 90% of the time before I need return to find/grep these days.

nafizh 5y ago

I was actually searching for a superior grep and found ripgrep and ugrep. Both are nice but ugrep wins for me for the versatility and stronger fuzzy matching.

SNosTrAnDbLe 5y ago

Thanks so much for ripgrep!! It has become my goto for searching large code bases!!

asicsp 5y ago

I have a chapter on ripgrep (includes lot of examples for Rust regex as well) here: https://learnbyexample.github.io/learn_gnugrep_ripgrep/ripgrep.html https://learnbyexample.github.io/learn_gnugrep_ripgrep/ripgr... The -r option is handy as well for some search and replace problems (faster than GNU sed, literal search with -F, can use PCRE2 when needed, etc): https://learnbyexample.github.io/substitution-with-ripgrep/ https://learnbyexample.github.io/substitution-with-ripgrep/ --- I'm currently checking out another tool written in Rust - frawk (https://github.com/ezrosent/frawk https://github.com/ezrosent/frawk) as an awk alternative

TheNewAndy 5y ago

I like this page for a comparison of ripgrep and similar tools: https://beyondgrep.com/feature-comparison/ https://beyondgrep.com/feature-comparison/

tompazourek 5y ago

I use `git grep` all the time, and I'm always amazed how crazy fast it is on huge repos. And, of course, that also uses gitignore, has many parameters (including path filters after --). I guess ripgrep might be worth trying. Anyone used both of these who can compare? I'd like to know what I'm missing out on.

BiteCode_dev 5y ago
rip grep works outside of a git repo
arunix 5y ago
I find the output of rg easier to read vs git grep. However the latter has some useful options ("--and", "--or") that rg lacks: https://github.com/BurntSushi/ripgrep/issues/875#issuecomment-378089924 https://github.com/BurntSushi/ripgrep/issues/875#issuecommen...

_emacsomancer_ 5y ago

There are a number of Emacs packages which provide ripgrep integration in emacs. It makes for a nice combination of tools.

BiteCode_dev 5y ago

Little trick if you ripgrep a lot of web projects: alias rg="rg --max-columns 200 --max-columns-preview" So now if you hit a minified css or js file, it will truncate any match longer than 200 characters instead of flooding your screen with a million chars line.

darkstarsys 5y ago
This is helpful, but it doesn't (yet) show you the actual matching part of the line, just the first N chars.

cies 5y ago

Out of the many traditional CLI commands that have modern counterparts (in Rust) nowadays, this is the only one that really stuck. I need it, it's THAT much better than grep. Thansk Burntsushi.

joshspankit 5y ago

THANK YOU FOR PROVIDING A “If you have not heard of _____ before, this is what it does” LINE RIGHT IN THE RELEASE NOTES. Seriously: the amount of time I’ve spent trying to figure out what something does after clicking an interesting HN headline just boggles me. I feel like I want to start using ripgrep just because of that line.

burntsushi 5y ago
You're welcome. :) Someone suggested it long ago, and adding that blurb is specifically part of my release process: https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CHECKLIST.md https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-CH...