15 ms·

Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

390 points by subset 3y ago

latexr 3y ago
The title needs “(2016)”. This is the original announcement, not new information.
- asicsp 3y ago
  Discussions: "Ripgrep – A new command line search tool" https://news.ycombinator.com/item?id=12564442 https://news.ycombinator.com/item?id=12564442 (740 points | Sept 23, 2016 | 209 comments) - there are discussions related to speed too "Ripgrep is faster (2016)" https://news.ycombinator.com/item?id=17941319 https://news.ycombinator.com/item?id=17941319 (98 points | Sept 8, 2018 | 40 comments)
stinos 3y ago
It's fast indeed. And I can't help keeping promoting the combination with fzf :) For those who want to try it out, this is a Powershell function but the same principle applies in any shell. Does ripgrep then puts fuzzy searching in the resulting files+text on top while showing context in bat: function frg { $result = rg --ignore-case --color=always --line-number --no-heading @Args | fzf --ansi ` --color 'hl:-1:underline,hl+:-1:underline:reverse' ` --delimiter ':' ` --preview "bat --color=always {1} --theme='Solarized (light)' --highlight-line {2}" ` --preview-window 'up,60%,border-bottom,+{2}+3/3,~3' if ($result) { & ($env:EDITOR).Trim("`"'") $result.Split(': ')[0] } } There are other ways to approach this, but for me this is a very fast way of nailing down 'I now something exists in this multi-repo project but don't know where exactly nor the exact name' edit this comes out of https://github.com/junegunn/fzf/blob/master/ADVANCED.md https://github.com/junegunn/fzf/blob/master/ADVANCED.md and even though you might not want to use most of what is in there, it's still worth glancing over it to get ideas of what you could do with it
- pknopf 3y ago
  Thanks for this!
- majkinetor 3y ago
  Not bad. For those who want to try it, install all prereqs with: choco install fzf bat ripgrep How do you scroll the preview window with keyboard ?
  - stinos 3y ago
    shift-up/down
  - nickjj 3y ago
    > How do you scroll the preview window with keyboard ? alias pf="fzf --preview='less {}' --bind shift-up:preview-page-up,shift-down:preview-page-down" That will let you run `pf` to preview files in less and lets you use shift + arrow keys to scroll the preview window. No dependencies are needed except for fzf. If you want to use ripgrep with fzf you can set FZF_DEFAULT_COMMAND to run rg such as `export FZF_DEFAULT_COMMAND="rg ..."` where ... are your preferred rg flags. This full setup is in my dotfiles at https://github.com/nickjj/dotfiles https://github.com/nickjj/dotfiles. I've made a video and blog post about it here: https://nickjanetakis.com/blog/customize-fzf-ctrl-t-binding-to-preview-files https://nickjanetakis.com/blog/customize-fzf-ctrl-t-binding-... I also made https://nickjanetakis.com/blog/using-fzf-to-preview-text-files-on-the-command-line-and-within-vim https://nickjanetakis.com/blog/using-fzf-to-preview-text-fil... which covers how to modify fzf's built in CTRL+t shortcut to allow for previews too. CTRL+t is a hotkey driven way to fuzzy match a list of files.
- wanderingmind 3y ago
  Infact I would recommend a step further to integrate rip-grep-all (rga) with fzf that can do a fuzzy search not just on text files but on all types of files including pdfs, zip files. More details here [1] [1] https://github.com/phiresky/ripgrep-all/wiki/fzf-Integration https://github.com/phiresky/ripgrep-all/wiki/fzf-Integration
  - fsiefken 3y ago
    That's really nice, thanks. Long ago there was the Google Desktop Search where you could 'Google' your local documents. But the difference is that that worked with an index, so I imagine it's faster if you have thousands of pdfs en epubs.
    - cb321 3y ago
      Even longer ago, there was `glimpse`: https://www.linuxjournal.com/article/1164 https://www.linuxjournal.com/article/1164 which is still available. [1] glimpse's index-builds are like 10X slower than `qgrep` mentioned elsethread. `qgrep` also seems to have faster search (though I only tried a few patterns) and `qgrep` does not allow spelling errors like `glimpse`. Neither `glimpse` nor `qgrep`, to my knowledge, directly supports pre-processing / document conversion (like `pdftotext`), though I imagine this would be easy to add to either replicating Desktop Search. (Indirectly, at some space cost, you could always dump conversions into a shadow file hierarchy, index that, and then translate path names.) [1] https://manpages.ubuntu.com/manpages/focal/man1/glimpse.1.html https://manpages.ubuntu.com/manpages/focal/man1/glimpse.1.ht...
- tambourine_man 3y ago
  Vim is almost broken for me without fzf+rg. Feels like I’m manually grinding coffee instead of using electricity.
  - stirfish 3y ago
    This comment got me to get out my French press and manually grind some beans. It wasn't a meditative and calming as I remember, and the coffee tastes a little...dusty. I guess it's time for me to update my vimrc.
    - orangepurple 3y ago
      Aeropress is a direct upgrade from french press and uses way less coffee
      - stirfish 3y ago
        I hadn't heard of this before. Thanks!
        orangepurple 3y ago
        Add one or a few drops of water to your roasted coffee beans with your hand and shake well after weighing it out to stop the grinds from sticking to the walls of your grinder from static.
        iknowstuff 3y ago
        This thread took a turn
        bch 3y ago
        I love that a ripgrep article has such a deeply nerdy coffee thread… > Add one or a few drops of water to your roasted coffee beans Ah, RDT (Ross Droplet Technique)[0]. A little atomizer (“spritz” bottle) of plain water serves well here. NB: this is for single-dose grinding - e.g. measuring a small amount of beans loaded into a grinder to grind immediately. If you have a grinder with a “big” hopper on top that has (e.g.) the weeks worth of coffee (even though you grind on-demand for ea. espresso/french press/aeropress/pourover/drip/…) this isn’t for you. [0] https://thebasicbarista.com/en-us/blogs/topics/how-rdt-broke-changed-the-game-benefits-and-reasons-why-people-spray-their-coffee-beans https://thebasicbarista.com/en-us/blogs/topics/how-rdt-broke...
- shoover 3y ago
  Oh. Thanks for the tip. This might make me finally embrace powershell. I’ve been using WSL+zsh+fzf as a Windows CLI for continuity with day job Mac tools, but git CLI performance is only usable inside the WSL file system.
  - fhlu 3y ago
    You can also add a small script to your WSL under `/usr/local/bin/git`: GIT_WINDOWS="/mnt/c/Program Files/Git/bin/git.exe" GIT_LINUX="/usr/bin/git" case "$(pwd -P)" in /mnt/?/*) case "$@" in # Needed to fix prompt, but it breaks things like paging, colours, etc rev-parse*) # running linux git for rev-parse seems faster, even without translating paths exec "$GIT_LINUX" "$@" ;; *) exec "$GIT_WINDOWS" -c color.ui=always "$@" ;; esac ;; *) exec "$GIT_LINUX" "$@" ;; esac This allows you to use `git` in your WSL shell but it'll pick whichever executable is suitable for the filesystem that the repo is in :)
    - shoover 3y ago
      Thank you. I will use this.
      - ciroduran 3y ago
        The code as written above only works if you haven't changed the mountpoint for your windows partition (i.e. from /mnt), consider that
  - stinos 3y ago
    This might make me finally embrace powershell Yeah, I have a bit of a love-hate relationship with it. But I actually have that with all shells out there. I don't know if it's just me or the shells, or (the most likely I think): a bit of both. But PS is available out of the box and using objects vs plain text is a major win in my book, and even though I still don't know half of the syntax by heart it feels less of an endless fight than other shells. And since I use the shell itself for rather basic things and for the rest only for tools (like shown here), we get along just fine.
- imakira 3y ago
  I wrote a bash version of this: function frg { result=`rg --ignore-case --color=always --line-number --no-heading "$@" | fzf --ansi \ --color 'hl:-1:underline,hl+:-1:underline:reverse' \ --delimiter ':' \ --preview "bat --color=always {1} --theme='Solarized (light)' --highlight-line {2}" \ --preview-window 'up,60%,border-bottom,+{2}+3/3,~3'` file="${result%%:*}" linenumber=`echo "${result}" | cut -d: -f2` if [ ! -z "$file" ]; then $EDITOR +"${linenumber}" "$file" fi }
  - zero_k 3y ago
    Wow, nice, thanks! :heart:
  - seanw444 3y ago
    I've never really seen PowerShell beyond minimal commands, but after seeing the parent, I definitely think it has the superior syntax of the shells. Especially for scripts.
    - bbkane 3y ago
      Maybe if the "popular" shells, but http://www.nushell.sh/ http://www.nushell.sh/ is looking better and better
      - evntdrvn 3y ago
        nu acknowledged powershell as one of its inspirations, yeah!!
      - steveklabnik 3y ago
        I use it near exclusively. Love nu.
    - Natfan 3y ago
      A lot of people sleep on PowerShell, possibly because some of the syntax is a little clunky (and quite slow compared to some other shells, I will freely admit). That being said, I'd argue object oriented programming is a massive improvement over text oriented programming. I never want to touch awk again!
- bloopernova 3y ago
  With fzf you can add lots of files to git while skipping some if you want: fza = "!git ls-files -m -o --exclude-standard | fzf -m --print0 | xargs -0 git add" With that in the [alias] section of a gitconfig file, running git fza brings up a list of modified and not yet added files, space toggles each entry and moves to the next entry. That alias as well as fzf+fd really speed up some parts of my workflow. Oh and shameless plug for my guide on what to include in your zsh setup on macOS: https://gist.github.com/aclarknexient/0ffcb98aa262c585c49d4b3f3ae24019 https://gist.github.com/aclarknexient/0ffcb98aa262c585c49d4b...
  - stinos 3y ago
    Add the preview to see what you're actually stashing: git ls-files -m -o --exclude-standard | fzf -m --print0 --preview "git diff {1}" | .... And that's just the start: it could even be that by binding a key to the fzf reload command to then display the diff in it's finder, and in turn a key to stage the selected line, you could turn that into an interactive git staging tool.
    - bloopernova 3y ago
      Nice. I'll have to try that out!
  - kstrauser 3y ago
    That blew my mind. I've used fzf a couple time here and there, but now I Get It. Thanks!
- w0m 3y ago
  This is a gem; thank you.
- taude 3y ago
  This is pretty much my exact use of ripgrep, too. I use it as a starting point to zero in on files/projects in a several-hundred repo codebase, and then go from there....
- shmerl 3y ago
  I use it neovim with fzf search.
stabbles 3y ago
But is any of them using `_mm256_sad_epu8` for small, literal strings?
- lifthrasiir 3y ago
  Not exactly but yes, it ultimately uses the `memchr` crate [1] which provides SIMD-optimized character and string search routines. But it uses `_mm256_cmpeq_epi8` instead of `_mm256_sad_epu8`. [1] https://docs.rs/memchr/latest/memchr/ https://docs.rs/memchr/latest/memchr/
  - burntsushi 3y ago
    And also the SIMD in aho-corasick, which is used whenever a small number of literals are searched for. For example, `foo|bar` or `(?i)foo`. https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56b3f28e9dd69a655316af0107b29b/src/packed/vector.rs https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56... But no `_mm256_sad_epu8`. What an oddly specific question..?
    - stabbles 3y ago
      What is really oddly specific is this instruction :p but I believe it is a common trick to quickly scan for short string matches. It computes `sum(|x[i] - y[i]|)` for consecutive `i` at different offsets, so it should be zero at substring matches. For context: https://epubs.siam.org/doi/pdf/10.1137/1.9781611972931.10 https://epubs.siam.org/doi/pdf/10.1137/1.9781611972931.10 I was slightly mistaken, the instruction of interest is _mm256_mpsadbw_epu8
      - burntsushi 3y ago
        Oh I see. Yes, that's what is commonly used in academic publications. But I've yet to see it used in the wild. I mentioned exactly that paper (I believe) in my write-up on Teddy: https://github.com/BurntSushi/aho-corasick/tree/master/src/packed/teddy https://github.com/BurntSushi/aho-corasick/tree/master/src/p...
      - 3y ago
zoobab 3y ago
git grep cannot even find a simple string in its repo.
- ctenb 3y ago
  What? Git grep is all you ever need in my experience, and it's ~faster than~ (edit: as fast as) ripgrep when searching a git repo.
  - dharmab 3y ago
    > it's faster than ripgrep when searching a git repo. Source? Ripgrep's benchmarks show it significantly faster.
    - ctenb 3y ago
      See sibling comment
  - larodi 3y ago
    I really doubt git grep can outperform ripgrep in any tests... please provide some proof.
    - ctenb 3y ago
      I tested this on a large repo in 2016 when I installed several tools (including rg and ag) to compare speed. I don't have the metrics anymore, but the results were pretty clear then. According to the benchmarks from the OP, git grep is pretty comparable to rg in a large git repo. I guess different benchmarks give slightly different results, but the OP acknowledges that git grep is very fast. Bonus is that it comes preinstalled with git and can search through commit history.
      - burntsushi 3y ago
        Author of ripgrep here. It really just depends. The way I like to characterize `git grep` (at present) is that it has sharp performance cliffs. ripgrep has them too, to be sure, but I think it has fewer of them. If you're just searching for a simple literal, `git grep` is decently fast: $ git remote -v origin git@github.com:torvalds/linux (fetch) origin git@github.com:torvalds/linux (push) $ git rev-parse HEAD f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 $ time LC_ALL=en_US.UTF-8 git grep -c -E 'PM_RESUME' Documentation/dev-tools/sparse.rst:3 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 arch/arm/mach-omap2/omap-secure.h:1 arch/arm/mach-omap2/pm33xx-core.c:1 arch/x86/kernel/apm_32.c:1 drivers/input/mouse/cyapa.h:1 drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/usb/mtu3/mtu3_hw_regs.h:1 include/uapi/linux/apm_bios.h:1 real 0.215 user 0.421 sys 1.226 maxmem 161 MB faults 0 $ time rg -c 'PM_RESUME' drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/usb/mtu3/mtu3_hw_regs.h:1 drivers/input/mouse/cyapa.h:1 arch/x86/kernel/apm_32.c:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 Documentation/dev-tools/sparse.rst:3 arch/arm/mach-omap2/pm33xx-core.c:1 arch/arm/mach-omap2/omap-secure.h:1 include/uapi/linux/apm_bios.h:1 real 0.078 user 0.259 sys 0.577 maxmem 15 MB faults 0 But if you switch it up and start adding regex things to your pattern, there can be substantial slowdowns: $ time LC_ALL=C git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 5.704 user 55.671 sys 0.585 maxmem 207 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 24.529 user 4:34.42 sys 0.753 maxmem 211 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -P '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 1.372 user 16.980 sys 0.647 maxmem 211 MB faults 1 $ time rg -c '\w{5,}\s+PM_RESUME' Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 0.082 user 0.226 sys 0.612 maxmem 18 MB faults 0 In the above cases, ripgrep has Unicode enabled. (It's enabled by default irrespective of locale settings. ripgrep doesn't interact with POSIX locales at all.)
nvoeiah 3y ago
[flagged]
- WJW 3y ago
  So? My distro doesn't come with almost any of the tools I need for day to day work. It has never been a problem for me to install a new editor or compiler on a new machine, I don't see why ripgrep would be any different. Especially since it's usually a single command to install anyway.
- deleted 3y ago
  [deleted]
jedisct1 3y ago
I switched from from ripgrep to ugrep and never looked back. It's just as fast, but also comes with fuzzy matching (which is super useful), a TUI (useful for code reviews), and can also search in PDFs, archives, etc. The optional Google search syntax also very convenient. https://ugrep.com https://ugrep.com
- ranting-moth 3y ago
  I'm scared that if I start using Google search syntax in my grepping that I'll mostly get results trying to sell me something :)
- hnfong 3y ago
  So I was casually searching for "ugrep vs ripgrep" articles, when I stumbled upon a couple reddit posts where apparently the authors of ugrep and ripgrep seemed to have a multi-year feud on reddit, eg. https://www.reddit.com/r/programming/comments/120wqvr/ripgrep_is_faster_than_grep_ag_git_grep_ucg_pt/jdn2ybv/?context=3 https://www.reddit.com/r/programming/comments/120wqvr/ripgre... So weird. I mean, it's just about some open source tool, right? :-/
  - GuB-42 3y ago
    Psst, don't tell him about emacs and vi(m)... There are feuds about open source tools all the time. Text editors, Linux distros, shells, programming languages, desktop environments, etc... And ugrep vs ripgrep may be a poster child for C++ vs Rust. It is not all bad, it drives progress, and it usually stays at a technical level, I've yet to see people killing each others for their choice of command line search tool.
  - fliife 3y ago
    This is so weird, even ripgrep's author is actively seeking conflict in ugrep's new release posts. Not a good colour on both of them.
    - orlp 3y ago
      At least how I read it the linked post was ugrep's author seeking conflict, not the ripgrep's.
MrHamdulay 3y ago
What's interesting is that ripgrep now also powers VS Code search with a Node.js wrapper. https://www.npmjs.com/package/@vscode/ripgrep https://www.npmjs.com/package/@vscode/ripgrep
- porwah 3y ago
  ...which is awesome if you can request/install VS Code but not ripgrep. You can find the rg binary in the VS installation (at least, I can on Windows at my place of employment).
  - JaDogg 3y ago
    Hello thank you for pointing this out. I hate how slow grep is in Windows :( and I cannot install rg (I have no choice in the OS at work)
    - mholub 3y ago
      What’s the nature of your work so you are not allowed to install software?
      - JaDogg 3y ago
        I'm not allowed to install arbitrary software, only those that are provided by IT. It is just company policy and I cannot change that.
- M4v3R 3y ago
  I've always wondered how in world search in VS Code is so fast given it's an Electron app - now I know.
- Thaxll 3y ago
  It's not new it has been in vscode for 7 years.
- heap_perms 3y ago
  I did not know this that's actually very interesting.
larodi 3y ago
well... it is not faster than qgrep :) even though the way both work - differs greatly, and even though qgrep is based on re2 - the speed comes from the presence of index. but then I wonder why people forget the qgrep option, since with large file stores it makes much more sense to use qgrep AND indices, rather than always go through all the files. this above all true UNLESS you need multi-line matches with UTF8, where ripgrep is not so fast, because it needs to fall back to the other PCRE2 lib
- burntsushi 3y ago
  Author of ripgrep here. Yes, qgrep uses indexing, which will always give it a leg up over other tools that don't use indexing. But of course, now you need to setup and maintain an index. The UX isn't quite as simple as "just run a search." But there isn't much of a mystery here. Someone might neglect to use qgrep for exactly the same reason that "grep is fast enough for me" might prevent someone from using ripgrep. And indeed, "grep is fast enough" is very much true in some non-trivial fraction of cases. There are many many searches in which you won't be able to perceive the speed difference between ripgrep and grep, if any exists. And, analogously, the difference between qgrep and ripgrep. The cases I'm thinking of tend to be small haystacks. If you have only a small thing to search, then perhaps even the speed of a "naive" grep is fast enough. So if ripgrep, say, completes a search of the Linux kernel in under 100ms, is that annoying enough to push you towards a different kind of tool that uses indexing? Maybe, depends on what you're doing. But probably not for standard interactive usage. This is my interpretation anyway of your wonderment of (in your words) "why people forget the qgrep option." YMMV. I have flirted with the idea of adding indexing to ripgrep: https://github.com/BurntSushi/ripgrep/issues/1497 https://github.com/BurntSushi/ripgrep/issues/1497 > this above all true UNLESS you need multi-line matches with UTF8, where ripgrep is not so fast, because it needs to fall back to the other PCRE2 lib That's not true. Multiline searches certainly do not require PCRE2. I don't know what you mean by "with UTF8," but the default regex engine has Unicode support. PCRE2 is a fully optional dependency of ripgrep. You can build ripgrep without PCRE2 and it will still have multiline search support.
  - thechao 3y ago
bluedays 3y ago
I can't think of a single time I've used grep where I thought "I wish this was faster".
- sgarland 3y ago
  Multi-GB log files. Even with LC_ALL=C, grep is painfully slow.
  - bigstrat2003 3y ago
    That's probably true - but good Lord, one should probably do something to reduce the size of log files that large.
    - sgarland 3y ago
      DB logs can get HUGE. logrotate for them is currently daily. I briefly wanted to tune it to help alleviate the issue, but honestly it didn’t and doesn’t matter, given the infrequency with which they’re directly accessed. No risk of running out of disk space, and the DBAs like them how they are, so meh. There are other things to worry about.
- rurban 3y ago
  On extremely slow systems, such as Windows. There I can search in multiple repos only with rg.
- alkonaut 3y ago
  Even if the answer is instant, you have a 50% performance improvement in your search just from typing "rg" instead of "grep"! From my perspective it's a no brainer. I don't HAVE a grep (because I don't have a Unix) so when I install a grep, any grep, reaching for rg is natural. It's modern and maintained. I have no scripts anywhere that might expect grep to be called "grep". Of course if you already have a grep (e.g. you run Unix/Linux) then the story is different. Your system probably already has a grep. Replacing it takes effort and that effort needs to have some return.
jmarchello 3y ago
ripgrep is easily one of my most used and most loved tools. I use it directly and also have it set as my grpprg in neovim.
nikbackm 3y ago
One thing I wish ripgrep had is support for AND conditions.
- kmarc 3y ago
  Maybe I'm missing something but I only use it with AND conditions (usually in the form of 'foo 'bar and it only matches lines with foo AND bar both present)
  - burntsushi 3y ago
    A search for `rg -e foo -e bar` will return lines that match either foo or bar. Some lines may have both, but it isn't required. The standard way to run "AND" queries is through shell pipelines. That is, `rg foo | rg bar` will only print lines containing both. But composition usually comes with costs. The output reverts to the standard grep format and it doesn't interact nicely with contextual options like -C/--context. See: https://github.com/BurntSushi/ripgrep/issues/875 https://github.com/BurntSushi/ripgrep/issues/875
    - bhaak 3y ago
      As I have overloaded my rg with a customized rg alias, I can't pipe multiple rg calls. Otherwise it would look like this: # rg nokogiri | rg linux <stdin>:11:Gemfile.lock:647: nokogiri (1.15.5-x86_64-linux) But that is a me problem. The workaround is of course just to pipe into grep instead.
      - burntsushi 3y ago
        Or `\rg`, which will use the command directly and skip your alias.
        bhaak 3y ago
        Oh, look at that. Nice. Still losing the coloring but you can't have everything.
susam 3y ago
I use ripgrep with the Emacs packages project.el (comes out of the box) and dumb-jump (needs to be installed). This may not be the most popular way of using rg but I have been very pleased with the overall experience. All it takes is running package-install to install the dumb-jump package and configuring the following hook: (add-hook 'xref-backend-functions #'dumb-jump-xref-activate) The Xref key sequences and commands work fine with it. If I type M-. (or C-u M-.) to find definitions of an identifier in a Python project, dumb-jump runs a command like the following, processes the results, and displays the results in an Xref buffer. rg --color never --no-heading --line-number -U --pcre2 --type py '\s*\bfoo\s*=[^=\n]+|def\s*foo\b\s*\(|class\s*foo\b\s*\(?' /path/to/git/project/ The above command shows how dumb-jump automatically restricts the search to the current file type within the current project directory. If no project directory is found, it defaults to the home directory. By the way, dumb-jump supports the silver searcher tool ag too which happens to be quite fast as well. If neither ag nor rg is found, it defaults to grep which as one would expect can be quite slow while searching the whole home directory.
- susam 3y ago
  Addendum to my comment above: Ripgrep can be used quite easily with the project.el package too that comes out of the box in Emacs. So it is not really necessary to install an external package to make use of ripgrep within Emacs. We first need to configure xref-search-program to ripgrep as shown below, otherwise it defaults to grep which can be quite slow on large directories: (setq xref-search-program 'ripgrep) Then a project search with C-x p g foo RET ends up executing a command like the following on the current project directory: rg -i --null -nH --no-heading --no-messages -g '!*/' -e foo The results are displayed in an Xref buffer again which in my opinion is the best thing about using external search tools within Emacs. The Xref key sequences like n (next match), p (previous match), RET (jump to source of match), C-o (show the source of the match in a split window), etc. make navigating the results a breeze!
- burntsushi 3y ago
  Author of ripgrep here. Looking at your regex---just by inspection, I haven't tried it, so I could be wrong---but I think you can drop the --pcre2 flag. I also think you can drop the second and third \b assertion. You might need the first one though.
mariopt 3y ago
What are the reasons for grep not being replaced/improved? This topic seems a bit old by now.
- capableweb 3y ago
  There are multiple alternatives you can already use as an alternative, like ripgrep. What are you proposing, switching out the command `grep` for another utility? Sounds like that could introduce a ton of breakage, for little value. People who want a faster grep will use a different thing, while people who use grep can continue to use it. Sounds like an ideal situation already.
  - eviks 3y ago
    It still wastes time of all the people on the road to that realization, including the time spent looking for a better alternative
  - asicsp 3y ago
    Another reason would be if you want to follow POSIX standards. For example, `GNU grep` supports `POSIXLY_CORRECT` environment variable.
- anonymous_sorry 3y ago
  These benchmarking results are seven years old, so perhaps it has been. My entirely anecdotal and unscientific impression is that rg and grep perform similarly on Linux (though rg has nicer defaults for searching through source code). The old version of grep that Apple preinstalls on the Mac was slower last time I checked though.
- burntsushi 3y ago
  There's like a whole host of things you could use to explain it. Inertia. Compatibility. Resistance to change. Innovator's dilemma. And so on. (I do not say any of these things pejoratively! All of those things apply to me too.) With respect to compatibility, see my FAQ on the topic: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos...
abnry 3y ago
I messed up my conda environments once when I changed my username. So many references to the old username in the conda folders. Ripgrep saved the day!
- mrweasel 3y ago
  Why not just make new ones? Regardless of whether I use Conda, Pyenv or virtualenv, I always consider the environments to be disposable.
  - abnry 3y ago
    It takes forever (like an hour) to install the data science stack plus pytorch and cuda libraries plus other hardware related libraries.
- eska 3y ago
  fastmod is another nice and user-friendly Rust tool for such cases.
nihaan 3y ago
[flagged]
T3RMINATED 3y ago
[dead]
PaulDavisThe1st 3y ago
And this may still be true in 2023, but the problem is that most of the parallelized grep replacements (e.g. ripgrep, ag, etc.) are SO much faster than grep that the much small speed differences between them doesn't provide much of a basis for differentiating them. I use ag (typically from inside Emacs) on a 900k LOC codebase and it is effectively instantaneous (on a 16 core Ryzen Threadripper 2950X). I just don't have a need to go from less than 1 second to "a bit less than less than 1 second". Speed is not the defining attribute of the "new greps" - they need to be assessed and compared in other ways.
- burntsushi 3y ago
  In 2016, I'd say speed was definitely a defining attribute. ag has very significant performance cliffs. You can see them in the blog post. But as I mentioned in my comparison to qgrep elsewhere in the thread, everyone has different workloads. And for some workloads, perf differences might not matter. It really just depends. 900 KLOC isn't that big, and indeed, for simple queries pretty much any non-naive grep is going to chew through it very very quickly. As for comparisons in other ways, at least for ag, it's on life support. I thought it was going to get removed from Debian, but it looks like someone rescued it: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=999962 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=999962 The blog post also compares Unicode support, and contextualizes its performance. ag essentially has zero Unicode support. Unicode support isn't universally applicable of course---you may not care about it---but it satisfies your non-perf comparison criteria. :-)
- GoblinSlayer 3y ago
  In my experience they are all horribly i/o bound and the search takes as long as files load from disk and that's quite long, after that the difference can't possibly be meaningful. When files are in cache search time is dominated by time it takes me to navigate the file system and write the command, and again performance difference can't possibly be meaningful.
  - burntsushi 3y ago
    > When files are in cache search time is dominated by time it takes me to navigate the file system and write the command This suggests your corpora are small. If you have small corpora, then it should be absolutely no surprise that one tool taking 40ms and another taking 20ms will matter for standard interactive usage.
xezian 3y ago
I've been using ripgrep for about 2 years now and I find in indispensable. The main reason I switched from grep was ease of use. From the README: "By default, ripgrep will respect gitignore rules and automatically skip hidden files/directories and binary files." Typing `rg search_term directory` is much better than the corresponding grep command, but the speed improvement is also a nice bonus. Random other helpful flag I use often is -M if any of the matches are way too long to read through and cause a lot of terminal chaos. Just add `-M 1000` or adjust the number for your needs and the really long matches will omit the text context in the results.
- hhda 3y ago
  Yeah, the -M command is wonderful (super handy for ignoring minified files that you don't want to see results from, etc), and also great is the -g command (eg `-g *.cs` and you'll just search in files that have the .cs extension). Also the fact that it is a standalone portable executable can be super handy. Often when working on a new machine, I'll drop in the executable and an alias for grep that points to rg, so if muscle memory kicks in and I type grep it will still use rg.
  - 3PS 3y ago
    If you're a fan of the -g flag to ripgrep then I also recommend checking out the -t flag, short for --type, which lets you search specific file types. You can see the full list with `rg --type-list`. For example, you could just search .cs files with `rg -tcs`. This flag is especially convenient if you want to search e.g. .yml and .yaml in one go, or .c and .h in one go, etc.
    - kzrdude 3y ago
      -t is useful but -g doesn't require lookup in help first. Maybe the worse is better principle?
      - 3PS 3y ago
        Tbh I've always just typed -t followed by something that feels intuitive and it's always worked. Never really bothered looking in help until I made the above comment.
entropie 3y ago
I searched in portage, and it seems there is another version working also with other documents like PDFs and doc. https://github.com/phiresky/ripgrep-all https://github.com/phiresky/ripgrep-all
gquere 3y ago
Semi off-topic, I've coded a ncurses-frontend to navigate and filter grep-like results which might be of interest to some of you: https://github.com/gquere/ngp2 https://github.com/gquere/ngp2
- reegnz 3y ago
  Why not just use :grep in vim and navigate the vim quick fix list?
devnine 3y ago
I've been using ripgrep for the last year to quickly search massive database dumps. I compared it with grep and it's a game changer.
bawolff 3y ago
Grep is already pretty much instant so what does it matter?
- vasergen 3y ago
  did you try it on big dataset? there is a huge difference actually
- rpigab 3y ago
  I love grep, but ripgrep really improves on it. Very often, I don't want to look for files that aren't tracked under Git VC, and I'm not looking for matches in binary files, so by default, ripgrep does that, which can cut time by 99%. I used to grep in small dirs, now I can I can ripgrep in my whole home, not that I do it, but I can. That + Sourcegraph on master branch, and it makes searching for any other thing than plain text feel sooo slow (Atlassian Confluence and Jira, Google docs, etc.). Thank you so much Burntsushi and contributors!
Night_Thastus 3y ago
I love ripgrep for the speed and the more sane defaults. I use it nearly every day. For those just using it to search through a codebase, don't forget -F for string literals.
bhasi 3y ago
I've been using ag forever - will check out ripgrep.
zinodaur 3y ago
Would be nice if ripgrep was drop in compatible with grep. I'd feel like a dick writing a shell script for other people to use and forcing them to install a new grep
- burntsushi 3y ago
  Never will be: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos... If you need drop-in compatibility with grep, then use grep. :-)
shmerl 3y ago
Feels like it should always be included by default on grep level now.
ashton314 3y ago
Using Ripgrep via Consult [1] in Emacs is bliss. It's like the rg+fzf thing that some have made, but all inside Emacs. I use the `consult-ripgrep` command all the time, and sometimes I use it to make project-wide edits too! Workflow is search with `consult-ripgrep` -> export results to buffer -> edit buffer -> commit edits back to files. Details at [2] (includes video of me working it) [1]: https://github.com/minad/consult#grep-and-find https://github.com/minad/consult#grep-and-find [2]: https://lambdaland.org/posts/2023-05-31_warp_factor_refactor/ https://lambdaland.org/posts/2023-05-31_warp_factor_refactor...
pie_flavor 3y ago
I love ripgrep, it's searched a directory in a half-second for a pattern that took GNU grep literally fifteen minutes.
Liam2010 3y ago
[dead]

390 points by subset 3y ago

latexr 3y ago

The title needs “(2016)”. This is the original announcement, not new information.

asicsp 3y ago
Discussions: "Ripgrep – A new command line search tool" https://news.ycombinator.com/item?id=12564442 https://news.ycombinator.com/item?id=12564442 (740 points | Sept 23, 2016 | 209 comments) - there are discussions related to speed too "Ripgrep is faster (2016)" https://news.ycombinator.com/item?id=17941319 https://news.ycombinator.com/item?id=17941319 (98 points | Sept 8, 2018 | 40 comments)

stinos 3y ago

It's fast indeed. And I can't help keeping promoting the combination with fzf :) For those who want to try it out, this is a Powershell function but the same principle applies in any shell. Does ripgrep then puts fuzzy searching in the resulting files+text on top while showing context in bat: function frg { $result = rg --ignore-case --color=always --line-number --no-heading @Args | fzf --ansi ` --color 'hl:-1:underline,hl+:-1:underline:reverse' ` --delimiter ':' ` --preview "bat --color=always {1} --theme='Solarized (light)' --highlight-line {2}" ` --preview-window 'up,60%,border-bottom,+{2}+3/3,~3' if ($result) { & ($env:EDITOR).Trim("`"'") $result.Split(': ')[0] } } There are other ways to approach this, but for me this is a very fast way of nailing down 'I now something exists in this multi-repo project but don't know where exactly nor the exact name' edit this comes out of https://github.com/junegunn/fzf/blob/master/ADVANCED.md https://github.com/junegunn/fzf/blob/master/ADVANCED.md and even though you might not want to use most of what is in there, it's still worth glancing over it to get ideas of what you could do with it

pknopf 3y ago
Thanks for this!
majkinetor 3y ago
Not bad. For those who want to try it, install all prereqs with: choco install fzf bat ripgrep How do you scroll the preview window with keyboard ?
- stinos 3y ago
  shift-up/down
- nickjj 3y ago
  > How do you scroll the preview window with keyboard ? alias pf="fzf --preview='less {}' --bind shift-up:preview-page-up,shift-down:preview-page-down" That will let you run `pf` to preview files in less and lets you use shift + arrow keys to scroll the preview window. No dependencies are needed except for fzf. If you want to use ripgrep with fzf you can set FZF_DEFAULT_COMMAND to run rg such as `export FZF_DEFAULT_COMMAND="rg ..."` where ... are your preferred rg flags. This full setup is in my dotfiles at https://github.com/nickjj/dotfiles https://github.com/nickjj/dotfiles. I've made a video and blog post about it here: https://nickjanetakis.com/blog/customize-fzf-ctrl-t-binding-to-preview-files https://nickjanetakis.com/blog/customize-fzf-ctrl-t-binding-... I also made https://nickjanetakis.com/blog/using-fzf-to-preview-text-files-on-the-command-line-and-within-vim https://nickjanetakis.com/blog/using-fzf-to-preview-text-fil... which covers how to modify fzf's built in CTRL+t shortcut to allow for previews too. CTRL+t is a hotkey driven way to fuzzy match a list of files.
wanderingmind 3y ago
Infact I would recommend a step further to integrate rip-grep-all (rga) with fzf that can do a fuzzy search not just on text files but on all types of files including pdfs, zip files. More details here [1] [1] https://github.com/phiresky/ripgrep-all/wiki/fzf-Integration https://github.com/phiresky/ripgrep-all/wiki/fzf-Integration
- fsiefken 3y ago
  That's really nice, thanks. Long ago there was the Google Desktop Search where you could 'Google' your local documents. But the difference is that that worked with an index, so I imagine it's faster if you have thousands of pdfs en epubs.
  - cb321 3y ago
    Even longer ago, there was `glimpse`: https://www.linuxjournal.com/article/1164 https://www.linuxjournal.com/article/1164 which is still available. [1] glimpse's index-builds are like 10X slower than `qgrep` mentioned elsethread. `qgrep` also seems to have faster search (though I only tried a few patterns) and `qgrep` does not allow spelling errors like `glimpse`. Neither `glimpse` nor `qgrep`, to my knowledge, directly supports pre-processing / document conversion (like `pdftotext`), though I imagine this would be easy to add to either replicating Desktop Search. (Indirectly, at some space cost, you could always dump conversions into a shadow file hierarchy, index that, and then translate path names.) [1] https://manpages.ubuntu.com/manpages/focal/man1/glimpse.1.html https://manpages.ubuntu.com/manpages/focal/man1/glimpse.1.ht...
tambourine_man 3y ago
Vim is almost broken for me without fzf+rg. Feels like I’m manually grinding coffee instead of using electricity.
- stirfish 3y ago
  This comment got me to get out my French press and manually grind some beans. It wasn't a meditative and calming as I remember, and the coffee tastes a little...dusty. I guess it's time for me to update my vimrc.
  - orangepurple 3y ago
    Aeropress is a direct upgrade from french press and uses way less coffee
    - stirfish 3y ago
      I hadn't heard of this before. Thanks!
      - orangepurple 3y ago
        Add one or a few drops of water to your roasted coffee beans with your hand and shake well after weighing it out to stop the grinds from sticking to the walls of your grinder from static.
        iknowstuff 3y ago
        This thread took a turn
        bch 3y ago
        I love that a ripgrep article has such a deeply nerdy coffee thread… > Add one or a few drops of water to your roasted coffee beans Ah, RDT (Ross Droplet Technique)[0]. A little atomizer (“spritz” bottle) of plain water serves well here. NB: this is for single-dose grinding - e.g. measuring a small amount of beans loaded into a grinder to grind immediately. If you have a grinder with a “big” hopper on top that has (e.g.) the weeks worth of coffee (even though you grind on-demand for ea. espresso/french press/aeropress/pourover/drip/…) this isn’t for you. [0] https://thebasicbarista.com/en-us/blogs/topics/how-rdt-broke-changed-the-game-benefits-and-reasons-why-people-spray-their-coffee-beans https://thebasicbarista.com/en-us/blogs/topics/how-rdt-broke...
shoover 3y ago
Oh. Thanks for the tip. This might make me finally embrace powershell. I’ve been using WSL+zsh+fzf as a Windows CLI for continuity with day job Mac tools, but git CLI performance is only usable inside the WSL file system.
- fhlu 3y ago
  You can also add a small script to your WSL under `/usr/local/bin/git`: GIT_WINDOWS="/mnt/c/Program Files/Git/bin/git.exe" GIT_LINUX="/usr/bin/git" case "$(pwd -P)" in /mnt/?/*) case "$@" in # Needed to fix prompt, but it breaks things like paging, colours, etc rev-parse*) # running linux git for rev-parse seems faster, even without translating paths exec "$GIT_LINUX" "$@" ;; *) exec "$GIT_WINDOWS" -c color.ui=always "$@" ;; esac ;; *) exec "$GIT_LINUX" "$@" ;; esac This allows you to use `git` in your WSL shell but it'll pick whichever executable is suitable for the filesystem that the repo is in :)
  - shoover 3y ago
    Thank you. I will use this.
    - ciroduran 3y ago
      The code as written above only works if you haven't changed the mountpoint for your windows partition (i.e. from /mnt), consider that
- stinos 3y ago
  This might make me finally embrace powershell Yeah, I have a bit of a love-hate relationship with it. But I actually have that with all shells out there. I don't know if it's just me or the shells, or (the most likely I think): a bit of both. But PS is available out of the box and using objects vs plain text is a major win in my book, and even though I still don't know half of the syntax by heart it feels less of an endless fight than other shells. And since I use the shell itself for rather basic things and for the rest only for tools (like shown here), we get along just fine.
imakira 3y ago
I wrote a bash version of this: function frg { result=`rg --ignore-case --color=always --line-number --no-heading "$@" | fzf --ansi \ --color 'hl:-1:underline,hl+:-1:underline:reverse' \ --delimiter ':' \ --preview "bat --color=always {1} --theme='Solarized (light)' --highlight-line {2}" \ --preview-window 'up,60%,border-bottom,+{2}+3/3,~3'` file="${result%%:*}" linenumber=`echo "${result}" | cut -d: -f2` if [ ! -z "$file" ]; then $EDITOR +"${linenumber}" "$file" fi }
- zero_k 3y ago
  Wow, nice, thanks! :heart:
- seanw444 3y ago
  I've never really seen PowerShell beyond minimal commands, but after seeing the parent, I definitely think it has the superior syntax of the shells. Especially for scripts.
  - bbkane 3y ago
    Maybe if the "popular" shells, but http://www.nushell.sh/ http://www.nushell.sh/ is looking better and better
    - evntdrvn 3y ago
      nu acknowledged powershell as one of its inspirations, yeah!!
    - steveklabnik 3y ago
      I use it near exclusively. Love nu.
  - Natfan 3y ago
    A lot of people sleep on PowerShell, possibly because some of the syntax is a little clunky (and quite slow compared to some other shells, I will freely admit). That being said, I'd argue object oriented programming is a massive improvement over text oriented programming. I never want to touch awk again!
bloopernova 3y ago
With fzf you can add lots of files to git while skipping some if you want: fza = "!git ls-files -m -o --exclude-standard | fzf -m --print0 | xargs -0 git add" With that in the [alias] section of a gitconfig file, running git fza brings up a list of modified and not yet added files, space toggles each entry and moves to the next entry. That alias as well as fzf+fd really speed up some parts of my workflow. Oh and shameless plug for my guide on what to include in your zsh setup on macOS: https://gist.github.com/aclarknexient/0ffcb98aa262c585c49d4b3f3ae24019 https://gist.github.com/aclarknexient/0ffcb98aa262c585c49d4b...
- stinos 3y ago
  Add the preview to see what you're actually stashing: git ls-files -m -o --exclude-standard | fzf -m --print0 --preview "git diff {1}" | .... And that's just the start: it could even be that by binding a key to the fzf reload command to then display the diff in it's finder, and in turn a key to stage the selected line, you could turn that into an interactive git staging tool.
  - bloopernova 3y ago
    Nice. I'll have to try that out!
- kstrauser 3y ago
  That blew my mind. I've used fzf a couple time here and there, but now I Get It. Thanks!
w0m 3y ago
This is a gem; thank you.
taude 3y ago
This is pretty much my exact use of ripgrep, too. I use it as a starting point to zero in on files/projects in a several-hundred repo codebase, and then go from there....
shmerl 3y ago
I use it neovim with fzf search.

stabbles 3y ago

But is any of them using `_mm256_sad_epu8` for small, literal strings?

lifthrasiir 3y ago
Not exactly but yes, it ultimately uses the `memchr` crate [1] which provides SIMD-optimized character and string search routines. But it uses `_mm256_cmpeq_epi8` instead of `_mm256_sad_epu8`. [1] https://docs.rs/memchr/latest/memchr/ https://docs.rs/memchr/latest/memchr/
- burntsushi 3y ago
  And also the SIMD in aho-corasick, which is used whenever a small number of literals are searched for. For example, `foo|bar` or `(?i)foo`. https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56b3f28e9dd69a655316af0107b29b/src/packed/vector.rs https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56... But no `_mm256_sad_epu8`. What an oddly specific question..?
  - stabbles 3y ago
    What is really oddly specific is this instruction :p but I believe it is a common trick to quickly scan for short string matches. It computes `sum(|x[i] - y[i]|)` for consecutive `i` at different offsets, so it should be zero at substring matches. For context: https://epubs.siam.org/doi/pdf/10.1137/1.9781611972931.10 https://epubs.siam.org/doi/pdf/10.1137/1.9781611972931.10 I was slightly mistaken, the instruction of interest is _mm256_mpsadbw_epu8
    - burntsushi 3y ago
      Oh I see. Yes, that's what is commonly used in academic publications. But I've yet to see it used in the wild. I mentioned exactly that paper (I believe) in my write-up on Teddy: https://github.com/BurntSushi/aho-corasick/tree/master/src/packed/teddy https://github.com/BurntSushi/aho-corasick/tree/master/src/p...
    - 3y ago

zoobab 3y ago

git grep cannot even find a simple string in its repo.

ctenb 3y ago
What? Git grep is all you ever need in my experience, and it's ~faster than~ (edit: as fast as) ripgrep when searching a git repo.
- dharmab 3y ago
  > it's faster than ripgrep when searching a git repo. Source? Ripgrep's benchmarks show it significantly faster.
  - ctenb 3y ago
    See sibling comment
- larodi 3y ago
  I really doubt git grep can outperform ripgrep in any tests... please provide some proof.
  - ctenb 3y ago
    I tested this on a large repo in 2016 when I installed several tools (including rg and ag) to compare speed. I don't have the metrics anymore, but the results were pretty clear then. According to the benchmarks from the OP, git grep is pretty comparable to rg in a large git repo. I guess different benchmarks give slightly different results, but the OP acknowledges that git grep is very fast. Bonus is that it comes preinstalled with git and can search through commit history.
    - burntsushi 3y ago
      Author of ripgrep here. It really just depends. The way I like to characterize `git grep` (at present) is that it has sharp performance cliffs. ripgrep has them too, to be sure, but I think it has fewer of them. If you're just searching for a simple literal, `git grep` is decently fast: $ git remote -v origin git@github.com:torvalds/linux (fetch) origin git@github.com:torvalds/linux (push) $ git rev-parse HEAD f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 $ time LC_ALL=en_US.UTF-8 git grep -c -E 'PM_RESUME' Documentation/dev-tools/sparse.rst:3 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 arch/arm/mach-omap2/omap-secure.h:1 arch/arm/mach-omap2/pm33xx-core.c:1 arch/x86/kernel/apm_32.c:1 drivers/input/mouse/cyapa.h:1 drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/usb/mtu3/mtu3_hw_regs.h:1 include/uapi/linux/apm_bios.h:1 real 0.215 user 0.421 sys 1.226 maxmem 161 MB faults 0 $ time rg -c 'PM_RESUME' drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/usb/mtu3/mtu3_hw_regs.h:1 drivers/input/mouse/cyapa.h:1 arch/x86/kernel/apm_32.c:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 Documentation/dev-tools/sparse.rst:3 arch/arm/mach-omap2/pm33xx-core.c:1 arch/arm/mach-omap2/omap-secure.h:1 include/uapi/linux/apm_bios.h:1 real 0.078 user 0.259 sys 0.577 maxmem 15 MB faults 0 But if you switch it up and start adding regex things to your pattern, there can be substantial slowdowns: $ time LC_ALL=C git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 5.704 user 55.671 sys 0.585 maxmem 207 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 24.529 user 4:34.42 sys 0.753 maxmem 211 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -P '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 1.372 user 16.980 sys 0.647 maxmem 211 MB faults 1 $ time rg -c '\w{5,}\s+PM_RESUME' Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 0.082 user 0.226 sys 0.612 maxmem 18 MB faults 0 In the above cases, ripgrep has Unicode enabled. (It's enabled by default irrespective of locale settings. ripgrep doesn't interact with POSIX locales at all.)

nvoeiah 3y ago

[flagged]

WJW 3y ago
So? My distro doesn't come with almost any of the tools I need for day to day work. It has never been a problem for me to install a new editor or compiler on a new machine, I don't see why ripgrep would be any different. Especially since it's usually a single command to install anyway.
deleted 3y ago
[deleted]

jedisct1 3y ago

I switched from from ripgrep to ugrep and never looked back. It's just as fast, but also comes with fuzzy matching (which is super useful), a TUI (useful for code reviews), and can also search in PDFs, archives, etc. The optional Google search syntax also very convenient. https://ugrep.com https://ugrep.com

ranting-moth 3y ago
I'm scared that if I start using Google search syntax in my grepping that I'll mostly get results trying to sell me something :)
hnfong 3y ago
So I was casually searching for "ugrep vs ripgrep" articles, when I stumbled upon a couple reddit posts where apparently the authors of ugrep and ripgrep seemed to have a multi-year feud on reddit, eg. https://www.reddit.com/r/programming/comments/120wqvr/ripgrep_is_faster_than_grep_ag_git_grep_ucg_pt/jdn2ybv/?context=3 https://www.reddit.com/r/programming/comments/120wqvr/ripgre... So weird. I mean, it's just about some open source tool, right? :-/
- GuB-42 3y ago
  Psst, don't tell him about emacs and vi(m)... There are feuds about open source tools all the time. Text editors, Linux distros, shells, programming languages, desktop environments, etc... And ugrep vs ripgrep may be a poster child for C++ vs Rust. It is not all bad, it drives progress, and it usually stays at a technical level, I've yet to see people killing each others for their choice of command line search tool.
- fliife 3y ago
  This is so weird, even ripgrep's author is actively seeking conflict in ugrep's new release posts. Not a good colour on both of them.
  - orlp 3y ago
    At least how I read it the linked post was ugrep's author seeking conflict, not the ripgrep's.

MrHamdulay 3y ago

What's interesting is that ripgrep now also powers VS Code search with a Node.js wrapper. https://www.npmjs.com/package/@vscode/ripgrep https://www.npmjs.com/package/@vscode/ripgrep

porwah 3y ago
...which is awesome if you can request/install VS Code but not ripgrep. You can find the rg binary in the VS installation (at least, I can on Windows at my place of employment).
- JaDogg 3y ago
  Hello thank you for pointing this out. I hate how slow grep is in Windows :( and I cannot install rg (I have no choice in the OS at work)
  - mholub 3y ago
    What’s the nature of your work so you are not allowed to install software?
    - JaDogg 3y ago
      I'm not allowed to install arbitrary software, only those that are provided by IT. It is just company policy and I cannot change that.
M4v3R 3y ago
I've always wondered how in world search in VS Code is so fast given it's an Electron app - now I know.
Thaxll 3y ago
It's not new it has been in vscode for 7 years.
heap_perms 3y ago
I did not know this that's actually very interesting.

larodi 3y ago

well... it is not faster than qgrep :) even though the way both work - differs greatly, and even though qgrep is based on re2 - the speed comes from the presence of index. but then I wonder why people forget the qgrep option, since with large file stores it makes much more sense to use qgrep AND indices, rather than always go through all the files. this above all true UNLESS you need multi-line matches with UTF8, where ripgrep is not so fast, because it needs to fall back to the other PCRE2 lib

burntsushi 3y ago
Author of ripgrep here. Yes, qgrep uses indexing, which will always give it a leg up over other tools that don't use indexing. But of course, now you need to setup and maintain an index. The UX isn't quite as simple as "just run a search." But there isn't much of a mystery here. Someone might neglect to use qgrep for exactly the same reason that "grep is fast enough for me" might prevent someone from using ripgrep. And indeed, "grep is fast enough" is very much true in some non-trivial fraction of cases. There are many many searches in which you won't be able to perceive the speed difference between ripgrep and grep, if any exists. And, analogously, the difference between qgrep and ripgrep. The cases I'm thinking of tend to be small haystacks. If you have only a small thing to search, then perhaps even the speed of a "naive" grep is fast enough. So if ripgrep, say, completes a search of the Linux kernel in under 100ms, is that annoying enough to push you towards a different kind of tool that uses indexing? Maybe, depends on what you're doing. But probably not for standard interactive usage. This is my interpretation anyway of your wonderment of (in your words) "why people forget the qgrep option." YMMV. I have flirted with the idea of adding indexing to ripgrep: https://github.com/BurntSushi/ripgrep/issues/1497 https://github.com/BurntSushi/ripgrep/issues/1497 > this above all true UNLESS you need multi-line matches with UTF8, where ripgrep is not so fast, because it needs to fall back to the other PCRE2 lib That's not true. Multiline searches certainly do not require PCRE2. I don't know what you mean by "with UTF8," but the default regex engine has Unicode support. PCRE2 is a fully optional dependency of ripgrep. You can build ripgrep without PCRE2 and it will still have multiline search support.
- thechao 3y ago

bluedays 3y ago

I can't think of a single time I've used grep where I thought "I wish this was faster".

sgarland 3y ago
Multi-GB log files. Even with LC_ALL=C, grep is painfully slow.
- bigstrat2003 3y ago
  That's probably true - but good Lord, one should probably do something to reduce the size of log files that large.
  - sgarland 3y ago
    DB logs can get HUGE. logrotate for them is currently daily. I briefly wanted to tune it to help alleviate the issue, but honestly it didn’t and doesn’t matter, given the infrequency with which they’re directly accessed. No risk of running out of disk space, and the DBAs like them how they are, so meh. There are other things to worry about.
rurban 3y ago
On extremely slow systems, such as Windows. There I can search in multiple repos only with rg.
alkonaut 3y ago
Even if the answer is instant, you have a 50% performance improvement in your search just from typing "rg" instead of "grep"! From my perspective it's a no brainer. I don't HAVE a grep (because I don't have a Unix) so when I install a grep, any grep, reaching for rg is natural. It's modern and maintained. I have no scripts anywhere that might expect grep to be called "grep". Of course if you already have a grep (e.g. you run Unix/Linux) then the story is different. Your system probably already has a grep. Replacing it takes effort and that effort needs to have some return.

jmarchello 3y ago

ripgrep is easily one of my most used and most loved tools. I use it directly and also have it set as my grpprg in neovim.

nikbackm 3y ago

One thing I wish ripgrep had is support for AND conditions.

kmarc 3y ago
Maybe I'm missing something but I only use it with AND conditions (usually in the form of 'foo 'bar and it only matches lines with foo AND bar both present)
- burntsushi 3y ago
  A search for `rg -e foo -e bar` will return lines that match either foo or bar. Some lines may have both, but it isn't required. The standard way to run "AND" queries is through shell pipelines. That is, `rg foo | rg bar` will only print lines containing both. But composition usually comes with costs. The output reverts to the standard grep format and it doesn't interact nicely with contextual options like -C/--context. See: https://github.com/BurntSushi/ripgrep/issues/875 https://github.com/BurntSushi/ripgrep/issues/875
  - bhaak 3y ago
    As I have overloaded my rg with a customized rg alias, I can't pipe multiple rg calls. Otherwise it would look like this: # rg nokogiri | rg linux <stdin>:11:Gemfile.lock:647: nokogiri (1.15.5-x86_64-linux) But that is a me problem. The workaround is of course just to pipe into grep instead.
    - burntsushi 3y ago
      Or `\rg`, which will use the command directly and skip your alias.
      - bhaak 3y ago
        Oh, look at that. Nice. Still losing the coloring but you can't have everything.

susam 3y ago

I use ripgrep with the Emacs packages project.el (comes out of the box) and dumb-jump (needs to be installed). This may not be the most popular way of using rg but I have been very pleased with the overall experience. All it takes is running package-install to install the dumb-jump package and configuring the following hook: (add-hook 'xref-backend-functions #'dumb-jump-xref-activate) The Xref key sequences and commands work fine with it. If I type M-. (or C-u M-.) to find definitions of an identifier in a Python project, dumb-jump runs a command like the following, processes the results, and displays the results in an Xref buffer. rg --color never --no-heading --line-number -U --pcre2 --type py '\s*\bfoo\s*=[^=\n]+|def\s*foo\b\s*\(|class\s*foo\b\s*\(?' /path/to/git/project/ The above command shows how dumb-jump automatically restricts the search to the current file type within the current project directory. If no project directory is found, it defaults to the home directory. By the way, dumb-jump supports the silver searcher tool ag too which happens to be quite fast as well. If neither ag nor rg is found, it defaults to grep which as one would expect can be quite slow while searching the whole home directory.

susam 3y ago
Addendum to my comment above: Ripgrep can be used quite easily with the project.el package too that comes out of the box in Emacs. So it is not really necessary to install an external package to make use of ripgrep within Emacs. We first need to configure xref-search-program to ripgrep as shown below, otherwise it defaults to grep which can be quite slow on large directories: (setq xref-search-program 'ripgrep) Then a project search with C-x p g foo RET ends up executing a command like the following on the current project directory: rg -i --null -nH --no-heading --no-messages -g '!*/' -e foo The results are displayed in an Xref buffer again which in my opinion is the best thing about using external search tools within Emacs. The Xref key sequences like n (next match), p (previous match), RET (jump to source of match), C-o (show the source of the match in a split window), etc. make navigating the results a breeze!
burntsushi 3y ago
Author of ripgrep here. Looking at your regex---just by inspection, I haven't tried it, so I could be wrong---but I think you can drop the --pcre2 flag. I also think you can drop the second and third \b assertion. You might need the first one though.

mariopt 3y ago

What are the reasons for grep not being replaced/improved? This topic seems a bit old by now.

capableweb 3y ago
There are multiple alternatives you can already use as an alternative, like ripgrep. What are you proposing, switching out the command `grep` for another utility? Sounds like that could introduce a ton of breakage, for little value. People who want a faster grep will use a different thing, while people who use grep can continue to use it. Sounds like an ideal situation already.
- eviks 3y ago
  It still wastes time of all the people on the road to that realization, including the time spent looking for a better alternative
- asicsp 3y ago
  Another reason would be if you want to follow POSIX standards. For example, `GNU grep` supports `POSIXLY_CORRECT` environment variable.
anonymous_sorry 3y ago
These benchmarking results are seven years old, so perhaps it has been. My entirely anecdotal and unscientific impression is that rg and grep perform similarly on Linux (though rg has nicer defaults for searching through source code). The old version of grep that Apple preinstalls on the Mac was slower last time I checked though.
burntsushi 3y ago
There's like a whole host of things you could use to explain it. Inertia. Compatibility. Resistance to change. Innovator's dilemma. And so on. (I do not say any of these things pejoratively! All of those things apply to me too.) With respect to compatibility, see my FAQ on the topic: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos...

abnry 3y ago

I messed up my conda environments once when I changed my username. So many references to the old username in the conda folders. Ripgrep saved the day!

mrweasel 3y ago
Why not just make new ones? Regardless of whether I use Conda, Pyenv or virtualenv, I always consider the environments to be disposable.
- abnry 3y ago
  It takes forever (like an hour) to install the data science stack plus pytorch and cuda libraries plus other hardware related libraries.
eska 3y ago
fastmod is another nice and user-friendly Rust tool for such cases.

nihaan 3y ago

[flagged]

T3RMINATED 3y ago

[dead]

PaulDavisThe1st 3y ago

And this may still be true in 2023, but the problem is that most of the parallelized grep replacements (e.g. ripgrep, ag, etc.) are SO much faster than grep that the much small speed differences between them doesn't provide much of a basis for differentiating them. I use ag (typically from inside Emacs) on a 900k LOC codebase and it is effectively instantaneous (on a 16 core Ryzen Threadripper 2950X). I just don't have a need to go from less than 1 second to "a bit less than less than 1 second". Speed is not the defining attribute of the "new greps" - they need to be assessed and compared in other ways.

burntsushi 3y ago
In 2016, I'd say speed was definitely a defining attribute. ag has very significant performance cliffs. You can see them in the blog post. But as I mentioned in my comparison to qgrep elsewhere in the thread, everyone has different workloads. And for some workloads, perf differences might not matter. It really just depends. 900 KLOC isn't that big, and indeed, for simple queries pretty much any non-naive grep is going to chew through it very very quickly. As for comparisons in other ways, at least for ag, it's on life support. I thought it was going to get removed from Debian, but it looks like someone rescued it: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=999962 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=999962 The blog post also compares Unicode support, and contextualizes its performance. ag essentially has zero Unicode support. Unicode support isn't universally applicable of course---you may not care about it---but it satisfies your non-perf comparison criteria. :-)
GoblinSlayer 3y ago
In my experience they are all horribly i/o bound and the search takes as long as files load from disk and that's quite long, after that the difference can't possibly be meaningful. When files are in cache search time is dominated by time it takes me to navigate the file system and write the command, and again performance difference can't possibly be meaningful.
- burntsushi 3y ago
  > When files are in cache search time is dominated by time it takes me to navigate the file system and write the command This suggests your corpora are small. If you have small corpora, then it should be absolutely no surprise that one tool taking 40ms and another taking 20ms will matter for standard interactive usage.

xezian 3y ago

I've been using ripgrep for about 2 years now and I find in indispensable. The main reason I switched from grep was ease of use. From the README: "By default, ripgrep will respect gitignore rules and automatically skip hidden files/directories and binary files." Typing `rg search_term directory` is much better than the corresponding grep command, but the speed improvement is also a nice bonus. Random other helpful flag I use often is -M if any of the matches are way too long to read through and cause a lot of terminal chaos. Just add `-M 1000` or adjust the number for your needs and the really long matches will omit the text context in the results.

hhda 3y ago
Yeah, the -M command is wonderful (super handy for ignoring minified files that you don't want to see results from, etc), and also great is the -g command (eg `-g *.cs` and you'll just search in files that have the .cs extension). Also the fact that it is a standalone portable executable can be super handy. Often when working on a new machine, I'll drop in the executable and an alias for grep that points to rg, so if muscle memory kicks in and I type grep it will still use rg.
- 3PS 3y ago
  If you're a fan of the -g flag to ripgrep then I also recommend checking out the -t flag, short for --type, which lets you search specific file types. You can see the full list with `rg --type-list`. For example, you could just search .cs files with `rg -tcs`. This flag is especially convenient if you want to search e.g. .yml and .yaml in one go, or .c and .h in one go, etc.
  - kzrdude 3y ago
    -t is useful but -g doesn't require lookup in help first. Maybe the worse is better principle?
    - 3PS 3y ago
      Tbh I've always just typed -t followed by something that feels intuitive and it's always worked. Never really bothered looking in help until I made the above comment.

entropie 3y ago

I searched in portage, and it seems there is another version working also with other documents like PDFs and doc. https://github.com/phiresky/ripgrep-all https://github.com/phiresky/ripgrep-all

gquere 3y ago

Semi off-topic, I've coded a ncurses-frontend to navigate and filter grep-like results which might be of interest to some of you: https://github.com/gquere/ngp2 https://github.com/gquere/ngp2

reegnz 3y ago
Why not just use :grep in vim and navigate the vim quick fix list?

devnine 3y ago

I've been using ripgrep for the last year to quickly search massive database dumps. I compared it with grep and it's a game changer.

bawolff 3y ago

Grep is already pretty much instant so what does it matter?

vasergen 3y ago
did you try it on big dataset? there is a huge difference actually
rpigab 3y ago
I love grep, but ripgrep really improves on it. Very often, I don't want to look for files that aren't tracked under Git VC, and I'm not looking for matches in binary files, so by default, ripgrep does that, which can cut time by 99%. I used to grep in small dirs, now I can I can ripgrep in my whole home, not that I do it, but I can. That + Sourcegraph on master branch, and it makes searching for any other thing than plain text feel sooo slow (Atlassian Confluence and Jira, Google docs, etc.). Thank you so much Burntsushi and contributors!

Night_Thastus 3y ago

I love ripgrep for the speed and the more sane defaults. I use it nearly every day. For those just using it to search through a codebase, don't forget -F for string literals.

bhasi 3y ago

I've been using ag forever - will check out ripgrep.

zinodaur 3y ago

Would be nice if ripgrep was drop in compatible with grep. I'd feel like a dick writing a shell script for other people to use and forcing them to install a new grep

burntsushi 3y ago
Never will be: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos... If you need drop-in compatibility with grep, then use grep. :-)

shmerl 3y ago

Feels like it should always be included by default on grep level now.

ashton314 3y ago

Using Ripgrep via Consult [1] in Emacs is bliss. It's like the rg+fzf thing that some have made, but all inside Emacs. I use the `consult-ripgrep` command all the time, and sometimes I use it to make project-wide edits too! Workflow is search with `consult-ripgrep` -> export results to buffer -> edit buffer -> commit edits back to files. Details at [2] (includes video of me working it) [1]: https://github.com/minad/consult#grep-and-find https://github.com/minad/consult#grep-and-find [2]: https://lambdaland.org/posts/2023-05-31_warp_factor_refactor/ https://lambdaland.org/posts/2023-05-31_warp_factor_refactor...

pie_flavor 3y ago

I love ripgrep, it's searched a directory in a half-second for a pattern that took GNU grep literally fifteen minutes.

Liam2010 3y ago

[dead]