4 ms·

git grep cannot even find a simple string in its repo.

by zoobab 3y ago

ctenb 3y ago
What? Git grep is all you ever need in my experience, and it's ~faster than~ (edit: as fast as) ripgrep when searching a git repo.
- dharmab 3y ago
  > it's faster than ripgrep when searching a git repo. Source? Ripgrep's benchmarks show it significantly faster.
  - ctenb 3y ago
    See sibling comment
- larodi 3y ago
  I really doubt git grep can outperform ripgrep in any tests... please provide some proof.
  - ctenb 3y ago
    I tested this on a large repo in 2016 when I installed several tools (including rg and ag) to compare speed. I don't have the metrics anymore, but the results were pretty clear then. According to the benchmarks from the OP, git grep is pretty comparable to rg in a large git repo. I guess different benchmarks give slightly different results, but the OP acknowledges that git grep is very fast. Bonus is that it comes preinstalled with git and can search through commit history.
    - burntsushi 3y ago
      Author of ripgrep here. It really just depends. The way I like to characterize `git grep` (at present) is that it has sharp performance cliffs. ripgrep has them too, to be sure, but I think it has fewer of them. If you're just searching for a simple literal, `git grep` is decently fast: $ git remote -v origin git@github.com:torvalds/linux (fetch) origin git@github.com:torvalds/linux (push) $ git rev-parse HEAD f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 $ time LC_ALL=en_US.UTF-8 git grep -c -E 'PM_RESUME' Documentation/dev-tools/sparse.rst:3 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 arch/arm/mach-omap2/omap-secure.h:1 arch/arm/mach-omap2/pm33xx-core.c:1 arch/x86/kernel/apm_32.c:1 drivers/input/mouse/cyapa.h:1 drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/usb/mtu3/mtu3_hw_regs.h:1 include/uapi/linux/apm_bios.h:1 real 0.215 user 0.421 sys 1.226 maxmem 161 MB faults 0 $ time rg -c 'PM_RESUME' drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/usb/mtu3/mtu3_hw_regs.h:1 drivers/input/mouse/cyapa.h:1 arch/x86/kernel/apm_32.c:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 Documentation/dev-tools/sparse.rst:3 arch/arm/mach-omap2/pm33xx-core.c:1 arch/arm/mach-omap2/omap-secure.h:1 include/uapi/linux/apm_bios.h:1 real 0.078 user 0.259 sys 0.577 maxmem 15 MB faults 0 But if you switch it up and start adding regex things to your pattern, there can be substantial slowdowns: $ time LC_ALL=C git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 5.704 user 55.671 sys 0.585 maxmem 207 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 24.529 user 4:34.42 sys 0.753 maxmem 211 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -P '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 1.372 user 16.980 sys 0.647 maxmem 211 MB faults 1 $ time rg -c '\w{5,}\s+PM_RESUME' Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 0.082 user 0.226 sys 0.612 maxmem 18 MB faults 0 In the above cases, ripgrep has Unicode enabled. (It's enabled by default irrespective of locale settings. ripgrep doesn't interact with POSIX locales at all.)
      - ctenb 3y ago
        Thanks for clarifying! I use `git grep -IPn --color=always --recurse-submodules` many times a day, every day. I hasn't yet let me down, but I don't search for unicode when working on source code. I do use regex though, using the -P switch.
  - kzrdude 3y ago
    git grep is shallow: only current git repo, rg is fully recursive, all submodules and also untracked (and not ignored) directories. In some trees, git grep will be a lot faster because it searches a smaller part of it.
    - ctenb 3y ago
      No, git grep can recurse, if you pass the flag, just like all other git commands. --recurse-submodules
masklinn 3y ago
I don’t think there’s been a point to using `git grep` since ack started parsing gitignore. As far as I’m concerned the use case of `git grep` is to search into non-checked-out trees (by giving it a tree-ish). And it’s not super great at that, because it searches a static tree-ish, so pickaxe filters are generally more useful (though they’re slow). Once again mercurial has/had more useful defaults, `hg grep` searches through the history by default, that’s it’s job.
- burntsushi 3y ago
  Small clarification: ack did not and does not respect your gitignore files. I just tried it myself, and indeed it doesn't. And this is consistent with the feature chart maintained by the author of ack: https://beyondgrep.com/feature-comparison/ https://beyondgrep.com/feature-comparison/ One practical result of this is that it will mean `ack` will be quite slow when searching typical checkouts of Node.js or Rust projects, because it won't automatically ignore the `node_modules` or `target` directories. In both cases, those directories can become enormous. `ack` will ignore things like `.git` by default though. I believe `ag` was the first widely used grep-like tool that attempted to respect your .gitignore files automatically. (Besides, of course, `git grep`. But `git grep` behaves a little differently. It only searches what is tracked in the repo, and that may or may not be in sync with the rules in your gitignores.)
  - ctenb 3y ago
    You can also pass a flag to git grep to ignore the index (--no-index), which makes it search untracked files as well
    - burntsushi 3y ago
      I know. That's beside the point of my comment. :-)

git grep cannot even find a simple string in its repo.

by zoobab 3y ago

git grep cannot even find a simple string in its repo.

What? Git grep is all you ever need in my experience, and it's ~faster than~ (edit: as fast as) ripgrep when searching a git repo.

dharmab 3y ago
> it's faster than ripgrep when searching a git repo. Source? Ripgrep's benchmarks show it significantly faster.
- ctenb 3y ago
  See sibling comment
larodi 3y ago
I really doubt git grep can outperform ripgrep in any tests... please provide some proof.
- ctenb 3y ago
  I tested this on a large repo in 2016 when I installed several tools (including rg and ag) to compare speed. I don't have the metrics anymore, but the results were pretty clear then. According to the benchmarks from the OP, git grep is pretty comparable to rg in a large git repo. I guess different benchmarks give slightly different results, but the OP acknowledges that git grep is very fast. Bonus is that it comes preinstalled with git and can search through commit history.
  - burntsushi 3y ago
    Author of ripgrep here. It really just depends. The way I like to characterize `git grep` (at present) is that it has sharp performance cliffs. ripgrep has them too, to be sure, but I think it has fewer of them. If you're just searching for a simple literal, `git grep` is decently fast: $ git remote -v origin git@github.com:torvalds/linux (fetch) origin git@github.com:torvalds/linux (push) $ git rev-parse HEAD f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 $ time LC_ALL=en_US.UTF-8 git grep -c -E 'PM_RESUME' Documentation/dev-tools/sparse.rst:3 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 arch/arm/mach-omap2/omap-secure.h:1 arch/arm/mach-omap2/pm33xx-core.c:1 arch/x86/kernel/apm_32.c:1 drivers/input/mouse/cyapa.h:1 drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/usb/mtu3/mtu3_hw_regs.h:1 include/uapi/linux/apm_bios.h:1 real 0.215 user 0.421 sys 1.226 maxmem 161 MB faults 0 $ time rg -c 'PM_RESUME' drivers/mtd/maps/pcmciamtd.c:1 drivers/net/wwan/t7xx/t7xx_reg.h:7 drivers/net/wwan/t7xx/t7xx_pci.c:15 drivers/net/wireless/intersil/hostap/hostap_cs.c:1 drivers/usb/mtu3/mtu3_hw_regs.h:1 drivers/input/mouse/cyapa.h:1 arch/x86/kernel/apm_32.c:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:3 Documentation/translations/zh_TW/sparse.txt:3 Documentation/dev-tools/sparse.rst:3 arch/arm/mach-omap2/pm33xx-core.c:1 arch/arm/mach-omap2/omap-secure.h:1 include/uapi/linux/apm_bios.h:1 real 0.078 user 0.259 sys 0.577 maxmem 15 MB faults 0 But if you switch it up and start adding regex things to your pattern, there can be substantial slowdowns: $ time LC_ALL=C git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 5.704 user 55.671 sys 0.585 maxmem 207 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -E '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 24.529 user 4:34.42 sys 0.753 maxmem 211 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -c -P '\w{5,}\s+PM_RESUME' Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 1.372 user 16.980 sys 0.647 maxmem 211 MB faults 1 $ time rg -c '\w{5,}\s+PM_RESUME' Documentation/translations/zh_CN/dev-tools/sparse.rst:1 Documentation/dev-tools/sparse.rst:1 Documentation/translations/zh_TW/sparse.txt:1 real 0.082 user 0.226 sys 0.612 maxmem 18 MB faults 0 In the above cases, ripgrep has Unicode enabled. (It's enabled by default irrespective of locale settings. ripgrep doesn't interact with POSIX locales at all.)
    - ctenb 3y ago
      Thanks for clarifying! I use `git grep -IPn --color=always --recurse-submodules` many times a day, every day. I hasn't yet let me down, but I don't search for unicode when working on source code. I do use regex though, using the -P switch.
- kzrdude 3y ago
  git grep is shallow: only current git repo, rg is fully recursive, all submodules and also untracked (and not ignored) directories. In some trees, git grep will be a lot faster because it searches a smaller part of it.
  - ctenb 3y ago
    No, git grep can recurse, if you pass the flag, just like all other git commands. --recurse-submodules

masklinn 3y ago

I don’t think there’s been a point to using `git grep` since ack started parsing gitignore. As far as I’m concerned the use case of `git grep` is to search into non-checked-out trees (by giving it a tree-ish). And it’s not super great at that, because it searches a static tree-ish, so pickaxe filters are generally more useful (though they’re slow). Once again mercurial has/had more useful defaults, `hg grep` searches through the history by default, that’s it’s job.

burntsushi 3y ago
Small clarification: ack did not and does not respect your gitignore files. I just tried it myself, and indeed it doesn't. And this is consistent with the feature chart maintained by the author of ack: https://beyondgrep.com/feature-comparison/ https://beyondgrep.com/feature-comparison/ One practical result of this is that it will mean `ack` will be quite slow when searching typical checkouts of Node.js or Rust projects, because it won't automatically ignore the `node_modules` or `target` directories. In both cases, those directories can become enormous. `ack` will ignore things like `.git` by default though. I believe `ag` was the first widely used grep-like tool that attempted to respect your .gitignore files automatically. (Besides, of course, `git grep`. But `git grep` behaves a little differently. It only searches what is tracked in the repo, and that may or may not be in sync with the rules in your gitignores.)
- ctenb 3y ago
  You can also pass a flag to git grep to ignore the index (--no-index), which makes it search untracked files as well
  - burntsushi 3y ago
    I know. That's beside the point of my comment. :-)