5 ms·

As another user mentioned, many POSIX and/or GNU utilities havent aged well. I respect trying to stay portable, but peoples needs change over time and these too

As another user mentioned, many POSIX and/or GNU utilities havent aged well. I respect trying to stay portable, but peoples needs change over time and these tools simply havent kept up. Like the other user, I use Fd now instead: https://github.com/sharkdp/fd https://github.com/sharkdp/fd As well as Silver Searcher: https://github.com/ggreer/the_silver_searcher https://github.com/ggreer/the_silver_searcher while Grep performance is pretty good, its also gotten pretty stale with regard to its defaults and options.

kragen 7y ago
Wouldn't you have the same problem with fd if you invoke it on unquoted globs? The problem is the semantics of the shell, not find.
- wffurr 7y ago
  Yes, but fd has implicit pattern matching without requiring the * character: Features: Convenient syntax: fd PATTERN instead of find -iname '*PATTERN*'
  - pwdisswordfish2 7y ago
    Seems a misfeature. Much of the time, I use find to look for files with a specific extension, which makes the trailing asterisk harmful.
    - johnp_ 7y ago
      fd -e py Also neatly bypasses the glob issue.
anonsivalley652 7y ago
For grepping, rg is faster and saner. https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep On end-user systems with fast I/O, it is usually a better use of resources to have a battery-/suspend-/reboot-aware indexing system that has path/extension white/blacklists and prioritizes file monitored changes. This way, searching can happen against an optimized text search DBMS much faster than waiting for zillions of IOPS that are basically wasted by instead spending them ahead-of-time in the background/idle indexing text files/metadata/structured data once when it changed/s and already know exactly where all occurrences of kWhateverCondition_PP3V42_G3H or \A[AB]{1,3}c+d\z live under ~/Projects without reading any actual files.
- eikenberry 7y ago
  Do either of these tools have a "grep" mode, so that I can alias grep to it and get similar, but improved, behavior? 25 years of habit is a lot of reinforcement to overcome and I'd love to just be able to take advantage of these tools with an alias. As is I just have a grep wrapping shell script to give it some sane defaults which works fairly well.
  - anonsivalley652 7y ago
    No, you're asking oranges to be bananas. Read the man page or user guide. It's not grep, it's better and already has saner defaults. https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md
  - ryannevius 7y ago
    The closest I know of is `rg -uuu`. From the manual: > -u, --unrestricted ... > Reduce the level of "smart" searching. A single -u won't respect .gitignore (etc.) files. Two -u flags will additionally search hidden files and directories. Three -u flags will additionally search binary files. > rg -uuu is roughly equivalent to grep -r.
  - burntsushi 7y ago
    Author of ripgrep here. > As is I just have a grep wrapping shell script to give it some sane defaults which works fairly well. Yes, that's what I did for about ten years before I wrote ripgrep. :-) > Do either of these tools have a "grep" mode, so that I can alias grep to it and get similar, but improved, behavior? Another commenter already mentioned `rg -uuu`, and that's pretty much the right answer. In a large number of cases, if you `alias grep=rg`, then most things will continue to work as you expect. If you really do not want any kind of smart filtering, then yes, you'll want `alias grep="rg -uuu"`. But otherwise, things like `command ... | rg pattern` and `rg pattern file ...` will continue to work just like grep. ripgrep also tries to use the same names for flags as grep, wherever possible. So you'll find a lot of overlap there too. Please note that ripgrep is not and is not intended to be a drop-in replacement. But I didn't go out of my way to be incompatible with grep either. So a lot of it should be quite familiar!
weberc2 7y ago
In this case the shell hasn’t aged well. Besides the globing feature, the bash language is arcane and few can actually reliably write correct bash despite it being ubiquitous. Personally I would like to see a completely reimagined take on the shell.
- eru 7y ago
  Have you looked at powershell, scsh or fish?
- AlexCoventry 7y ago
  shellcheck helps a ton, though.
mikenew 7y ago
> Smart case: the search is case-insensitive by default. It switches to case-sensitive if the pattern contains an uppercase character*. This is what I want as a default for pretty much any text search.
- josteink 7y ago
  Emacs have had this since forever. It’s pretty neat.
  - Lio 7y ago
    Vim also provides it if you use: :set smartcase
userbinator 7y ago
As another user mentioned, many POSIX and/or GNU utilities havent aged well What do you mean by that? Users have gotten more stupid and uneducated over time? I've seen "when in doubt, escape it" in several books about UNIX and shell, and escapes and wildcard expansion are one of the most beginner lessons in a lot of other materials. However, the Internet has let everyone have a voice, for better or worse, and as a result anyone who barely knows something can now write a misguided "tutorial" about it. In other words: don't blame the tools, nor advocate writing dumbed-down replacements which lack the composability and generality the originals had; blame the proliferation of barely-correct educational material that has spread cancerously over the Internet.
- svnpenn 7y ago
  I have written hundreds of shell scripts, several which are hundreds of lines long. When I say they havent aged well, Im speaking from experience. When I use something like: find -name *.jpg I know what it actually does. But what it actually doesnt isnt what it should do. The current behavior returns matched files... unless no files match. Then it returns the literal string "*.jpg". That behavior is usually going to be desired by no one. Same with: grep $SOMETHING it going to fail if the variable has any spaces. Again something most people arent going to want. These both can be worked around with Shell options and "IFS" respectively, but people shouldnt have to do that. The default behavior should be sane, but in many cases the POSIX standard isnt that. For goodness sake, the "venerable" find tool doesnt even have a "maxdepth" option: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/f...
  - userbinator 7y ago
    You completely ignored my point, which is that none of this behaviour should be surprising or unexpected to anyone who has taken the time to think about how it all works together, and that this flexibility is the inherent way in which the UNIX command line is so powerful. it going to fail if the variable has any spaces Quoting and escaping. I mentioned that already, it's a basic part of the understanding and is not surprising at all. Again something most people arent going to want. You're assuming a lot. The fact that the variable substitution could expand to multiple arguments (because it's really just simple textual substitution) is in fact very powerful and useful (you can pass multiple arguments in one variable, for example), and you can always quote if you really want it to expand to one argument, the same way you would quote strings containing spaces to make them one argument. This is very consistent; observe that if $SOMETHING contains "foo bar" (without the quotes), then grep $SOMETHING is exactly the same as grep foo bar but what you seem to be suggesting is that it becone grep "foo bar" ...what? Where did those quotes come from? More importantly, how do you propose to remove them? My point is, the current behaviour is simple and consistent as well as powerful and flexible, and I suspect people are just going "it doesn't work the way I thought it would" and downvoting my comment when they really haven't thought about the perfectly logical explanation for why things are the way they are. The original developers of the shell language were most certainly not stupid.
BeeOnRope 7y ago
That's not going to directly solve this problem since the globs are expanded before the invoked process is even called with its arguments.
fit2rule 7y ago
For me, Silver Searcher often fails to find matches in files that 'grep' will, easily. I've not been able to determine why this is, but it happens often enough that I consider SS to be highly unreliable, even if its fast. So, it seems that there are still some issues to be sorted here.
- bmn__ 7y ago
  This claim needs adequate proof, please provide an example. Otherwise the HN reader has to assume you made a user error.
  - fit2rule 7y ago
    Maybe it has something to do with my not understanding its case sensitivity rules .. which I've learned about in this thread as being different. As soon as it happens again I'll bring it here as an example, but like I said I stopped using silversearcher because of it getting in my way like that.
choeger 7y ago
I would argue that this case shows that the shell(s) have a problem with a feature being enabled by default. In hindsight it might have been smarter to put globbing expressions in special quotes than the other way round.
Aardwolf 7y ago
Imho also anything that outputs or uses multiples of 512 bytes (blocks) by default Sure, disks may have sectors and there may be some use cases, but we use bytes, kilobytes, kibibytes, etc now for data sizes.

kragen 7y ago

Wouldn't you have the same problem with fd if you invoke it on unquoted globs? The problem is the semantics of the shell, not find.

wffurr 7y ago
Yes, but fd has implicit pattern matching without requiring the * character: Features: Convenient syntax: fd PATTERN instead of find -iname '*PATTERN*'
- pwdisswordfish2 7y ago
  Seems a misfeature. Much of the time, I use find to look for files with a specific extension, which makes the trailing asterisk harmful.
  - johnp_ 7y ago
    fd -e py Also neatly bypasses the glob issue.

anonsivalley652 7y ago

For grepping, rg is faster and saner. https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep On end-user systems with fast I/O, it is usually a better use of resources to have a battery-/suspend-/reboot-aware indexing system that has path/extension white/blacklists and prioritizes file monitored changes. This way, searching can happen against an optimized text search DBMS much faster than waiting for zillions of IOPS that are basically wasted by instead spending them ahead-of-time in the background/idle indexing text files/metadata/structured data once when it changed/s and already know exactly where all occurrences of kWhateverCondition_PP3V42_G3H or \A[AB]{1,3}c+d\z live under ~/Projects without reading any actual files.

eikenberry 7y ago
Do either of these tools have a "grep" mode, so that I can alias grep to it and get similar, but improved, behavior? 25 years of habit is a lot of reinforcement to overcome and I'd love to just be able to take advantage of these tools with an alias. As is I just have a grep wrapping shell script to give it some sane defaults which works fairly well.
- anonsivalley652 7y ago
  No, you're asking oranges to be bananas. Read the man page or user guide. It's not grep, it's better and already has saner defaults. https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md
- ryannevius 7y ago
  The closest I know of is `rg -uuu`. From the manual: > -u, --unrestricted ... > Reduce the level of "smart" searching. A single -u won't respect .gitignore (etc.) files. Two -u flags will additionally search hidden files and directories. Three -u flags will additionally search binary files. > rg -uuu is roughly equivalent to grep -r.
- burntsushi 7y ago
  Author of ripgrep here. > As is I just have a grep wrapping shell script to give it some sane defaults which works fairly well. Yes, that's what I did for about ten years before I wrote ripgrep. :-) > Do either of these tools have a "grep" mode, so that I can alias grep to it and get similar, but improved, behavior? Another commenter already mentioned `rg -uuu`, and that's pretty much the right answer. In a large number of cases, if you `alias grep=rg`, then most things will continue to work as you expect. If you really do not want any kind of smart filtering, then yes, you'll want `alias grep="rg -uuu"`. But otherwise, things like `command ... | rg pattern` and `rg pattern file ...` will continue to work just like grep. ripgrep also tries to use the same names for flags as grep, wherever possible. So you'll find a lot of overlap there too. Please note that ripgrep is not and is not intended to be a drop-in replacement. But I didn't go out of my way to be incompatible with grep either. So a lot of it should be quite familiar!

weberc2 7y ago

In this case the shell hasn’t aged well. Besides the globing feature, the bash language is arcane and few can actually reliably write correct bash despite it being ubiquitous. Personally I would like to see a completely reimagined take on the shell.

eru 7y ago
Have you looked at powershell, scsh or fish?
AlexCoventry 7y ago
shellcheck helps a ton, though.

mikenew 7y ago

> Smart case: the search is case-insensitive by default. It switches to case-sensitive if the pattern contains an uppercase character*. This is what I want as a default for pretty much any text search.

josteink 7y ago
Emacs have had this since forever. It’s pretty neat.
- Lio 7y ago
  Vim also provides it if you use: :set smartcase

userbinator 7y ago

As another user mentioned, many POSIX and/or GNU utilities havent aged well What do you mean by that? Users have gotten more stupid and uneducated over time? I've seen "when in doubt, escape it" in several books about UNIX and shell, and escapes and wildcard expansion are one of the most beginner lessons in a lot of other materials. However, the Internet has let everyone have a voice, for better or worse, and as a result anyone who barely knows something can now write a misguided "tutorial" about it. In other words: don't blame the tools, nor advocate writing dumbed-down replacements which lack the composability and generality the originals had; blame the proliferation of barely-correct educational material that has spread cancerously over the Internet.

svnpenn 7y ago
I have written hundreds of shell scripts, several which are hundreds of lines long. When I say they havent aged well, Im speaking from experience. When I use something like: find -name *.jpg I know what it actually does. But what it actually doesnt isnt what it should do. The current behavior returns matched files... unless no files match. Then it returns the literal string "*.jpg". That behavior is usually going to be desired by no one. Same with: grep $SOMETHING it going to fail if the variable has any spaces. Again something most people arent going to want. These both can be worked around with Shell options and "IFS" respectively, but people shouldnt have to do that. The default behavior should be sane, but in many cases the POSIX standard isnt that. For goodness sake, the "venerable" find tool doesnt even have a "maxdepth" option: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/f...
- userbinator 7y ago
  You completely ignored my point, which is that none of this behaviour should be surprising or unexpected to anyone who has taken the time to think about how it all works together, and that this flexibility is the inherent way in which the UNIX command line is so powerful. it going to fail if the variable has any spaces Quoting and escaping. I mentioned that already, it's a basic part of the understanding and is not surprising at all. Again something most people arent going to want. You're assuming a lot. The fact that the variable substitution could expand to multiple arguments (because it's really just simple textual substitution) is in fact very powerful and useful (you can pass multiple arguments in one variable, for example), and you can always quote if you really want it to expand to one argument, the same way you would quote strings containing spaces to make them one argument. This is very consistent; observe that if $SOMETHING contains "foo bar" (without the quotes), then grep $SOMETHING is exactly the same as grep foo bar but what you seem to be suggesting is that it becone grep "foo bar" ...what? Where did those quotes come from? More importantly, how do you propose to remove them? My point is, the current behaviour is simple and consistent as well as powerful and flexible, and I suspect people are just going "it doesn't work the way I thought it would" and downvoting my comment when they really haven't thought about the perfectly logical explanation for why things are the way they are. The original developers of the shell language were most certainly not stupid.

BeeOnRope 7y ago

That's not going to directly solve this problem since the globs are expanded before the invoked process is even called with its arguments.

fit2rule 7y ago

For me, Silver Searcher often fails to find matches in files that 'grep' will, easily. I've not been able to determine why this is, but it happens often enough that I consider SS to be highly unreliable, even if its fast. So, it seems that there are still some issues to be sorted here.

bmn__ 7y ago
This claim needs adequate proof, please provide an example. Otherwise the HN reader has to assume you made a user error.
- fit2rule 7y ago
  Maybe it has something to do with my not understanding its case sensitivity rules .. which I've learned about in this thread as being different. As soon as it happens again I'll bring it here as an example, but like I said I stopped using silversearcher because of it getting in my way like that.

choeger 7y ago

I would argue that this case shows that the shell(s) have a problem with a feature being enabled by default. In hindsight it might have been smarter to put globbing expressions in special quotes than the other way round.

Aardwolf 7y ago

Imho also anything that outputs or uses multiples of 512 bytes (blocks) by default Sure, disks may have sectors and there may be some use cases, but we use bytes, kilobytes, kibibytes, etc now for data sizes.