3 ms·

Do the creators/maintainers of these tools ever try to get their improvements merged into the tools they aim to replace? And does it ever happen? For a while I'

by joemi 2y ago

Do the creators/maintainers of these tools ever try to get their improvements merged into the tools they aim to replace? And does it ever happen? For a while I've heard about things like ripgrep and such that seem to be so much better and faster than grep, so why wouldn't those kinds of improvements get brought into grep? (Note: I'm not asking this from a "down with the old ways!" perspective, but just out of curiosity. I assume there's a reason people are making separate tools instead of improving the existing ones, I just don't know what that reason is.)

landr0id 2y ago
Some algorithm improvements ripgrep uses could be brought into grep, but ripgrep at its core just operates differently. It uses threads by default, assumes unicode, has a completely different regex engine, amongst other things. It could also probably be argued that some things from ripgrep would be pretty difficult to port from Rust to C or C++ safely. burntsushi has a blog post on it here: https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/
- 3eb7988a1663 2y ago
  All of the behind the scenes improvements aside, ripgrep just has significantly better defaults. Recursiveness, ignore binaries, parsing .gitignore, etc. I do not think it would be possible to get grep to alter these configurations. It would break decades of ossified scripts.
arp242 2y ago
I've written some tools like this; some released, some not (mostly because unfinished). The problem with "merging it upstream" is that it's just a lot of friction. There are also some ideas I have and having my own tool gives me the freedom to explore and experiment without worrying too much what other people think or compatibility. And then there's the language choice, as well as code quality. I don't really want to start a huge discussion about this, but it should be pretty obvious that many people are more comfortable and productive using languages that are not C, and that some of these tools don't have the best C code you can find. A lot of "Rewrite in Rust" (or Go, Python, what-have-you) isn't really about the "Rewrite in Rust" as such, but rather about "Rewrite so I can play around with ideas I have". Also see my comment from a while ago when someone asked "why do this when $other_tool already exists?": https://github.com/arp242/elles/issues/1#issuecomment-2168559003 https://github.com/arp242/elles/issues/1#issuecomment-216855...
gitaarik 2y ago
Ripgrep is written in Rust while the original grep in C. So it doesn't make sense to try to combine them. Ripgrep is rather the modern rewrite, and we keep the old grep around for backwards compatibility.
burntsushi 2y ago
One of the biggest improvements reported by my users is the smart filtering enabled by default in ripgrep. That can't be contributed back to GNU grep. Also, people have tried to make GNU grep uses multiple threads. As far as I know, none of those attempts have led to a merged patch. There are a boatload of other reasons to be honest as well. And there's no reason why I specifically need to do it. I've written extensively on how and why ripgrep is fast, all the way down to details in the regex engine. There is no mystery here, and anyone can take these techniques and try to port them to another grep.
- joemi 2y ago
  Thanks, it's really nice to get an answer from the creator of one of these tools!

by joemi 2y ago

landr0id 2y ago

Some algorithm improvements ripgrep uses could be brought into grep, but ripgrep at its core just operates differently. It uses threads by default, assumes unicode, has a completely different regex engine, amongst other things. It could also probably be argued that some things from ripgrep would be pretty difficult to port from Rust to C or C++ safely. burntsushi has a blog post on it here: https://blog.burntsushi.net/ripgrep/ https://blog.burntsushi.net/ripgrep/

3eb7988a1663 2y ago
All of the behind the scenes improvements aside, ripgrep just has significantly better defaults. Recursiveness, ignore binaries, parsing .gitignore, etc. I do not think it would be possible to get grep to alter these configurations. It would break decades of ossified scripts.

arp242 2y ago

I've written some tools like this; some released, some not (mostly because unfinished). The problem with "merging it upstream" is that it's just a lot of friction. There are also some ideas I have and having my own tool gives me the freedom to explore and experiment without worrying too much what other people think or compatibility. And then there's the language choice, as well as code quality. I don't really want to start a huge discussion about this, but it should be pretty obvious that many people are more comfortable and productive using languages that are not C, and that some of these tools don't have the best C code you can find. A lot of "Rewrite in Rust" (or Go, Python, what-have-you) isn't really about the "Rewrite in Rust" as such, but rather about "Rewrite so I can play around with ideas I have". Also see my comment from a while ago when someone asked "why do this when $other_tool already exists?": https://github.com/arp242/elles/issues/1#issuecomment-2168559003 https://github.com/arp242/elles/issues/1#issuecomment-216855...

gitaarik 2y ago

Ripgrep is written in Rust while the original grep in C. So it doesn't make sense to try to combine them. Ripgrep is rather the modern rewrite, and we keep the old grep around for backwards compatibility.

burntsushi 2y ago

One of the biggest improvements reported by my users is the smart filtering enabled by default in ripgrep. That can't be contributed back to GNU grep. Also, people have tried to make GNU grep uses multiple threads. As far as I know, none of those attempts have led to a merged patch. There are a boatload of other reasons to be honest as well. And there's no reason why I specifically need to do it. I've written extensively on how and why ripgrep is fast, all the way down to details in the regex engine. There is no mystery here, and anyone can take these techniques and try to port them to another grep.

joemi 2y ago
Thanks, it's really nice to get an answer from the creator of one of these tools!