7 ms·

Powershell actually ships one! I think you have a good point. I was going to build my own as C# ships alot of configuration options that can DRASTICALLY improve

by maldev 4y ago

Powershell actually ships one! I think you have a good point. I was going to build my own as C# ships alot of configuration options that can DRASTICALLY improve speed, which arn't on by default. But, we can test Redmonds version and get at least a decent idea of if it's suitable. I'll run some tests on that and get back to you in an hour or so. I'll use the same tests you have on https://github.com/BurntSushi/ripgrep/blob/master/benchsuite/runs/2016-09-20-ubuntu1604-ec2/raw.csv https://github.com/BurntSushi/ripgrep/blob/master/benchsuite... My testing will be on my Dev VM, in a WSL container on ubuntu 22.04, targeting a built linux kernel. With powershell from the Microsoft blobs they ship for linux. And Ripgrep from just Apt Install Ripgrep. With 5 warmup cycles for each.

burntsushi 4y ago
Please be sure to read my blog post that I linked very carefully. Benchmarking greps is rather difficult and quite subtle. Are you talking about findstr? If so, I'm not sure that's available on Linux. So it doesn't seem like a viable grep to me. If the program you want to be a grep only runs on Windows, then that isn't really viable. Also, Windows file system operations tend to be quite a bit slower than Linux. Also, where is the source for findstr?
- maldev 4y ago
  Alright, so initial testing result. I'm not going to even bother putting into excel, RipGrep blew the hell out of Powershells built in stuff. Like, we're talking exponentially quicker, so well done. (3 minutes for Pwsh vs 2 seconds Ripgrep) But I don't think it's the fault of C# and I think it's the fault of some poor coding on redmonds side, and very good coding on your side. And since I have to make up 12 hours tomorrow at work, but have zero tickets. I'll probably go in and write a Grep utility in C# and I think I can get similar speed to the Rust implementation. C# is also cross platform as of a couple years ago, as well as is powershell. So my testing was running Powershell on linux against the newest linux kernel source code + the build artifacts.
  - majkinetor 4y ago
    You can't compare rg with other full RE engines. Try to repeat the test by switching its regex engine to use PCRE2 ( --pcre2 ).
    - burntsushi 4y ago
      What? Can you say more? In many cases, ripgrep's speed is due in part to integrations with the default regex engine that don't happen with PCRE2.
      - majkinetor 4y ago
        Why would you compare C#/pwsh PCRE with somethign that is not PCRE and hence has different features. Compare with rg that has --pcre option set if you want to get real comparison.
        burntsushi 4y ago
        Why wouldn't you? The other poster hasn't even said which regex engine or which tool they're measuring. PCRE is absolutely a competitor to Rust's regex crate and it makes sense to compare them. If all you're doing is measuring the time it takes to find `foo` or `\w+_foo`, then the specific regex engine being used is just an implementation detail. I'm not saying that using --pcre2 is wrong. It is certainly a useful comparison. But using the default regex engine when the task is "measure a grep" is certainly not wrong either.
  - burntsushi 4y ago
    I think you've kind of already moved the goal posts. Your original comment spoke as if it were factual. But now you're saying "I think it can be done." But you don't know. And on top of that, there have been many greps written in a variety of languages: Go, Python, Perl, Ruby. All of them, that I've tested, are slower than most greps written in C, C++ and Rust. (In some cases, the greps written in Go can be quite fast, but they have very steep performance cliffs that are easy to trip over.) There are very good reasons to explain this state of affairs. C# is not something I have a lot of experience with. I don't spend time with Windows-only technologies. C# may technically work on Linux these days, but I don't know of any popular general purpose CLI tooling built with it that works on Unix systems. Why is that? My suspicion is that there are good reasons for it. But if there aren't, then there is a market opening waiting for folks to plow forward. Basically, I think you are wrong on multiple levels. I think you both under-estimate the difficulty of writing a fast grep and what exactly it takes to do it, and I think you're completely wrong about when Rust is useful. You completely neglect the safety aspect of it, and seem to believe it is not on even footing with C or C++ when it comes to performance. But it most certainly is, in general. > But I don't think it's the fault of C# and I think it's the fault of some poor coding on redmonds side Making baseless assertions feels like a pattern that is forming in your comments. Where is Redmond's code? Can you read it and link to me the parts that are "poor"? If not, from whence comes the accusation? And if you insist with your interpretation of reality, then it follows that I therefore must be a good coder. And if a good coder says to you, "I do not know how to write a fast grep in Go, Python, Perl or Ruby," then why don't you take them seriously?
burntsushi 4y ago
And does findstr support regex?

by maldev 4y ago

burntsushi 4y ago

Please be sure to read my blog post that I linked very carefully. Benchmarking greps is rather difficult and quite subtle. Are you talking about findstr? If so, I'm not sure that's available on Linux. So it doesn't seem like a viable grep to me. If the program you want to be a grep only runs on Windows, then that isn't really viable. Also, Windows file system operations tend to be quite a bit slower than Linux. Also, where is the source for findstr?

maldev 4y ago
Alright, so initial testing result. I'm not going to even bother putting into excel, RipGrep blew the hell out of Powershells built in stuff. Like, we're talking exponentially quicker, so well done. (3 minutes for Pwsh vs 2 seconds Ripgrep) But I don't think it's the fault of C# and I think it's the fault of some poor coding on redmonds side, and very good coding on your side. And since I have to make up 12 hours tomorrow at work, but have zero tickets. I'll probably go in and write a Grep utility in C# and I think I can get similar speed to the Rust implementation. C# is also cross platform as of a couple years ago, as well as is powershell. So my testing was running Powershell on linux against the newest linux kernel source code + the build artifacts.
- majkinetor 4y ago
  You can't compare rg with other full RE engines. Try to repeat the test by switching its regex engine to use PCRE2 ( --pcre2 ).
  - burntsushi 4y ago
    What? Can you say more? In many cases, ripgrep's speed is due in part to integrations with the default regex engine that don't happen with PCRE2.
    - majkinetor 4y ago
      Why would you compare C#/pwsh PCRE with somethign that is not PCRE and hence has different features. Compare with rg that has --pcre option set if you want to get real comparison.
      - burntsushi 4y ago
        Why wouldn't you? The other poster hasn't even said which regex engine or which tool they're measuring. PCRE is absolutely a competitor to Rust's regex crate and it makes sense to compare them. If all you're doing is measuring the time it takes to find `foo` or `\w+_foo`, then the specific regex engine being used is just an implementation detail. I'm not saying that using --pcre2 is wrong. It is certainly a useful comparison. But using the default regex engine when the task is "measure a grep" is certainly not wrong either.
- burntsushi 4y ago
  I think you've kind of already moved the goal posts. Your original comment spoke as if it were factual. But now you're saying "I think it can be done." But you don't know. And on top of that, there have been many greps written in a variety of languages: Go, Python, Perl, Ruby. All of them, that I've tested, are slower than most greps written in C, C++ and Rust. (In some cases, the greps written in Go can be quite fast, but they have very steep performance cliffs that are easy to trip over.) There are very good reasons to explain this state of affairs. C# is not something I have a lot of experience with. I don't spend time with Windows-only technologies. C# may technically work on Linux these days, but I don't know of any popular general purpose CLI tooling built with it that works on Unix systems. Why is that? My suspicion is that there are good reasons for it. But if there aren't, then there is a market opening waiting for folks to plow forward. Basically, I think you are wrong on multiple levels. I think you both under-estimate the difficulty of writing a fast grep and what exactly it takes to do it, and I think you're completely wrong about when Rust is useful. You completely neglect the safety aspect of it, and seem to believe it is not on even footing with C or C++ when it comes to performance. But it most certainly is, in general. > But I don't think it's the fault of C# and I think it's the fault of some poor coding on redmonds side Making baseless assertions feels like a pattern that is forming in your comments. Where is Redmond's code? Can you read it and link to me the parts that are "poor"? If not, from whence comes the accusation? And if you insist with your interpretation of reality, then it follows that I therefore must be a good coder. And if a good coder says to you, "I do not know how to write a fast grep in Go, Python, Perl or Ruby," then why don't you take them seriously?

burntsushi 4y ago

And does findstr support regex?