3 ms·
> Does piping rg output to wc -l affect time(1) output? Oh yes absolutely! If `rg` is printing to a tty, it will automatically enable showing line numbers and
by burntsushi 3y ago
> Does piping rg output to wc -l affect time(1) output?
Oh yes absolutely! If `rg` is printing to a tty, it will automatically enable showing line numbers and printing with colors. Both of those have costs (over and beyond just printing matches and their byte offsets) that appear irrelevant to your use case. Neither of those things are done by ired. It's not about `wc -l` specifically, but about piping into anything. And of course, with `wc -l`, you avoid the time needed to actually render the results. But I used `wc -l` with ired too, so I "normalized" the benchmarking model and simplified it.
But either way, my most recent comment before this one capitulated to your demands and avoided the use of piping results into anything. It was for this reason that I showed commands with `--color=never -N`.
And yes, `grep -Eo` gets slower with more `.`. ripgrep does too, but is a bit more robust than GNU grep. I already demonstrated this in my most recent previous comment and even wrote some words about it explicitly and that regex engines can't typically do as well with increasing window sizes like this when compared to a purpose built tool like ired. Nevertheless, ired is still slower than ripgrep in most of the tests I showed in my previous comment.
But optimally speaking, could something even faster than both ired and ripgrep be built? I believe so, yes. But only for some workloads, I suspect, with high match frequency. And ain't nobody going to build such a thing just to save a few milliseconds. Lol. The key is really to implement the windowing explicitly instead of relying on the regex engine to do it for you. Alternatively, one could add a special optimization pass to the regex engine that recognizes "windowing" patterns and does something clever. I have a ticket open for something similar[1].
[1]: https://github.com/rust-lang/regex/issues/802 https://github.com/rust-lang/regex/issues/802