4 ms·

@burntsushi - thanks for the benchmarks - certainly rg is amazingly fast! I was wondering if somehow rak and rg could be combined to get the benefit of both wor

@burntsushi - thanks for the benchmarks - certainly rg is amazingly fast! I was wondering if somehow rak and rg could be combined to get the benefit of both worlds (eg. use the strong ffi support in raku and rust to interface them) ... but this would only make any sense if you could apply the rg speed boost to raku regex and thus get the syntax benefits too (sidebar - Larry took to opportunity with perl6/raku to properly reinvent regexes for unicode - there is quite lot to unpack in raku regexes: graphemes management, unicode props as char classes, named regexes, etc.) soo - I suggest this proposal to anyone looking for a cool rust/unicode project ... extend ripgrep from PCRE/ECMA262 regex 1.0 to embrace the "21st century" regex syntax of raku some initial reading is here: - https://dev.to/bbkr/utf-8-glyphs-and-graphemes-331b https://dev.to/bbkr/utf-8-glyphs-and-graphemes-331b - https://docs.raku.org/language/regexes https://docs.raku.org/language/regexes - https://github.com/edumentab/p6-ecma262regex https://github.com/edumentab/p6-ecma262regex

burntsushi 3y ago
To be clear, to do this one would likely need to write an entirely new regex engine. ripgrep's default engine uses finite automata. It's not PCRE2 or ECMA. Many of the things in Raku likely can't easily be implemented in finite automata. ripgrep does have fairly complete UTS#18 Level 1 support already.
- lizmat 3y ago
  > likely need to write an entirely new regex engine Indeed. Also, it should be noted that the Raku Regex engine is not a state machine as such. The regex "slang" in Raku, is basically just another way to write code. Code that uses some state machine primitives underneath, for sure, but still code. A grammar in Raku is nothing other than a class in which the methods are codegenned using the regex syntax. But a class nonetheless, which can also have attributes and "proper" methods. This gives the Raku regex engine the flexibility needed to be able to parse Raku source code, and generate executable bytecode from that.
- librasteve 3y ago
  forgive me for assumption that rg does PCRE(2)/ECMA, we had a bit of a chat on raku and UTS#18 here ... https://www.reddit.com/r/rakulang/comments/15ge67k/uts_18_unicode_regular_expressions_raku/ https://www.reddit.com/r/rakulang/comments/15ge67k/uts_18_un... ... raku does not attmept to implement that explicitly but (by virtue of being an HLL and having built ins such as Set operators) comes quite close to it at all levels.
  - burntsushi 3y ago
    Nice, thank you for the link! And yeah, I figured Raku might not follow UTS#18 to the letter of the law with respect to syntax, but likely has all the concepts available. Which makes sense given what Raku is.

burntsushi 3y ago

To be clear, to do this one would likely need to write an entirely new regex engine. ripgrep's default engine uses finite automata. It's not PCRE2 or ECMA. Many of the things in Raku likely can't easily be implemented in finite automata. ripgrep does have fairly complete UTS#18 Level 1 support already.

lizmat 3y ago
> likely need to write an entirely new regex engine Indeed. Also, it should be noted that the Raku Regex engine is not a state machine as such. The regex "slang" in Raku, is basically just another way to write code. Code that uses some state machine primitives underneath, for sure, but still code. A grammar in Raku is nothing other than a class in which the methods are codegenned using the regex syntax. But a class nonetheless, which can also have attributes and "proper" methods. This gives the Raku regex engine the flexibility needed to be able to parse Raku source code, and generate executable bytecode from that.
librasteve 3y ago
forgive me for assumption that rg does PCRE(2)/ECMA, we had a bit of a chat on raku and UTS#18 here ... https://www.reddit.com/r/rakulang/comments/15ge67k/uts_18_unicode_regular_expressions_raku/ https://www.reddit.com/r/rakulang/comments/15ge67k/uts_18_un... ... raku does not attmept to implement that explicitly but (by virtue of being an HLL and having built ins such as Set operators) comes quite close to it at all levels.
- burntsushi 3y ago
  Nice, thank you for the link! And yeah, I figured Raku might not follow UTS#18 to the letter of the law with respect to syntax, but likely has all the concepts available. Which makes sense given what Raku is.