5 ms·
Not directly related, but recently I've built a regex engine that was a bit different: I implemented the semantics of Java's java.util.regex.Pattern on top of J
by sjrd 5y ago
Not directly related, but recently I've built a regex engine that was a bit different: I implemented the semantics of Java's java.util.regex.Pattern on top of JavaScript's RegExp. This is for Scala.js' implementation of Pattern. [1]
It turns out that they are wildly incompatible by default, so directly passing through the regular expressions to RegExp is incorrect in most cases. That was what we used to do before, though. Other compilers like GWT use the same technique.
So I implemented a complete compiler from Java regular expressions to JavaScript regexes that preserves semantics. It supports virtually everything, and fails fast for the few things that are not supported (like grapheme clusters or canonical equivalence). Some things are very tricky to get right, like atomic groups or handling of surrogate pairs when the 'u' flag is not natively supported.
The advantages of that strategy over reimplementing a complete engine (which is what TeaVM does) are a) code size and b) we get the performance of native RegExp's (which are JIT-compiled to native code, these days).
[1] https://github.com/scala-js/scala-js/pull/4455 https://github.com/scala-js/scala-js/pull/4455
- aarchi 5y agoWow that's a phenomenal effort! It's surprisingly complex converting between regexp dialects.
- bassdropvroom 5y agoThis would be cool to extract out and use within regexr[0] [0] https://regexr.com/ https://regexr.com/
- sjrd 5y agoI'd be happy to help, if someone wants to take the lead, and there is buy in from the maintainers of Regexr. I believe my contact is available in my profile. Otherwise, I'm really not hard to find on the internet. ;)
- eevilspock 5y agoNot directly related to your not directly related: Are there any regexp syntax that works across main language platforms (e.g. JS, C, Rust, Ruby, Go) with identical semantics? Is there a subset of each language's native regexp syntax that is common across all of them? Or perhaps a third-party regexp library that has been implemented on each language? I've found one: Google's C++ RE2, which has wrappers for many languages.
- sjrd 5y agoI'm not an expert in regexes in other platforms, but already if you want a common subset between Java and JS, you're limited to the following constructs: * Literal characters * Some escape sequences for literal characters: \t \r \n \xHH \uUUUU as well as the ones for the regex metacharacters such as \[ \( etc. * Groups, sequences and alternatives * Look-ahead (positive and negative) * Look-behind, if supported (ES 2018+) * Back-references to existing groups opened before the back-references, but you need to make sure that they're not syntactically followed by a digit, otherwise parsing is different between JS and Java * Greedy and lazy quantifiers * Custom character classes with [], without intersections or inner custom character classes That's assuming you have 'u' support in JS and that you use it (ECMAScript 2015), and that you don't use any other flag. (you can use the 'g' flag in JS or not; it affects the behavior of the methods, but not of the regex itself). If you don't have 'u' support, it's only the same if you restrict your regex and your input to ASCII characters. Some things that are unexpectedly NOT portable: * Most pre-defined character classes like \s and the like * CASE_INSENSITIVE/'i' and MULTILINE/'m' * \p{...} character classes * \b and \B (there's a bug in OpenJDK: https://bugs.openjdk.java.net/browse/JDK-8264160 https://bugs.openjdk.java.net/browse/JDK-8264160)
- slimsag 5y agoAlso: * Named groups are not supported everywhere. * Lookahead/lookbehind are not supported in Go and Rust (unless using fancy-regexp, from what I understand) Also beware that character classes like `[]` have differing Unicode support, some operate on ASCII, unicode codepoints, or on extended grapheme clusters.. so YMMV. As you start to expand to more languages, the common feature set is _far_ more limited.
- Siira 5y agoAny PCRE to JS convertor you would know of?
- sjrd 5y agoI'm afraid not. Otherwise I would have had some head start for this one. ;-)
- SonOfLilit 5y agoI also wrote a regex-to-regex compiler, although in my case it was for a more human-centered regex DSL with a reasonable migration story. https://www.slideshare.net/AurSaraf/re3-modern-regex-syntax-with-a-focus-on-adoption https://www.slideshare.net/AurSaraf/re3-modern-regex-syntax-... https://github.com/sonoflilit/re2 https://github.com/sonoflilit/re2