3 ms·
> Regex engines neither know nor care what a character is. They match bytes. I think this is mostly not true, for example there's some interesting behavior in
by camgunz 2y ago
> Regex engines neither know nor care what a character is. They match bytes.
I think this is mostly not true, for example there's some interesting behavior in POSIX-compliant engines:
"Other than POSIX-compliant engines part of a POSIX-compliant system, none of the regex flavors discussed in this tutorial support collating sequences.
Note that a fully POSIX-compliant regex engine treats ch as a single character when the locale is set to Czech. This means that [^x]emie also matches chemie. [^x] matches a single character that is not an x, which includes ch in the Czech POSIX locale.
In any other regular expression engine, or in a POSIX engine using a locale that does not treat ch as a digraph, [^x]emie matches the misspelled word cemie but not chemie, as [^x] cannot match the two characters ch."
If you were just matching bytes you wouldn't do any of this. These engines clearly know that "ch" is two characters and two bytes under some locales and 1 character and two bytes under others.
Same stuff goes for the Unicode regexes: using the traditional patterns leads to unexpected results: "In Unicode, à can be encoded as two code points: U+0061 (a) followed by U+0300 (grave accent). In this situation, . applied to à will match a without the accent. ^.$ will fail to match, since the string consists of two code points. ^..$ matches à." [1]. \p{Letter} works correctly here though, again showing that these engines know about more than just bytes.
[0]: https://www.regular-expressions.info/posixbrackets.html#coll https://www.regular-expressions.info/posixbrackets.html#coll
[1]: https://www.regular-expressions.info/unicode.html https://www.regular-expressions.info/unicode.html