3 ms·
I wonder if that's putting things through unicode "canonical normalization", or than custom rules. Let's see what it does with `U+00BC Vulgar Fraction One Quar
by jrochkind1 6y ago
I wonder if that's putting things through unicode "canonical normalization", or than custom rules.
Let's see what it does with `U+00BC Vulgar Fraction One Quarter Unicode Character`... ¼
Nope it allows it instead of turning into `1/4`, so that's not canonical normalization. I guess it's custom rules? Or some other unicode transformation we're not thinking of, or other third-party re-usable transformation.
- masklinn 6y ago> I guess it's custom rules? Or some other unicode transformation we're not thinking of, or other third-party re-usable transformation. They just blacklisted (or whitelisted) blocks or categories.
- jrochkind1 6y agoConverting a U+2009 THIN SPACE into an ordinary ascii space is not black/whitelisting.
- masklinn 6y agoTrue, there's almost certainly a whitespace normalisation pass at one point as well, likely during / around the processing of what little makup HN has.