3 ms·
Unfortunately that won't work. Twitter doesn't support the full Unicode; they strip out any Unicode code points that's not used by "normal" people.
by desdiv 11y ago
Unfortunately that won't work. Twitter doesn't support the full Unicode; they strip out any Unicode code points that's not used by "normal" people.
- TazeTSchnitzel 11y ago> they strip out any Unicode code points that's not used by "normal" people. Such as?
- goodplay 11y agoLTR and RTL marks (all of them), which are used when the default Unicode bidi algorithm fails to display two intertwined scripts correctly. They are especially useful when writing short technical tweets. I bet normal people would use them too if they knew about them and were given a chance, but normal people can't because twitter doesn't allow codepoints that aren't used by normal people.
- TazeTSchnitzel 11y agoStripping out LTR and RTL override might be reasonable because they're frequently abused to screw up website layouts. But stripping all of the marks is just discrimination against people who need to write bi-directional text.
- cbd1984 11y ago> Stripping out LTR and RTL override might be reasonable because they're frequently abused to screw up website layouts. As long as Twitter allows Zalgo it isn't that concerned about website layouts. (Zalgo: Rampant abuse of stacking combining diacritical marks above and below the line. It can get really, really... uh... impressive, in a Lovecraftian typewriter art kind of way.)
- goodplay 11y agoThe issue of effecting web layout can be resolved by simply appending the appropriate resetting codepoint to the tweets. (U+202C for bidi overrides)
- amptorn 11y agoWell, null, to start with. Other ASCII control characters. Consecutive line breaks, too.
- mikeash 11y agoInteresting, is there a list of what they strip out? I know they apply normalization so that would eliminate a few, but I think that wouldn't do too much on its own.
- simplicio 11y agoAre you sure? I just posted a tweet in Cuneiform, which is outside the BMP. So their view of "normal people" seems to encompass ancient Sumerians, at least.