5 ms·

Fun fact: because flags are a political nature, and Unicode did not want to limit the length of country codes, any number of Regional Indicator Symbol code poin

Fun fact: because flags are a political nature, and Unicode did not want to limit the length of country codes, any number of Regional Indicator Symbol code points can be placed back to back and treated as an extended grapheme cluster. That means that what Unicode today considers "one character" can be infinite in size.

kps 7y ago
And worse, the ‘operators’ are postfix or infix, so if you're handling text sequentially — and let's say, split across network packets with an arbitrary delay in between — you can't even tell you've read a complete character until you receive the next one. Typing in many Latin languages generates diacritics using ‘dead keys’, which are prefix operators. (Originally, on a physical typewriter, these were keys that simply struck the paper without triggering the mechanism to advance the carriage.) If Unicode had taken this hint, life would be easier. (ISO/IEC 6937 had prefix diacritics, but it was too late.)
- tialaramex 7y ago
  Complete recognition before processing. It's not the law, just a good idea, but you should do it anyway. In cryptography this lesson had to keep being learned. "Oh, I received this partial data, and I processed it, and then in the next packet I received the MAC and I realised the bad guys altered the data, oops" Process whole strings, the meaning of a partial string may not be what you hope, even without fancy writing systems and Unicode encoding. "Give the money to Steph" - OK, will do "...enson" - Crap, OK, somebody chase Steph and get our money back, meanwhile here's James Stephenson's money "... once he gives you the key" - Aargh. Somebody chase down James and get the key off him.
  - lanstin 7y ago
    Wow so interesting an idea. But so many streaming algorithms etc. people stick megabytes of crap into one framed message.
Dylan16807 7y ago
Sure, but you can also just stack infinite accent marks. Either way there's already a recommendation in this area. I'm not sure if it strictly fits this particular scenario but the "UAX15-D3. Stream-Safe Text Format" suggests that you should support sequences at least 31 code points long.
Latty 7y ago
To be fair, I believe this was true well before flags—diacritics played this role previously, as in "Zalgo Text".

kps 7y ago

And worse, the ‘operators’ are postfix or infix, so if you're handling text sequentially — and let's say, split across network packets with an arbitrary delay in between — you can't even tell you've read a complete character until you receive the next one. Typing in many Latin languages generates diacritics using ‘dead keys’, which are prefix operators. (Originally, on a physical typewriter, these were keys that simply struck the paper without triggering the mechanism to advance the carriage.) If Unicode had taken this hint, life would be easier. (ISO/IEC 6937 had prefix diacritics, but it was too late.)

tialaramex 7y ago
Complete recognition before processing. It's not the law, just a good idea, but you should do it anyway. In cryptography this lesson had to keep being learned. "Oh, I received this partial data, and I processed it, and then in the next packet I received the MAC and I realised the bad guys altered the data, oops" Process whole strings, the meaning of a partial string may not be what you hope, even without fancy writing systems and Unicode encoding. "Give the money to Steph" - OK, will do "...enson" - Crap, OK, somebody chase Steph and get our money back, meanwhile here's James Stephenson's money "... once he gives you the key" - Aargh. Somebody chase down James and get the key off him.
- lanstin 7y ago
  Wow so interesting an idea. But so many streaming algorithms etc. people stick megabytes of crap into one framed message.

Dylan16807 7y ago

Sure, but you can also just stack infinite accent marks. Either way there's already a recommendation in this area. I'm not sure if it strictly fits this particular scenario but the "UAX15-D3. Stream-Safe Text Format" suggests that you should support sequences at least 31 code points long.

Latty 7y ago

To be fair, I believe this was true well before flags—diacritics played this role previously, as in "Zalgo Text".