4 ms·
Except that with utf-16 you still have to give up any hope of O(1) string operations, unless you don't care about accuracy. Only UTF-32 allows O(1) index based
by pwagland 8y ago
Except that with utf-16 you still have to give up any hope of O(1) string operations, unless you don't care about accuracy. Only UTF-32 allows O(1) index based operations.
For example, 🧐 (aka face with monocle) needs 2 UTF-16 'characters', since the UTF code point is U+1F9D0, or, in UTF-8: F0 9F A7 90.
And this ignored the point that others have made about combining characters, such as
woman farmer
Unicode: U+1F469 U+1F3FD U+200D U+1F33E, UTF-8: F0 9F 91 A9 F0 9F 8F BD E2 80 8D F0 9F 8C BE
That beast needs 7 UTF-16 or 15 UTF-8 "characters". I.e. 14 bytes vs 15 bytes. And this is still only one displayed character.
- bradleyjg 8y agoIn 2004 no one cared about face with monocle or woman farmer (I still don’t). The question was why did people pick UTF-16 back then. The answer is they were willing to make accuracy trade-offs for performance. Especially where they thought those trade-offs would only impact a small percentage of their users.
- kyle-rb 8y agoTo clarify the woman farmer being 14 or 15 bytes: the example given is four different combined characters, and is additionally a "medium skin tone woman farmer". A woman farmer requires the "woman" and "ear of rice" emoji, with a zero-width joiner character between them. To change the skin tone from the default yellow, a "medium skin tone" (type 4) modifier is added after the woman, but before the joiner. So the sequence "U+1F469 U+1F3FD U+200D U+1F33E" represents "woman skin-tone-4 joiner ear-of-rice". And in the UTF-8 bytes, the first four bytes are "woman", the next four are "skin-tone-4", then three for "zero-width-joiner", and finally four for "ear-of-rice". Emojipedia helpfully lists codepoints for each multi-character emoji: https://emojipedia.org/female-farmer/ https://emojipedia.org/female-farmer/