3 ms·
> Instead iterating through every byte like ""[n] Pedantic note: JS iterates through UTF-16 2-byte code units. It's that way for compatibility reasons (used to
by DiThi 9y ago
> Instead iterating through every byte like ""[n]
Pedantic note: JS iterates through UTF-16 2-byte code units. It's that way for compatibility reasons (used to be UCS-2), but it's the worst of both worlds: not as good as UTF-8 for ASCII, can't do operations in O(1) time like UTF-32.
- IshKebab 9y agoYeah and this means VSCode's language server protocol uses UTF-16 indices for positions too. Pain in the arse.
- chrismorgan 9y agoCalling it UTF-16 code units as distinct from UCS-2 is dubious. The water is very muddy there, because bad surrogates are allowed, thus the strings are arguably better considered UCS-2 rather than UTF-16. See https://mathiasbynens.be/notes/javascript-encoding https://mathiasbynens.be/notes/javascript-encoding for a discussion of the topic. I’m perpetually sad that a couple of big players decided to go all in on UCS-2/UTF-16 when it should already have been apparent that UTF-8 was roughly uniformly superior and that UCS-2/UTF-16 would have major issues. (I speak as one who was a child at the time, but has read a fair bit about the state of things back then. So don’t trust my yearning optimism.) The worst part about UTF-16 is how it poisoned Unicode with surrogate pairs. UTF-32 and UTF-8 are both good because they didn’t need any special consideration from Unicode to make them work. But UTF-16, extending UCS-2 as it does… ugh.