2 ms·
It is true that Javascript, like many other languages (Java, Win32 wide chars, etc) has to deal with the problem that they assumed unicode code points could not
by 0x0 9y ago
It is true that Javascript, like many other languages (Java, Win32 wide chars, etc) has to deal with the problem that they assumed unicode code points could not exceed the integer value 65535, so you have to deal with the surrogate pairs. So I guess that is one way to "encode" all planes of unicode in a backward compatible way. Bytes don't enter the picture though!
It is still very important in general to distinguish a string from a byte array, and I felt like the article was fairly counter-productive with all its talk about "UTF-8 strings" (which doesn't make much sense, either you have a byte-array that you can apply UTF-8 decoding on to get a string out of, or you already have a string, in which case encodings in the traditional sense (UTF-8, ISO-8859-1 latin1, etc) doesn't apply)
- burntsushi 9y ago> was fairly counter-productive with all its talk about "UTF-8 strings" Eh. I agree and disagree. I agree in the sense that the phrase "UTF-8 string" is generally a misnomer and is a good signal that there's some confusion somewhere, but I don't find think I find it as damning as you do. In particular, not all languages represent strings as sequences of codepoints, and instead make their internal UTF-8 byte representation a first class part of their string API. Two languages that come to mind are Go and Rust, where Go's conventionally uses UTF-8 and Rust uses enforced UTF-8. But in both cases, accessing the raw bytes is not only a standard operation, but is necessary whenever you want to do high performance string processing. That is, if someone said Go/Rust had "UTF-8 strings," that wouldn't be altogether wrong. UTF-8 is a first class aspect of both string APIs, while both provide standard functions you'd expect from Unicode strings.