3 ms·
That helps put some things in perspective. Unicode is not a well-understood topic among programmers, I've noticed (at least in the US), and for something so imp
by mtdewcmu 12y ago
That helps put some things in perspective. Unicode is not a well-understood topic among programmers, I've noticed (at least in the US), and for something so important it's surprisingly difficult to find information. The distinction between UCS-2 and UTF-16 must have made sense at one time, but it makes no sense anymore, and it's not necessarily easy to track down the explanation for why there are two 16-bit formats. Thanks.
- mikeash 12y agoThe whole thing is pretty much incomprehensible without understanding the history. It's a bit crazy how much historical baggage Unicode has managed to accumulate in just a couple of decades (and most of it from just the first few years). It sounds like you have it figured out, but just in case: UCS-2 is the original pure 16-bit encoding from the days when nobody could conceive of needing more than 65536 characters for anything. UTF-16 is the hacked up "oops, it turns out there are a lot more Chinese characters than we realized" version that tries to maintain some compatibility with software built for UCS-2. Given that there was only a five-year period between the first release of Unicode and the introduction of UTF-16, it's kind of amazing how pervasive the 16-bit encoding got. It's thoroughly baked into Java, where "char" was supposed to be a unicode character, but ended up being a unicode character or a piece of a UTF-16 surrogate pair. Windows APIs and Cocoa's NSString went through the same process. JavaScript too, and the spec still allows implementations to only support UCS-2.
- mtdewcmu 12y agoThe c and c++ apis seem to make things worse. The right solution for charset conversion seems to be iconv.