5 ms·
General question: Why didn't the UTF-8 boys and girls make it safe in the first place? This doesn't sound like rocket science. "This character maps to that char
by 3pt14159 17y ago
General question: Why didn't the UTF-8 boys and girls make it safe in the first place? This doesn't sound like rocket science. "This character maps to that character, this character to that one." I don't understand how we have unicode snowmen, but we can't safely round trip characters.
- flogic 17y agoThere isn't a good idea that a standards body hasn't fucked up. Maybe it's because these problems are harder than they seem. Or maybe they're just plain screwed up. Or maybe it's Adobe.
- halostatue 17y agoSome of it has to do with Han Unification (http://en.wikipedia.org/wiki/Han_unification http://en.wikipedia.org/wiki/Han_unification). Mostly, though, it's because some of these characters are overloaded. If you've got a Windows system, go into the DOS window and type "chcp 932" (you may need the Japanese language files installed). When you type '\', you'll get '¥' (making "C:\Program Files\" look like "C:¥Program Files¥"). In the systems where what become CP932 were first used, the backslash wasn't necessary in Japanese, so that character point was used to encode the yen symbol. Other systems used the backslash, so it was encoded as a different point. When JIS unified the existing Japanese code pages, it couldn't very well go back in time to change all that old data, so it merged the two encodings on many things. So, there's only one Unicode codepoint for the yen glyph ¥, but in this one encoding there's two different characters for it. This is the most blatant example of a problem with Unicode transcoding, but as far as I know, it's not the only one. See http://www.mail-archive.com/linux-utf8@nl.linux.org/msg02337.html http://www.mail-archive.com/linux-utf8@nl.linux.org/msg02337... for what could be done, but probably won't.