3 ms·
I think the point was that anything above 7-bit ASCII will be represented differently in Latin-1 vs UTF-8; i.e. ¢ (U+00A2) is rendered as 0xA2 in Latin1 and 0xC
by Firehed 14y ago
I think the point was that anything above 7-bit ASCII will be represented differently in Latin-1 vs UTF-8; i.e. ¢ (U+00A2) is rendered as 0xA2 in Latin1 and 0xC2A2 in UTF-8 - and 0xC2A2 in Latin1 will be displayed as ¢.
It gets far worse in 3-byte UTF8 characters, but I don't believe any of them exist natively in Latin1 (see: euro symbol)
Assuming I'm reading these various character tables right, at least ;)
So a more accurate version of what you quoted would be "UTF-8 and Latin-1 only overlap for 7-bit ASCII"
- kbolino 14y agoNot to detract from your points, all good, but: 0xC2A2 will be rendered as ¢ only if it's encoded in UTF-16/UCS-2 big endian and misinterpreted as ISO-8859-1/Windows-1252. If it's encoded in little endian (much more common on Intel x86 computers), then it would be rendered as ¢Â when misinterpreted.
- kelnos 14y agoThat doesn't really make sense. If someone's intending to encode ¢ in utf-8, endianness does not come into play, as it's a stream of octets, not of anything larger that you can chunk such that you could swap bytes. At any rate, if you were to encode ¢ in UTF-16BE, it would be 0x00a2, not 0xc2a2. If a piece of software then misinterpreted it as latin1, likely you'd get nothing at all due to the embedded NUL. $ echo -n ¢ | iconv -f UTF-8 -t UTF-16BE | hexdump -C 00000000 00 a2 |..|
- kbolino 14y ago"That doesn't really make sense." Indeed. I either completely misread the parent post, or else it said something different when I responded to it (knowing myself, I'm going with the former).