3 ms·
> Many charsets decode fine as UTF-8 as long as the message happens to fit in ASCII. At which point the message is effectively ASCII. UTF-8 is a superset of AS
by duskwuff 3y ago
> Many charsets decode fine as UTF-8 as long as the message happens to fit in ASCII.
At which point the message is effectively ASCII. UTF-8 is a superset of ASCII, so "decoding" ASCII as UTF-8 is fine.
(Yes, I know there are some Japanese text encodings where 0x5c is decoded as "¥" instead of "\". But they're sometimes treated as backslashes even though they look like ¥ symbols so handling them "correctly" is complicated.)
- Izkata 3y agoI'm guessing they're thinking of Extended ASCII (the 8-bit one that's actually multiple different encodings, but the lower half is shared with 7-bit ASCII and so that part does fit in UTF-8 while the upper half likely won't if the message actually uses it).
- actionfromafar 3y agoYes. And some test messages will decode fine and then suddenly no.
- eqvinox 3y ago"Fun" fact: some video subtitle formats (ASS specifically) use "text{\b1}bold" to format things — but since they were primarily used to subtitle Japanese anime, this frequently became "text{¥b1}bold". Which is all good and well, except when those subtitles moved to UTF-8 they kept the ¥. So now you have to support ¥ (0xC2 0xA5) as a markup/control character in those subtitles.
- jwilk 3y agoISO-2022-JP (sometimes?) disguises perfectly as ASCII: $ echo は | iconv -t ISO-2022-JP | hd 00000000 1b 24 42 24 4f 1b 28 42 0a |.$B$O.(B.| 00000009