5 ms·
Isn't utf-8 just a string of bytes where each byte represents a sign? Why is it hard?
by tormeh 13y ago
Isn't utf-8 just a string of bytes where each byte represents a sign? Why is it hard?
- sp332 13y agoNo. http://en.wikipedia.org/wiki/UTF-8#Description http://en.wikipedia.org/wiki/UTF-8#Description How could you encode millions of characters into just 256 values?
- tormeh 13y agoHonestly, I thought that was what utf-16 was for. I thought the number was the bitlength of a sign.
- ygra 13y agoThe number says how wide a code unit is. However, it doesn't say anything about how many code units are required to encode a single code point. UTF-8 needs 1–4, UTF-16 needs 1–2 and UTF-32 needs 1. They all are able to represent all encoded characters of Unicode, it's just that the individual bytes are different.
- _ZeD_ 13y agobecause is not.
- kalleboo 13y agoNo, single characters can be represented by a string of multiple bytes. And you can construct an invalid string of bytes that would cause a Unicode error. The problem comes from what to you in these applications when you come across a Unicode error. Such as, if the user tries to open a file where the filename contains a string of bytes that would be invalid Unicode. Refuse to open the file?
- tormeh 13y agoIsn't the standard approach to just show a box or similar gibberish? I've seen enough of those.
- ygra 13y agoBoxes are just the font not having that glyph – that doesn't destroy data. Question marks, U+FFFD or random gibberish are arguably at least as disastrous as refusing to open a file (maybe more because you don't actually notice something is wrong).