3 ms·
Why would you need to be compatible with "all kinds of Unicode", by which I assume you mean different encodings? You just need to be fully compatible with one
by rspeer 10y ago
Why would you need to be compatible with "all kinds of Unicode", by which I assume you mean different encodings?
You just need to be fully compatible with one kind of Unicode (UTF-8) and use it consistently. And null-terminated UTF-8 is no more or less safe than null-terminated ASCII, because the only UTF-8 codepoint that uses 0x00 is the same one as in ASCII.
- fauigerzigerk 10y agoThat's true provided you validate all UTF-8 input. As you probably know, UTF-8 has a lead byte that indicates how many of the following bytes are part of the same code point. If your code blindly believes what that first byte claims, then you could easily get tricked into reading beyond '\0'.
- deleted 10y ago[deleted]
- rspeer 10y agoYes, you should validate the UTF-8 at some point before you actually manipulate the contents of the string. Before that, you can just treat it as a pile of bytes. You don't need to go down different paths of how you read those bytes based on the codepoint lengths until you're actually doing something with codepoints.