2 ms·
> Or allow multi-byte leaders and continue forever. That loses one very useful property of UTF-8: you always know, by looking at a single byte, whether it's th
by cesarb 7y ago
> Or allow multi-byte leaders and continue forever.
That loses one very useful property of UTF-8: you always know, by looking at a single byte, whether it's the first byte of a code point or not. It also loses the property that no valid UTF-8 encoding is a subset of another valid UTF-8 encoding. It's better to stop at 36 bits (which keeps another useful property: you'll never find a 0xFF byte within an UTF-8 string, and you'll only find a 0x00 byte when it's encoding the U+0000 code point).