10 ms·
UTF-8 is easy to figure out. However, it's only a doorway to Unicode, and Unicode is not simple. If you do anything more than chopping codepoints and passing t
by pornel 15y ago
UTF-8 is easy to figure out. However, it's only a doorway to Unicode, and Unicode is not simple.
If you do anything more than chopping codepoints and passing them around — use a library. There's a lot of technical complexity due to quirks of Unicode and inherent complexity of world's diverse writing systems.
Avoid using concept of a "character" as much as you can, as it's fuzzy, e.g. there are combining characters and ligatures (codepoint != character).
Be aware that string comparison cannot be done just by comparing codepoints, and there are different levels of "sameness" of Unicode strings coming from different normalisations, e.g. NFC and NFKD.
Case-insensitive comparison cannot be done by lowercasing a string: http://www.moserware.com/2008/02/does-your-code-pass-turkey-test.html http://www.moserware.com/2008/02/does-your-code-pass-turkey-...
Unfortunately I don't know much about RTL text, and there's lots of traps there too (e.g. there are control characters for controlling text direction).