3 ms·
Encoding text is not the problem with Unicode. The problem is the complexity at the higher abstraction level. At the top level of abstraction is the abstract c
by nabla9 10y ago
Encoding text is not the problem with Unicode. The problem is the complexity at the higher abstraction level.
At the top level of abstraction is the abstract character aka user perceived character and grapheme clusters (a sequence of coded characters that should be kept together). Mapping from code-points to abstract characters is not total, injective, or surjective.
Unicode is not just standard for encoding. It's also standard for representation, and handling text.
The exact semantics of UTF-8 string in all cases is something that only few programmers are able to comprehend. I know for sure that I don't and I don't know anyone who does. Interchanging and storing UTF-8 strings and hoping for the best is the standard practice and it works well, but it shows the overreach that Unicode standard is.