3 ms·
User-perceived characters are not graphemes, they are grapheme clusters. You can look at unicode stings in at least four different levels of abstraction: bytes
by nabla9 12y ago
User-perceived characters are not graphemes, they are grapheme clusters.
You can look at unicode stings in at least four different levels of abstraction: bytes, code points, code units and grapheme clusters. Only advantage UTF-32 has over others is that allcode units fit into single code point (atleast I think so)
If you want a vector where each user-perceived character and whitespace matches one element in the vector, probably the easiest way is to create vector where each element is short unicode string that matches grapheme cluster.
- deathanatos 12y ago> Only advantage UTF-32 has over others is that allcode units fit into single code point (atleast I think so) All code points can be encoded as a single code unit in UTF-32. Code points are the things like U+0065 LATIN SMALL LETTER E; code units are what you encode code points as in a given Unicode encoding — i.e., octets in UTF-8 and 32-bit integers in UTF-32.
- nabla9 12y agoYes. Thank you. Things that don't necessarily fit into single UTF-32 code unit: combining character sequence and grapheme cluster.
- masklinn 12y ago> You can look at unicode stings in at least four different levels of abstraction: bytes, code points, code units and grapheme clusters. There's also glyphs, which is what you get after the final rendering through a font, and which may not map 1:1 to any other level of abstraction.