3 ms·
>3 Grapheme Cluster Boundaries >It is important to recognize that what the user thinks of as a “character”—a basic unit of a writing system for a language—may
by nabla9 3y ago
>3 Grapheme Cluster Boundaries
>It is important to recognize that what the user thinks of as a “character”—a basic unit of a writing system for a language—may not be just a single Unicode code point. Instead, that basic unit may be made up of multiple Unicode code points. To avoid ambiguity with the computer use of the term character, this is called a user-perceived character. For example, “G” + grave-accent is a user-perceived character: users think of it as a single character, yet is actually represented by two Unicode code points. These user-perceived characters are approximated by what is called a grapheme cluster, which can be determined programmatically.
- sebstefan 3y agoOh my god, is there ever anything simple about unicode
- WorldMaker 3y agoCompared to the ancient world of EBCDIC versus ASCII versus various ISO standards versus country-defined encodings versus Extended EBCDIC code pages versus Extended ASCII code pages which varied depending on operating system, nearest flag pole, network adapter, time of day, etc…: Unicode will forever be a simpler walk in the park. It's complexity is a relief compared to where we've been. It's definitely not simple, but it will forever be far simpler than what our grandmothers had to work with if they were writing international software.
- ssokolow 3y agoGive the "Indic scripts" section of https://manishearth.github.io/blog/2017/01/15/breaking-our-latin-1-assumptions/ https://manishearth.github.io/blog/2017/01/15/breaking-our-l... a read. TL;DR: Unicode is complicated because some non-Latin writing systems are complicated and those non-Latin writing systems account for over a quarter of the world's population. (They're either majority or present in India, Indonesia, Pakistan, Bangladesh, the Philippines, etc.)
- nottorp 3y ago> These user-perceived characters are approximated by what is called a grapheme cluster, which can be determined programmatically. From everything i've read or heard about unicode, "determined programmatically" is false?