12 ms·
> Not UTF-8 characters, UTF-8 code points. UTF-8 characters (more properly "extended grapheme clusters") can be of any length. Technically the only thing UTF-8
by anonymfus 6y ago
> Not UTF-8 characters, UTF-8 code points. UTF-8 characters (more properly "extended grapheme clusters") can be of any length.
Technically the only thing UTF-8 encodes is Unicode code points, as higher level abstractions like grapheme clusters are irrelevant for it. UTF-8 parser's job is to turn a sequence of octets to a sequence of code points.
So-called "user-perceived characters" approximated as grapheme clusters is a relatively recent innovation in Unicode terminology. Compare the modern version of the spec (https://www.unicode.org/reports/tr29/ https://www.unicode.org/reports/tr29/) since Unicode 5.1.0 (2008):
> It is important to recognize that what the user thinks of as a “character”—a basic unit of a writing system for a language—may not be just a single Unicode code point. Instead, that basic unit may be made up of multiple Unicode code points. To avoid ambiguity with the computer use of the term character, this is called a user-perceived character. For example, “G” + grave-accent is a user-perceived character: users think of it as a single character, yet is actually represented by two Unicode code points. These user-perceived characters are approximated by what is called a grapheme cluster, which can be determined programmatically.
With the previous one (https://www.unicode.org/reports/tr29/tr29-11.html https://www.unicode.org/reports/tr29/tr29-11.html) from Unicode 5.0.0 (2006):
> One or more Unicode characters may make up what the user thinks of as a character or basic unit of the language. To avoid ambiguity with the computer use of the term character, this is called a grapheme cluster. For example, “G” + acute-accent is a grapheme cluster: it is thought of as a single character by users, yet is actually represented by two Unicode code points.
So you can notice "the computer use of the term character" in both. Notice also that these grave and acute accents used as examples existed in pre-Unicode encodings too, and we still call them "combining characters".
Hence if we adhere to that old "computer use of the term character" then "UTF-8 characters" are Unicode code points, and "Unicode characters" don't exist; but if we use a modern colloquial meaning of characters as Unicode's "user-perceived characters" then "UTF-8 characters" don't exist.