3 ms·
> First of all, grapheme counting depends on normalization and there's like 6 Unicode normalization algorithms to consider, depending on locale and intended use
by Dylan16807 4y ago
> First of all, grapheme counting depends on normalization and there's like 6 Unicode normalization algorithms to consider, depending on locale and intended use/display style.
It says "The boundary specifications are stated in terms of text normalized according to Normalization Form NFD (see Unicode Standard Annex #15, “Unicode Normalization Forms” [UAX15]). In practice, normalization of the input is not required."
And there's also "Even in Normalization Form NFC, a syllable block may contain a precomposed Hangul syllable in the middle."
So are you sure normalization matters? I'm excluding K normalizations since they're not for general use and alter a bunch of characters on purpose. But also that's only NFC/NFD/NFKC/NFKD, are there two others or was "six" just a misremembering?
- rurban 4y agoYou forgot FCC and FCD. In practice only NFC is used and sometimes NFD for speed
- WorldMaker 4y ago"like 6" was intentional hyperbole, but I appreciate the technical clarifications. > I'm excluding K normalizations since they're not for general use "Not for general use" doesn't mean that they aren't somebody's use and still a complicating factor in everything else.
- Dylan16807 4y agoNobody that cares about keeping text intact uses K. And you don't have to care about how they count characters. It won't affect your system, and they're probably counting something dumb. NFC/NFD will give you the same graphemes. Or to put it another way, there's only two real normalizations, and there's some weird junk off to the side.