5 ms·
"That's only for codepoints. Actual characters you see on screen are often combinations of codepoints." So, there is no encoding that actually has 1 word (of w
by protomyth 11y ago
"That's only for codepoints. Actual characters you see on screen are often combinations of codepoints."
So, there is no encoding that actually has 1 word (of whatever byte length) to a character on the screen?
- colanderman 11y agoYou might be interested in the Supercombiner: http://sbp.so/supercombiner http://sbp.so/supercombiner
- Aardwolf 11y agoWhat language uses that many diacritics tho? Unicode is supposed to support all written languages - supporting a u with 100 diacritics seems beyond that scope :)
- colanderman 11y agoMathematics. Not 100, but symbols are often decorated with 3 diacritics or so. That's a lot to cram into one fixed-size word.
- gsnedders 11y agoThere's an unbounded number of possible graphemes per Unicode, because they are formed of a start character and an unbounded number of combining characters following them. As such, no fixed-length encoding can actually enumerate all possible graphemes.
- protomyth 11y agoWhat is the practicality of an unbounded number of possible graphemes? Does anyone in real writing of languages and not some concocted test case need this? I understood there is some conflict in the encoding of kanji into unicode, does this relate?
- masklinn 11y ago> What is the practicality of an unbounded number of possible graphemes? That question doesn't really make sense. Unbounded cluster sizes is simply a result of unicode design because there's no reason to bound it. > Does anyone in real writing of languages and not some concocted test case need this? In the real writing of languages there are languages with grapheme clusters of size 4+ e.g. क्षि (Devanagari kshi) is a single cluster made up of क, ्, ष and ि. > I understood there is some conflict in the encoding of kanji into unicode, does this relate? It's unrelated. Han Unification compressed the number of clusters by unifying han characters across simplified Chinese, traditional Chinese, Japanese and Korean. Han characters were an issue in the original 16 bits unicode because supporting all of them was estimated as possibly topping out 100,000 codepoints (where 16 bits unicode only supported 65k codepoints). AFAIK Han characters don't use much if any composition (outside of romanisation). Brahmic scripts, Hebrew and Arabic scripts, on the other hand, pile on diacritics.
- protomyth 11y ago> That question doesn't really make sense. Unbounded cluster sizes is simply a result of unicode design because there's no reason to bound it. If I'm trying to draw a character on the screen, it makes a lot of sense. I guess I'm struck about how hard it would be to select the 4th through 8th character displayed on the screen from rows 3 through 13.
- lambda 11y agoDisplaying characters on the screen, and giving the ability to select text, is indeed quite a difficult and complex question in the face of full support of the world's written languages. Glyphs vary in width. There are combining characters. There are ligatures (and in some languages, ligatures are necessary for the text to be readable, not just stylistic). There is contextual reordering of characters. There's bi-directional text, which can switch direction mid-sentence when quoting some Roman text in Hebrew or vice versa. There's vertical text. There are a number of fonts, font weights, font styles, font sizes, font features like alternative characters, language dependent selection of glyphs, and so on, plus fallback if the user doesn't have the designated fonts or fonts that cover the given Unicode range installed. There's line breaking, which can't always operate on spaces because not all languages include spaces between words. There's justification of text, distributing the spacing between characters to make a block of text even on both sides. All of this complexity exists regardless of the encoding that you use. Simply picking a fixed width encoding will not do anything about the vast majority of the complexity; and the fixed width encoding will cause problems of its own (size bloat of strings in memory), as well as leading people astray in thinking that maybe they can get away with making some of the same assumptions that apply to ASCII on a fixed-width terminal (and even there, assuming that 1 character equals 1 unit of width fails in the case of tabs), when in fact they can't and most text handling beyond simply appending strings together is best handled by dedicated libraries that have had hundreds of developer-years of effort put into solving all of these problems.