4 ms·

> Displaying one nominal glyph per Unicode character does not result in readable text for many languages Completely unreadable, even to people with a basic und

by usrusr 8y ago

> Displaying one nominal glyph per Unicode character does not result in readable text for many languages Completely unreadable, even to people with a basic understanding of the inner workings of Unicode and some practice with the font, or just not matching the norms of correctness? Decipherable but clearly not correct would mark the sweet spot between being just as bad as placeholders and being so good that it undermines efforts for having a proper font for a given language.

jfk13 8y ago
The degree of badness varies among languages/scripts. In some cases, yes, most readers would be able to decipher the intended content. In others, no, it really would be unreadable, except to those geeky enough to also comfortably "decipher" things like \x74\x68\x69\x73. Would you be happy routinely reading your terminal text in that form?
- yorwba 8y ago
  > it really would be unreadable, except to those geeky enough to also comfortably "decipher" things like \x74\x68\x69\x73 Can you give an example?
  - jfk13 8y ago
    Try something in Devanagari script such as "क्षत्रिय" and see how it looks with Unifont. The results may vary, depending whether some kind of shaping engine (like Uniscribe, Pango, HarfBuzz, etc) gets involved in the process, but I'm confident it will be far from correct, and most readers would struggle to recognise it.
    - mmastrac 8y ago
      I think your example can be illustrated by adding spaces between the characters like so: क्षत्रिय vs क् ष त्रि य (note how the first "glyph" decomposed into two drastically different characters)
      - jfk13 8y ago
        Actually, your "decomposed" version still has clusters that are different from the individual characters making them up. There's क्, which is क plus ् (OK, that one's easy); more significantly, you've left त्रि as a single unit whereas it is made up of त + ् + र + ि. So "spelled-out", क्षत्रिय is made up of क ् ष त ् र ि य . I don't know if Unifont tries to render the combining characters as zero-width overstrikes, or what, but much more than that is needed for a readable result.
        jessaustin 8y ago
        Wow thanks for spelling that out. That seems like a challenging script!
        a1369209993 8y ago
        That looks plenty readable. Ugly as hell, sure, but not any worse then 3N6|_15H \/\/R1773N 45 1337 5P34|<, for example. It's a vast improvement over [0915][094D][0937][0924][094D][0930][093F][092F].
        Arnavion 8y ago
        The problem is that the vowel ` ि` is in a misleading position in the decomposed form. It implies a spelling of kshatrayi instead of kshatriya, unless you happen to know that Unicode requires it to be there and the font engine is supposed to move it to its correct place in the ligature. (This comment is a good equivalent of the problem in English. https://news.ycombinator.com/item?id=19575942 https://news.ycombinator.com/item?id=19575942 )
    - ramses0 8y ago
      Try online here: https://fontlibrary.org/en/font/gnu-unifont https://fontlibrary.org/en/font/gnu-unifont It does look different (likely incorrect), but is definitely better than blank boxes and question marks.
      - extra88 8y ago
        > definitely better than blank boxes and question marks I disagree, I think it's better for it to be apparent when text is wrong or unreadable, even to people not literate in the language. I wondered if the problem had to do with Unifont being monospaced but I pasted "क्षत्रिय" in on the Inconsolata page and it appears correct (i.e. the lines correspond to how they look in the font on this page). https://fontlibrary.org/en/font/inconsolata https://fontlibrary.org/en/font/inconsolata
        jfk13 8y ago
        Inconsolata doesn't support devanagari at all, so when you try that, you're seeing whatever fallback font your browser chooses. (IMO, the fact that fontlibrary.org doesn't indicate this is a serious defect in its presentation.)
        extra88 8y ago
        Good point. fontlibrary.org lists each font's language support in the sidebar but since the page emphasizes the display of the font, it would be better if they prevented such fallback.
        jfk13 8y ago
        Its idea of "language support" is also less than ideal, as it just considers the character repertoire and not whether the font can actually render the characters in a typographically correct way. Thus, it claims Unifont has "support" for various South and South-East Asian languages where exactly this issue applies: the text is simply wrong when displayed linearly using a 1:1 character/glyph mapping.
      - 8y ago
chipuni 8y ago
Unreadable. Arabic has a strong difference between letters of the same word, and different words. There is no exact equivalent in English... b&u&t&i&t&m&i&g&h&t&b&e&s&o&m&e&t&h&i&n&g&l&i&k&e&t&h&i&s
- usrusr 8y ago
  If that example is representative, I'd definitely put it in the "readable" bin. Leaps and bounds from what you'd want to read, but infinitely more useful than a sequence of identical placeholders. Even just being able to tell wether two strings might be identical or not would be an improvement over blank placeholders.
- svat 8y ago
  There's also complex text layout (vowel-sign reordering) for Indic scripts, where some (not all) vowel signs require repositioning from phonetic order to visual order. Adding just that aspect to your example (without other complications like positioning of matras or conjunct consonant glyphs) makes it something closer to: b&t&u&t&i&m&g&h&t&i&b&s&e&m&o&t&e&h&n&g&i&l&k&i&t&e&h&s&i

by usrusr 8y ago

jfk13 8y ago

The degree of badness varies among languages/scripts. In some cases, yes, most readers would be able to decipher the intended content. In others, no, it really would be unreadable, except to those geeky enough to also comfortably "decipher" things like \x74\x68\x69\x73. Would you be happy routinely reading your terminal text in that form?

yorwba 8y ago
> it really would be unreadable, except to those geeky enough to also comfortably "decipher" things like \x74\x68\x69\x73 Can you give an example?
- jfk13 8y ago
  Try something in Devanagari script such as "क्षत्रिय" and see how it looks with Unifont. The results may vary, depending whether some kind of shaping engine (like Uniscribe, Pango, HarfBuzz, etc) gets involved in the process, but I'm confident it will be far from correct, and most readers would struggle to recognise it.
  - mmastrac 8y ago
    I think your example can be illustrated by adding spaces between the characters like so: क्षत्रिय vs क् ष त्रि य (note how the first "glyph" decomposed into two drastically different characters)
    - jfk13 8y ago
      Actually, your "decomposed" version still has clusters that are different from the individual characters making them up. There's क्, which is क plus ् (OK, that one's easy); more significantly, you've left त्रि as a single unit whereas it is made up of त + ् + र + ि. So "spelled-out", क्षत्रिय is made up of क ् ष त ् र ि य . I don't know if Unifont tries to render the combining characters as zero-width overstrikes, or what, but much more than that is needed for a readable result.
      - jessaustin 8y ago
        Wow thanks for spelling that out. That seems like a challenging script!
      - a1369209993 8y ago
        That looks plenty readable. Ugly as hell, sure, but not any worse then 3N6|_15H \/\/R1773N 45 1337 5P34|<, for example. It's a vast improvement over [0915][094D][0937][0924][094D][0930][093F][092F].
        Arnavion 8y ago
        The problem is that the vowel ` ि` is in a misleading position in the decomposed form. It implies a spelling of kshatrayi instead of kshatriya, unless you happen to know that Unicode requires it to be there and the font engine is supposed to move it to its correct place in the ligature. (This comment is a good equivalent of the problem in English. https://news.ycombinator.com/item?id=19575942 https://news.ycombinator.com/item?id=19575942 )
  - ramses0 8y ago
    Try online here: https://fontlibrary.org/en/font/gnu-unifont https://fontlibrary.org/en/font/gnu-unifont It does look different (likely incorrect), but is definitely better than blank boxes and question marks.
    - extra88 8y ago
      > definitely better than blank boxes and question marks I disagree, I think it's better for it to be apparent when text is wrong or unreadable, even to people not literate in the language. I wondered if the problem had to do with Unifont being monospaced but I pasted "क्षत्रिय" in on the Inconsolata page and it appears correct (i.e. the lines correspond to how they look in the font on this page). https://fontlibrary.org/en/font/inconsolata https://fontlibrary.org/en/font/inconsolata
      - jfk13 8y ago
        Inconsolata doesn't support devanagari at all, so when you try that, you're seeing whatever fallback font your browser chooses. (IMO, the fact that fontlibrary.org doesn't indicate this is a serious defect in its presentation.)
        extra88 8y ago
        Good point. fontlibrary.org lists each font's language support in the sidebar but since the page emphasizes the display of the font, it would be better if they prevented such fallback.
        jfk13 8y ago
        Its idea of "language support" is also less than ideal, as it just considers the character repertoire and not whether the font can actually render the characters in a typographically correct way. Thus, it claims Unifont has "support" for various South and South-East Asian languages where exactly this issue applies: the text is simply wrong when displayed linearly using a 1:1 character/glyph mapping.
    - 8y ago

chipuni 8y ago

Unreadable. Arabic has a strong difference between letters of the same word, and different words. There is no exact equivalent in English... b&u&t&i&t&m&i&g&h&t&b&e&s&o&m&e&t&h&i&n&g&l&i&k&e&t&h&i&s

usrusr 8y ago
If that example is representative, I'd definitely put it in the "readable" bin. Leaps and bounds from what you'd want to read, but infinitely more useful than a sequence of identical placeholders. Even just being able to tell wether two strings might be identical or not would be an improvement over blank placeholders.
svat 8y ago
There's also complex text layout (vowel-sign reordering) for Indic scripts, where some (not all) vowel signs require repositioning from phonetic order to visual order. Adding just that aspect to your example (without other complications like positioning of matras or conjunct consonant glyphs) makes it something closer to: b&t&u&t&i&m&g&h&t&i&b&s&e&m&o&t&e&h&n&g&i&l&k&i&t&e&h&s&i