4 ms·

What's the Unicode codepoint of the character? Is it a single Unicode codepoint or multiple? (I tried to search for info on the web, but Chrome kept crashing.)

by kens 9y ago

What's the Unicode codepoint of the character? Is it a single Unicode codepoint or multiple? (I tried to search for info on the web, but Chrome kept crashing.) Supposedly it's a normal Telugu character, but how could that have gotten past even minimal testing in Hyderabad? Or is it some strange combining corner case? Telugu script is kind of unusual in Unicode, since it is syllabic but has separate codepoints for vowels and consonants, so character rendering is done by combining a vowel and consonant character into a totally different character. (Apologies for probably oversimplifying and mangling that explanation.) This is unlike Japanese katakana and hiragana, which have separate Unicode code points for each syllable so rendering is straightforward. https://en.wikipedia.org/wiki/Telugu_script https://en.wikipedia.org/wiki/Telugu_script

esnard 9y ago
Multiple codepoints. U+0C1C U+0C4D U+0C1E U+200C U+0C3E
- kens 9y ago
  So: "JA" "VIRAMA" "NYA" "ZERO-WIDTH NON-JOINER" "VOWEL AA" That looks like a non-trivial character. I wonder how many other crashing characters can be generated from this pattern. JA is a consonant, VIRAMA indicates a lack of vowel, NYA is another consonant, and it's non-joined(?) with the vowel AA. Can a Telugu speaker explain what that's supposed to do?
  - Manishearth 9y ago
    Approximately a couple thousand. It happens for Telugu, Devanagari, and Bengali, for pretty much any (C,C,V) choice in Telugu, for any (C,C,V) choice in Devanagari where the second consonant is 'ra', and any such choice in Telugu where the second consonant is 'ra' or 'ya'. Some vowels don't work. <consonant, virama, consonant> usually forms a ligature in most indic scripts. usually, that ligature is formed by munging the first consonant and tacking it on to the second. However, in Telugu, it works the other way around -- the first consonant stays the same, but the second consonant is munged and placed below it. As a result, many forms render <virama, consonant> as a sort of "composite combining character", with a placeholder for where the first consonant goes, and the second consonant below it. In Devanagari, the 'ra' consonant also does this, and 'ra' and 'ya' in Telugu. This also happens for all Kannada consonants, but I can't trigger the bug with Kannada. ZWNJ isn't really specified for Devanagari or Telugu; it can both make the vowel render separately or do nothing. In Bengali for some vowels it changes their form; however this bug occurs for more than just those vowels.
- MidnightRaver 9y ago
  You mean జ్ఞ‌ా?
deleted 9y ago
[deleted]
sharmajai 9y ago
<nitpick>All Unicode characters map, one-to-one, to their code points. A code point being a numeric identifier. It's a grapheme that combines multiple characters to form a unit of writing.</nitpick>
- eridius 9y ago
  <nitpick>The Unicode standard does not have a single definition for "character" because there's multiple interpretations. One reasonable interpretation is "a grapheme cluster".</nitpick> More specifically, here's what the Unicode Consortium glossary defines for "Character": > Character. (1) The smallest component of written language that has semantic value; refers to the abstract meaning and/or shape, rather than a specific shape (see also glyph), though in code tables some form of visual representation is essential for the reader’s understanding. (2) Synonym for abstract character. (3) The basic unit of encoding for the Unicode character encoding. (4) The English name for the ideographic written elements of Chinese origin. [See ideograph (2).]
  - sharmajai 9y ago
    I don't see which of those 4 definitions supports the grapheme cluster interpretation.
    - eridius 9y ago
      The very first one. é has semantic value. ´ by itself doesn't.
      - sharmajai 9y ago
        Of course it does, because it can be combined with other characters. This is the semantic meaning: https://en.wikipedia.org/wiki/Acute_accent https://en.wikipedia.org/wiki/Acute_accent
        eridius 9y ago
        An accent mark by itself has zero semantic meaning in a written context. It's a modifier. But you need to know what it's modifying in order to assign it any sort of meaning. We're talking about semantic meaning within the context of a written language, not technical details.

by kens 9y ago

esnard 9y ago

Multiple codepoints. U+0C1C U+0C4D U+0C1E U+200C U+0C3E

kens 9y ago
So: "JA" "VIRAMA" "NYA" "ZERO-WIDTH NON-JOINER" "VOWEL AA" That looks like a non-trivial character. I wonder how many other crashing characters can be generated from this pattern. JA is a consonant, VIRAMA indicates a lack of vowel, NYA is another consonant, and it's non-joined(?) with the vowel AA. Can a Telugu speaker explain what that's supposed to do?
- Manishearth 9y ago
  Approximately a couple thousand. It happens for Telugu, Devanagari, and Bengali, for pretty much any (C,C,V) choice in Telugu, for any (C,C,V) choice in Devanagari where the second consonant is 'ra', and any such choice in Telugu where the second consonant is 'ra' or 'ya'. Some vowels don't work. <consonant, virama, consonant> usually forms a ligature in most indic scripts. usually, that ligature is formed by munging the first consonant and tacking it on to the second. However, in Telugu, it works the other way around -- the first consonant stays the same, but the second consonant is munged and placed below it. As a result, many forms render <virama, consonant> as a sort of "composite combining character", with a placeholder for where the first consonant goes, and the second consonant below it. In Devanagari, the 'ra' consonant also does this, and 'ra' and 'ya' in Telugu. This also happens for all Kannada consonants, but I can't trigger the bug with Kannada. ZWNJ isn't really specified for Devanagari or Telugu; it can both make the vowel render separately or do nothing. In Bengali for some vowels it changes their form; however this bug occurs for more than just those vowels.
MidnightRaver 9y ago
You mean జ్ఞ‌ా?

deleted 9y ago

[deleted]

sharmajai 9y ago

<nitpick>All Unicode characters map, one-to-one, to their code points. A code point being a numeric identifier. It's a grapheme that combines multiple characters to form a unit of writing.</nitpick>

eridius 9y ago
<nitpick>The Unicode standard does not have a single definition for "character" because there's multiple interpretations. One reasonable interpretation is "a grapheme cluster".</nitpick> More specifically, here's what the Unicode Consortium glossary defines for "Character": > Character. (1) The smallest component of written language that has semantic value; refers to the abstract meaning and/or shape, rather than a specific shape (see also glyph), though in code tables some form of visual representation is essential for the reader’s understanding. (2) Synonym for abstract character. (3) The basic unit of encoding for the Unicode character encoding. (4) The English name for the ideographic written elements of Chinese origin. [See ideograph (2).]
- sharmajai 9y ago
  I don't see which of those 4 definitions supports the grapheme cluster interpretation.
  - eridius 9y ago
    The very first one. é has semantic value. ´ by itself doesn't.
    - sharmajai 9y ago
      Of course it does, because it can be combined with other characters. This is the semantic meaning: https://en.wikipedia.org/wiki/Acute_accent https://en.wikipedia.org/wiki/Acute_accent
      - eridius 9y ago
        An accent mark by itself has zero semantic meaning in a written context. It's a modifier. But you need to know what it's modifying in order to assign it any sort of meaning. We're talking about semantic meaning within the context of a written language, not technical details.