4 ms·

I disagree, if anything they didn't go far enough. Unifying Chinese and Japanese characters making them locale dependent was, IMO, a mistake. The kind of proble

I disagree, if anything they didn't go far enough. Unifying Chinese and Japanese characters making them locale dependent was, IMO, a mistake. The kind of problem I have with this can be seen with the so called "Turkish I" problem. The Turkish language has 4 'i's: i, I, ı and İ. Unicode decided to encode the first two using the points for latin lower case i and upper case I. In Turkish, the capitalization rules say that i and ı are lower case, and their upper case counterparts are İ and I. You can see that if you have a byte stream for which you don't know the locale, you cannot correctly apply capitalization to it. That way all 'I's would be unambiguous in all contexts. This is not as trivial a problem as it sounds[1]. This could have been avoided if Unicode had a Turkish i and I. You can extrapolate this issue to entire languages. [1] https://gizmodo.com/382026/a-cellphones-missing-dot-kills-two-people-puts-three-more-in-jail https://gizmodo.com/382026/a-cellphones-missing-dot-kills-tw...

WalterBright 9y ago
This is beyond the scope of Unicode. Unicode should be how text is displayed. If there is meaning that is not in the display of the character, then it is outside the scope of Unicode. After all, we have many uses for the letter 'a', such as a) a*b=c and b) a as in apple. Should those 'a's have different Unicode code points? Semantic meaning comes from context, and there is no context for a Unicode code point. Trying to insert semantic meaning is both a mistake and a patently impossible task. The article points out some of the wreckage attempting to do this causes.
- avian 9y ago
  > Should those 'a's have different Unicode code points? Unicode does have separate code points for mathematical symbols. See for instance U+1D44E MATHEMATICAL ITALIC SMALL A. They see little use in practice though, apart from people using them for funky fonts on Reddit and such.
  - WalterBright 9y ago
    I didn't know that, thanks for the info. But the use of a) and b) remains, as does ascii art, [a] for footnotes, it's just endless.
    - comex 9y ago
      > But the use of a) and b) remains, as does ascii art, [a] for footnotes, it's just endless. You mean like ⒜ or ⓐ or ᵃ? Not to be confused with any of ªａ𝐚𝑎𝒂𝒶𝓪𝔞𝕒𝖆𝖺𝗮𝘢𝙖𝚊ₐ. Unicode has a whole lot of stuff that nobody uses… There's also a separate Cyrillic а.
      - khaled 9y ago
        I’ve seen all of these used quite extensively. It depends on what kind of fields you are familiar with.
    - deleted 9y ago
      [deleted]
  - khaled 9y ago
    They are used by OpenType math fonts and systems that use them (e.g. the new equation editor in Word from 2007 on wards, and Unicode-capable TeX systems with the `unicode-math` package).
WalterBright 9y ago
It doesn't even work for the Turkish i. One cannot proofread the text to see if the right i is used - nobody is going to check the code point numbers.
- estebank 9y ago
  That's where defaults become important. When switching to a Turkish locale the system would try to use the Turkish version for the characters, but of course this would make it harder for Turkish speakers to switch back and forth with other languages... I guess we've always been in the days of "worse is better", like using CSV even though ASCII encodes characters specific to record separation[1]. [1]: https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text
username223 9y ago
So thanks to Unicode, things that look the same can have different representations, and things that look different can have the same one. (Limited) text used to be easy, now it's even worse than recording pen-strokes in a vector graphics format. How is this still useful?

WalterBright 9y ago

This is beyond the scope of Unicode. Unicode should be how text is displayed. If there is meaning that is not in the display of the character, then it is outside the scope of Unicode. After all, we have many uses for the letter 'a', such as a) a*b=c and b) a as in apple. Should those 'a's have different Unicode code points? Semantic meaning comes from context, and there is no context for a Unicode code point. Trying to insert semantic meaning is both a mistake and a patently impossible task. The article points out some of the wreckage attempting to do this causes.

avian 9y ago
> Should those 'a's have different Unicode code points? Unicode does have separate code points for mathematical symbols. See for instance U+1D44E MATHEMATICAL ITALIC SMALL A. They see little use in practice though, apart from people using them for funky fonts on Reddit and such.
- WalterBright 9y ago
  I didn't know that, thanks for the info. But the use of a) and b) remains, as does ascii art, [a] for footnotes, it's just endless.
  - comex 9y ago
    > But the use of a) and b) remains, as does ascii art, [a] for footnotes, it's just endless. You mean like ⒜ or ⓐ or ᵃ? Not to be confused with any of ªａ𝐚𝑎𝒂𝒶𝓪𝔞𝕒𝖆𝖺𝗮𝘢𝙖𝚊ₐ. Unicode has a whole lot of stuff that nobody uses… There's also a separate Cyrillic а.
    - khaled 9y ago
      I’ve seen all of these used quite extensively. It depends on what kind of fields you are familiar with.
  - deleted 9y ago
    [deleted]
- khaled 9y ago
  They are used by OpenType math fonts and systems that use them (e.g. the new equation editor in Word from 2007 on wards, and Unicode-capable TeX systems with the `unicode-math` package).

WalterBright 9y ago

It doesn't even work for the Turkish i. One cannot proofread the text to see if the right i is used - nobody is going to check the code point numbers.

estebank 9y ago
That's where defaults become important. When switching to a Turkish locale the system would try to use the Turkish version for the characters, but of course this would make it harder for Turkish speakers to switch back and forth with other languages... I guess we've always been in the days of "worse is better", like using CSV even though ASCII encodes characters specific to record separation[1]. [1]: https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text

username223 9y ago

So thanks to Unicode, things that look the same can have different representations, and things that look different can have the same one. (Limited) text used to be easy, now it's even worse than recording pen-strokes in a vector graphics format. How is this still useful?