4 ms·

Unicode is "designed to support the use of text in all of the world's writing systems that can be digitized" Unicode needs tab, space, form feed, and carriage

Unicode is "designed to support the use of text in all of the world's writing systems that can be digitized" Unicode needs tab, space, form feed, and carriage return. Unicode needs U+200E LEFT-TO-RIGHT MARK and U+200F RIGHT-TO-LEFT MARK to switch between left-to-right and right-to-left languages. Unicode needs U+115F HANGUL CHOSEONG FILLER and U+1160 HANGUL JUNGSEONG FILLER to typeset Korean. Unicode needs U+200C ZERO WIDTH NON-JOINER to encode that two characters should not be connected by a ligature. Unicode needs U+200B ZERO WIDTH SPACE to indicate a word break opportunity without actually inserting a visible space. Unicode needs MONGOLIAN FREE VARIATION SELECTORs to encode the traditional Mongolian alphabet.

WalterBright 7mo ago
[flagged]
- WalterBright 7mo ago
  Look Ma xt! N ! e tee S T larip (No Unicode needed.)
- bulbar 7mo ago
  That's a very narrow view of the world. One example: In the past I have handled bilingual english-arabic files with switches within the same line and Arabic is written from left to right. There are also languages that are written from to to bottom. Unicode is not exclusively for coding, to the contrary, pretty sure it's only a small fraction of how Unicode is used. > Somehow people didn't need invisible characters when printing books. They didn't need computers either so "was seemingly not needed in the past" is not a good argument.
  - pibaker 7mo ago
    > That's a very narrow view of the world. But not one that would surprise anyone familiar with WalterBright's antics on this website…
    - WalterBright 7mo ago
      At least my antics do not include insulting people.
  - WalterBright 7mo ago
    > That's a very narrow view of the world. Yes, it is. Unicode has undergone major mission creep, thinking it is now a font language and a formatting language. Naturally, this has lead to making it a vector for malicious actors. (The direction reversing thing has been used to insert malicious text that isn't visible to the reader.) > Unicode is not exclusively for coding I never mentioned coding. > They didn't need computers Unicode is for characters, not formatting. Formatting is what HTML is for, and many other formatting standards. Neither is it for meaning.
- chongli 7mo ago
  Unicode is for human beings, not machines.
  - WalterBright 7mo ago
    How does invisible Unicode text fit into that?
    - chongli 7mo ago
      It's not text, it's control characters, which have always been in character sets going back to ASCII.
      - WalterBright 7mo ago
        ASCII having a few obsolete control characters does not justify opening the floodgates.
        zorpner 7mo ago
        Over 25% of the original ASCII specification is control characters.
        WalterBright 7mo ago
        True. And nearly all of them are obsolete. Many were intended for control flow on an interactive terminal, which have long since passed into obsolescence. When was the last time you embedded a CTRL-C in text? The only ones that matter any more are newline and space.
- jmusall 7mo ago
  The fact is that there were so many character sets in use before Unicode because all these things were needed or at least wanted by a lot of people. Here's a great blog post by Nikita Prokopov about it: https://tonsky.me/blog/unicode/ https://tonsky.me/blog/unicode/
  - WalterBright 7mo ago
    Sometimes you gotta say no. Trying to please every hare brained idea leads to madness. Normalized code point sequences are another WTF feature.
    - jmusall 7mo ago
      Of course! I bet there are tons of ideas that didn't make it into Unicode, for better of worse. Where you draw the line is kind of arbitrary. You, personally, can of course opt out of all of that by restricting yourself to ASCII only, for example. But the rest of the world will continue to use Unicode.
      - WalterBright 7mo ago
        > restricting yourself to ASCII only My early compilers used code pages to work with Japanese, French and German customers. The original idea of Unicode was absolutely brilliant and I was all for it. D was an early total adopter of Unicode (C and C++ followed years later). I rejected code page support for D. It's mission was to support all the letters in all the languages, which was a good straightforward mission. But then came fonts, formatting, layout, rendering, casing, sort ordering, normalization, combining, vote-for-my-letter-and-Ill-vote-for-yours, emoji, icons, semantic meanings, elvish, people who invent things and campaign to put them in so they'll leave a mark in history, ...

WalterBright 7mo ago

[flagged]

WalterBright 7mo ago
Look Ma xt! N ! e tee S T larip (No Unicode needed.)
bulbar 7mo ago
That's a very narrow view of the world. One example: In the past I have handled bilingual english-arabic files with switches within the same line and Arabic is written from left to right. There are also languages that are written from to to bottom. Unicode is not exclusively for coding, to the contrary, pretty sure it's only a small fraction of how Unicode is used. > Somehow people didn't need invisible characters when printing books. They didn't need computers either so "was seemingly not needed in the past" is not a good argument.
- pibaker 7mo ago
  > That's a very narrow view of the world. But not one that would surprise anyone familiar with WalterBright's antics on this website…
  - WalterBright 7mo ago
    At least my antics do not include insulting people.
- WalterBright 7mo ago
  > That's a very narrow view of the world. Yes, it is. Unicode has undergone major mission creep, thinking it is now a font language and a formatting language. Naturally, this has lead to making it a vector for malicious actors. (The direction reversing thing has been used to insert malicious text that isn't visible to the reader.) > Unicode is not exclusively for coding I never mentioned coding. > They didn't need computers Unicode is for characters, not formatting. Formatting is what HTML is for, and many other formatting standards. Neither is it for meaning.
chongli 7mo ago
Unicode is for human beings, not machines.
- WalterBright 7mo ago
  How does invisible Unicode text fit into that?
  - chongli 7mo ago
    It's not text, it's control characters, which have always been in character sets going back to ASCII.
    - WalterBright 7mo ago
      ASCII having a few obsolete control characters does not justify opening the floodgates.
      - zorpner 7mo ago
        Over 25% of the original ASCII specification is control characters.
        WalterBright 7mo ago
        True. And nearly all of them are obsolete. Many were intended for control flow on an interactive terminal, which have long since passed into obsolescence. When was the last time you embedded a CTRL-C in text? The only ones that matter any more are newline and space.
jmusall 7mo ago
The fact is that there were so many character sets in use before Unicode because all these things were needed or at least wanted by a lot of people. Here's a great blog post by Nikita Prokopov about it: https://tonsky.me/blog/unicode/ https://tonsky.me/blog/unicode/
- WalterBright 7mo ago
  Sometimes you gotta say no. Trying to please every hare brained idea leads to madness. Normalized code point sequences are another WTF feature.
  - jmusall 7mo ago
    Of course! I bet there are tons of ideas that didn't make it into Unicode, for better of worse. Where you draw the line is kind of arbitrary. You, personally, can of course opt out of all of that by restricting yourself to ASCII only, for example. But the rest of the world will continue to use Unicode.
    - WalterBright 7mo ago
      > restricting yourself to ASCII only My early compilers used code pages to work with Japanese, French and German customers. The original idea of Unicode was absolutely brilliant and I was all for it. D was an early total adopter of Unicode (C and C++ followed years later). I rejected code page support for D. It's mission was to support all the letters in all the languages, which was a good straightforward mission. But then came fonts, formatting, layout, rendering, casing, sort ordering, normalization, combining, vote-for-my-letter-and-Ill-vote-for-yours, emoji, icons, semantic meanings, elvish, people who invent things and campaign to put them in so they'll leave a mark in history, ...