5 ms·

Oh, cool! Some more links for my "Unicode is a tire fire" collection. (And no, I don't want ASCII back, but finally something sane).

mananaysiempre 4y ago
Honestly, to me it looks more like human writing is much more complex than typewriter-tyranny Latin would lead you to believe than that Unicode is a tire fire. (OpenType is a tire fire, though, befitting its history as a final peace treaty between Adobe, Apple, and Microsoft after a decade or two of shifting two-against-one alliances. Good luck convincing the half-dozen or so main font foundries to switch to something else.) I mean, look at Gujarati[1] and tell me how that thing is ever supposed to fit into character cells. Or think how Hebrew (RTL) text with (LTR) numbers should handle arrow keys in selections[2]—the problem was born in cursed depths where unspeakable ancient monsters breed, you lost (possibly your mind) the moment you decided to solve it. Watch a Japanese user type a romanization of their sentence on a QWERTY keyboard, displayed using the syllabary, and then have the computer guess the word boundaries and offer the appropriate ideograms in a little systemwide overlay[3]—watch and despair. And none of these have anything to do with the way you’ve encoded the strings into bytes. (Unless you tell the Hebrew users to type or process text in reverse, which is also utter insanity.) [1] https://r12a.github.io/scripts/gujarati/gu.html https://r12a.github.io/scripts/gujarati/gu.html [2] https://ltr.wtf/explained/bidiintro.html#selections https://ltr.wtf/explained/bidiintro.html#selections [3] https://en.wikipedia.org/wiki/File:IME_demonstratie_-_Matsuo_Bashou_-_Furu_ikeya_kawazu_tobikomu_mizuno_oto.png https://en.wikipedia.org/wiki/File:IME_demonstratie_-_Matsuo...
- jandrese 4y ago
  I've never seen a Unicode editor that handled Japanese top-to-bottom right-to-left oriented text. Setting up a terminal to switch between Japanese, Farsi, and German on the same screen would be quite a challenge. That's hard even on paper.
- still_grokking 4y ago
  The bit encoding is only a very small part of Unicode. That alone would be likely bearable. It's the whole thing that is problematic. Unicode mixes way to many concepts in an unholy way. The result is insanity. The original sin is of course that content and appearance / layout aren't strictly separated. The whole idea that a text format should be "typewriter compatible" is also a huge part of the problem. Human script is way too complex that one could pretend that a typewriter abstraction would suffice. But Unicode tries hard to keep this leaky abstraction (because "backwards compatibility"; even it's actually not compatible in all kinds of "corner cases", BIDI being only one). The result is the dumpster fire we've got (and that we likely never ever will get rid of, as it's "good enough" for the majority; I mean as long as you're using European scripts, and no scientific notation, math, and such…).
a1369209993 4y ago
> And no, I don't want ASCII back, but finally something sane ASCII was sane (give or take minor annoyances like hexadecimal digits or DEL); the problem was it was too narrowly scoped for general-purpose use. In the context it was actually developed for (exclusively English text on machines that struggled (or didn't bother) to even render 256 different characters, nevermind thousands), ASCII was a broadly-reasonable character encoding that even had half the encoding space left over for something like UTF-8 to build a sensible, more general-purpose encoding in. Unfortunately, we got specifically UTF-8, with the dumpster fire that is Unicode tied around its neck like a string of albatrosses.
- rockwotj 4y ago
  Speaking of "getting" UTF8, I really enjoy this story on it's inception: https://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt https://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt

mananaysiempre 4y ago

Honestly, to me it looks more like human writing is much more complex than typewriter-tyranny Latin would lead you to believe than that Unicode is a tire fire. (OpenType is a tire fire, though, befitting its history as a final peace treaty between Adobe, Apple, and Microsoft after a decade or two of shifting two-against-one alliances. Good luck convincing the half-dozen or so main font foundries to switch to something else.) I mean, look at Gujarati[1] and tell me how that thing is ever supposed to fit into character cells. Or think how Hebrew (RTL) text with (LTR) numbers should handle arrow keys in selections[2]—the problem was born in cursed depths where unspeakable ancient monsters breed, you lost (possibly your mind) the moment you decided to solve it. Watch a Japanese user type a romanization of their sentence on a QWERTY keyboard, displayed using the syllabary, and then have the computer guess the word boundaries and offer the appropriate ideograms in a little systemwide overlay[3]—watch and despair. And none of these have anything to do with the way you’ve encoded the strings into bytes. (Unless you tell the Hebrew users to type or process text in reverse, which is also utter insanity.) [1] https://r12a.github.io/scripts/gujarati/gu.html https://r12a.github.io/scripts/gujarati/gu.html [2] https://ltr.wtf/explained/bidiintro.html#selections https://ltr.wtf/explained/bidiintro.html#selections [3] https://en.wikipedia.org/wiki/File:IME_demonstratie_-_Matsuo_Bashou_-_Furu_ikeya_kawazu_tobikomu_mizuno_oto.png https://en.wikipedia.org/wiki/File:IME_demonstratie_-_Matsuo...

jandrese 4y ago
I've never seen a Unicode editor that handled Japanese top-to-bottom right-to-left oriented text. Setting up a terminal to switch between Japanese, Farsi, and German on the same screen would be quite a challenge. That's hard even on paper.
still_grokking 4y ago
The bit encoding is only a very small part of Unicode. That alone would be likely bearable. It's the whole thing that is problematic. Unicode mixes way to many concepts in an unholy way. The result is insanity. The original sin is of course that content and appearance / layout aren't strictly separated. The whole idea that a text format should be "typewriter compatible" is also a huge part of the problem. Human script is way too complex that one could pretend that a typewriter abstraction would suffice. But Unicode tries hard to keep this leaky abstraction (because "backwards compatibility"; even it's actually not compatible in all kinds of "corner cases", BIDI being only one). The result is the dumpster fire we've got (and that we likely never ever will get rid of, as it's "good enough" for the majority; I mean as long as you're using European scripts, and no scientific notation, math, and such…).

a1369209993 4y ago

> And no, I don't want ASCII back, but finally something sane ASCII was sane (give or take minor annoyances like hexadecimal digits or DEL); the problem was it was too narrowly scoped for general-purpose use. In the context it was actually developed for (exclusively English text on machines that struggled (or didn't bother) to even render 256 different characters, nevermind thousands), ASCII was a broadly-reasonable character encoding that even had half the encoding space left over for something like UTF-8 to build a sensible, more general-purpose encoding in. Unfortunately, we got specifically UTF-8, with the dumpster fire that is Unicode tied around its neck like a string of albatrosses.

rockwotj 4y ago
Speaking of "getting" UTF8, I really enjoy this story on it's inception: https://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt https://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt