5 ms·
Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey,
by WalterBright 2y ago
Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example:
1. normalization
2. backwards running text (hey, why not add spiral running text?)
3. fonts
4. invisible characters
5. multiple code points with the same glyph
6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!)
7. made up languages (Elvish? Come on!)
8. you vote for my made-up emoticon, and I'll vote for yours!
- Retr0id 2y agoLanguage itself is a pile of ugly graffiti and ramshackle addons. It would be weird if Unicode didn't reflect this.
- jart 2y agoHey at least we got the astral planes. https://justine.lol/dox/unicode.txt https://justine.lol/dox/unicode.txt
- p_l 2y agoHow to say you don't know what Unicode is for without saying it. 1, 2, 4, 5, 6, and, unfortunately, 8, all fall under "ability to encode written text from all human languages". And that includes historical. Some of the issues (5 & 6) are due semantic difference even if the resulting glyph looks the same. Unfortunately you can't expect programmers to understand pesky little thing like languages having different writing, so you end up with normalisation to handle the fact that one system sent "a + ogonek accent" and another (properly) sent "a with ogonek" (these print the same but are semantically different!), and now you need to figure out normalisation in order to be able to compare strings. 7. just like 8 are down to proposal of specific new forms of writing to add to Unicode. Elvish had one since 1997 but only now got a tentative "we will talk about it". Klingon, which is IIRC more complete language including native speakers (...weird things happened sometimes) does not have outside of private use area. Emojis were added because they were used with incompatible encodings first, even before unicode happened, and without including something like SIXEL into unicode they were unrepresentable (and with SIXEL would lose semantic information)
- news_to_me 2y ago> "a + ogonek accent" and another (properly) sent "a with ogonek" (these print the same but are semantically different!) How can these possibly be semantically different? Isn’t the point of combining characters to create semantic characters that are the combination of those parts?
- p_l 2y agoThere's a semantic difference between "accented letter" and "different letter that happens to visually look like another language's accented letter". "Ą" in polish is not "A" with some accent. And the idea behind unicode was to preserve human written text, including keeping track of things like "this is letter A1 with an accent, but this is letter A2 that looks visually similar to A1 with accent but is different semantically". Of course then worries about code page size resulted in the stupidity of Han unification, so Unicode is a bit broken.
- Dylan16807 2y agoUnless there's some nuance I'm missing, I think you're reading too much into the word "accent". Especially because the codepoint is actually called "Combining Ogonek". And for anyone writing in Cyrillic, it's actually more accurate to use the combining form, even as its own letter, because the only precomposed form technically uses a latin A. But my main point is that I do not think there is supposed to be any semantic difference in Unicode based on whether you use precomposed or decomposed code points.
- eviks 2y agoBut it is precisely "a with some accent", you just have two ways to encode it for
- p_l 2y ago"Ą" is a separate letter in polish alfabet, not an accented variant of "A". There are writing systems where combining accents are used to represent just variation on a letter. Use of combining characters for "Ą" (and "Ć" and "Ł" and many other so-called "polish letters") is, at best, a historical artefact of trying to write them in deficient encodings.
- chthonicdaemon 2y agoAll languages are made up. For that matter, all glyphs are made up, too.
- bandie91 2y agothere is not only a quantitative difference between a conlang designed by a small group (or 1 person) and a "human" language developed organically in the span of centuries by millions of speakers, but also qualitative.
- WalterBright 2y agoWill you accept my made-up-on-the-spot language for Unicode, too?
- mnau 2y agoJust use private use area for your made-up-on-the-spot language. It's not a problem. Do you have a symbols that satisfy the unicode consortium criteria? Submit a proposal. Here is a good guide how to do is and be successfull: https://github.com/jloughry/Unicode/?tab=readme-ov-file#lessons_learnt https://github.com/jloughry/Unicode/?tab=readme-ov-file#less... Edit: I looked at some proposal for in Unicode 16. Some proposals are of poor quality (e.g. leafless tree emoji), some are good (numerals for Eastern Pwo Karen and Pa’O).
- deleted 2y ago[deleted]
- Aardwolf 2y agoFor me it's how they inconsistently, backwards-incompatibly, make some existing characters outside of the emoji-plane (and especially when in technical/mathematical blocks) render colored by default, rather than keep everything colored related in the emoji plane (making copies if needed rather than affecting old character, the semantics are very different anyway), e.g. https://imgur.com/a/Ugi7K1i https://imgur.com/a/Ugi7K1i and https://imgur.com/a/UMppZHG https://imgur.com/a/UMppZHG
- mnau 2y agoAs someone that whose native language isn't representable purely by ASCII, I celebrate it. Plus the first 128 codepoints are same as ASCII in UT-8. Is Unicode kind of messy? Sure, but that's just natural consequences of writing systems being messy. Every point you made was for a sensible reason that is in a scope of Unicode mission (representing all text in all writing systems).
- WalterBright 2y agoI'm sure that books can be printed in your language without any need for semantic information in the characters.
- mnau 2y agoYes, they can. Is it a problem that they do? I don't think so. Using semantic symbols seems far better option. Most fonts simply map multiple codepoints to a single glyph while dealing with all fun stuff like ligatures and all fun from GSUB tables (and fun company tables in fonts). Honestly, I see semantic information as an absolute win and good choice. If unicode didn't contain it, it would have to be somewhere else (or making ratehr unpleasant choices like having fj together). It's an illusion that it wouldn't. People want pretty text. Rest of the world doesn't care. They want pretty text everywhere. Instead of hating unicode, there would be hating "glyph points" plus "markup" (that would be literally everwere, from email to form editors) have all kinds of problems.
- WalterBright 2y ago> Using semantic symbols seems far better option Except it doesn't actually work. 'a' has a zillion different semantic meanings, all dependent on context. There is no crisis with somebody reading a book and misunderstanding which particular semantic meaning it has, because it is inferred from the context. Semantic meaning always comes from context, and Unicode cannot fix that. People can use the mathematical code point for 'a' instead of the text 'a' and the semantic Unicode meaning is meaningless because the reader will see, like the letter 'a' in because, that it is a text meaning. The only thing you get with multiple code points for 'a' is you can send out multiple identical appearing texts, but are different Unicode, so you can determine who leaked the memo. Unicode's extremely limited markup ability helps nobody.
- RiverCrochet 2y ago> covered with ugly graffiti and ramshackle addons Unfortunately there is plently of precendent for this ramshacklism. Like ACK/NAK - those are protocol signals, not characters! ENQ? What even is Shift In/Shift Out (SI/SO)? Then the database characters toward the end there FS, RS, GS, US. > backwards running text (hey, why not add spiral running text?) You jest, but you do have cursor positioning ANSI sequences which are designed to let text draw anywhere on your screen. And make it blink! You also don't find it weird to have a destructive "clear-screen" sequence? > glyphs defined by multiple code points I wonder when they started putting the slash across the 0 to differentiate from the O. > you vote for my made-up emoticon, and I'll vote for yours! I mean you do have the private Unicode range where you can actually do that. But before that, SIXEL graphics.
- kps 2y ago> Like ACK/NAK - those are protocol signals, not characters! American Standard Code for Information Interchange
- WalterBright 2y ago> you do have cursor positioning ANSI sequences which are designed to let text draw anywhere on your screen They are not part of the ASCII standard. > I wonder when they started putting the slash across the 0 to differentiate from the O That's an issue for fonts, not character sets.
- saagarjha 2y agoUnicode is quite elegant in its encoding too. If you're going to criticize it for its content, maybe start with talking about how ASCII also has invisible characters and those that people rarely use.
- WalterBright 2y agoUnicode encoding is elegant, until it added combining characters. As for ASCII, I am well aware of its invisible characters. I am a veteran of many many years in the trenches fighting tab wars. (Is it 4 spaces?, or 8? 3?) I eventually realized that the only solution is to never use tab characters. People use markup languages for formatting - Markdown, style sheets, Latex, etc. Unicode's markup is just a sad, crappy reinvention of actual markup languages.
- NoMoreNicksLeft 2y agoThey're all made-up languages, some were just made-up a little bit more transparently.
- yencabulator 2y agoI can't wait for when the majority of Unicode codepoints/glyphs are emojis that are no longer fashionable! That'll be a really weird relic of history, later.
- ygra 2y agoIt would probably be like other letters like þ that are no longer fashionable in some languages. Or not-so-small parts of Hanzi. Or completely dead scripts. That being said, Emoji are a drop in a bucket when it comes to the number of encoded code points. Nicely enough, by encoding emoji outside the BMP, you can now use characters from astral planes in a lot more places without software breaking.
- shepherdjerred 2y agoWhat would be the alternative? I think Unicode is pretty great. You can pretty easily imagine a world where we had a bunch of different encodings with none being dominant.
- WalterBright 2y agoThe alternative would be to strip out all the fonts, markup, semantics, directions, and normalization. Unicode should simply be a character set. And life would be much simpler.
- Findecanor 2y ago> 2. backwards running text (hey, why not add spiral running text?) Unicode encodes code points in logical order rather than visual order: the order in which text is supposed to be collated and spoken rather than the visual order. One tricky issue is when both directions exist in the same text. Unicode can encode nesting of text in one direction within another. For example, text consisting of an English word and a Hebrew word can be encoded as either the English embedded in Hebrew or the Hebrew embedded in English: both would render the same but collate differently. Is there a better way?
- WalterBright 2y agoI've seen newspapers with txet sdrawkcab in them. Note that the last sentence has text in both directions. I didn't need Unicode for that - nobody does. Uni ! c edo Why is there no Unicode markup for that?
- MisterTea 2y agoMost of this is pretty useful for reproducing a wide gamut of human language. It gets completely fucked when it comes to fonts with png's embedded in svg's and other INSANE matryoshka doll nesting of bitmap/vector rendering technologies. I also half hate emoji as it pollutes human writable text with bitmaps that are difficult to reproduce by hand on paper with a writing instrument - it's not text. I say half hate as it allows us a standard set of icons to use that can be easily rendered in-line with text or on their own.
- account42 2y ago9. color variants 10. code points where the appropriate glyph depends on the language (CJK unification)