3 ms·
The elephant in the room is that Unicode is a steaming pile of shit. It is the source of all the complexity that results in people picking and choosing what the
by kazinator 3y ago
The elephant in the room is that Unicode is a steaming pile of shit. It is the source of all the complexity that results in people picking and choosing what they dare to try implementing, and document other things as being off the plate (the application programmer can fend for themselves as best they can, based on what of the missing cruft they need).
- jakeogh 3y agoThe lack of superscript uppercase C F Q S X Y and Z really accents U+1F4A9.
- Georgelemental 3y agoWhat would you change?
- kps 3y agoNot them, but: - Combining characters, joiners, modifiers, etc. should have been prefix, not suffix/infix, and in blocks by arity. This would have allowed detecting incomplete sequences, allowed code to skip combinations without knowing their meaning, and made dead-key input trivial. - A standard normalized form should have been specified up front. - Once blocks overflowed, subsequent blocks should have been made large enough that more that one would not be necessary. Unicode has ten different Latin blocks. You've got ‘Kana Extended-B’, ‘Kana Supplement’, ‘Kana Extended-A’ (in that order!), and ‘Small Kana Extension’. (If there's an explanation for ‘Supplement’ vs ‘Extended’ vs ‘Extension’, I've never seen it.) - Character ordering is inconsistent even within scripts, e.g. ÀÁÂ… àáâ but ĀāĂ㥹. - Assigning code points for the UTF-16 encoding is ridiculous. - The contentious cases of CJK unification should have been disambiguated promptly, before use of the originals became entrenched. (I know where to find 2K code points they could have used…) - They recently changed some characters to have ‘emoji presentation’ by default, meaning that existing text needs to be edited to preserve its original appearance.
- JdeBP 3y agoDead-key input, if done according to ISO 9995, is trivial for exactly that reason. Type all of the combining diacritics, then the base character. * https://jdebp.uk/Softwares/nosh/guide/commands/console-fb-realizer.xml#DEAD-KEYS https://jdebp.uk/Softwares/nosh/guide/commands/console-fb-re...
- cryptonector 3y ago> - Combining characters, joiners, modifiers, etc. should have been prefix, not suffix/infix, and in blocks by arity. This would have allowed detecting incomplete sequences, allowed code to skip combinations without knowing their meaning, and made dead-key input trivial. Yes, this is a very good suggestion. UTF-8 is a multi-byte encoding that is self-synchronizing in either direction. Multi-codepoint characters/glyphs should have been similarly self-synchronizing in either direction. > - A standard normalized form should have been specified up front. There are four. The only reasonable simplifications here would have been to get rid of NFC and NFKC, and maybe NFKD, leaving only NFD, and then maybe also remove all precompositions (but this would have made take-up of Unicode take much longer because it would have required more complexity in implementations earlier). > - Once blocks overflowed, subsequent blocks should have been made large enough that more that one would not be necessary. Unicode has ten different Latin blocks. You've got ‘Kana Extended-B’, ‘Kana Supplement’, ‘Kana Extended-A’ (in that order!), and ‘Small Kana Extension’. That's a function of how allocations were made, especially before UTF-8 and UTF-16, when we were stuck with just the BMP. Well, anyways, we'd need a time machine to get any of these things fixed now. > - Character ordering is inconsistent even within scripts, e.g. ÀÁÂ… àáâ but ĀāĂ㥹. Do you mean codepoint ordering? Hardly matters: the actual ordering will vary by locale anyways. > - Assigning code points for the UTF-16 encoding is ridiculous. That's not a thing. > - The contentious cases of CJK unification should have been disambiguated promptly. (I know where to find 2K code points they could have used…) That would have required getting representatives from those countries to sit down at the UC table sooner. > - They recently changed some characters to have ‘emoji presentation’ by default, meaning that existing text needs to be edited to preserve its original appearance. Blech. Occasionally the UC has to make backwards-incompatible changes, but they should try really hard to keep those to the absolute bare minimum -- zero would be ideal.
- kazinator 3y agoI would throw out all obscure languages and useless symbols. One character = one code point. I would use 32 bits for a code point. The lower 16 bits would be code; the upper 16 bits contain some flags and fields for classification. This would allow simple bit operations to inquire about important properties. There would be a 4-bit flexible type code, leaving 12 bits, some of which would have a code-specific meaning, others fixed. Or something like that. The goal would be to have a code that programs can work with, without requiring megabytes and megabytes of meta-data about the code.
- cryptonector 3y agoIt's not Unicode's fault. Even if we got rid of precompositions there would still be a tremendous amount of complexity in Unicode that all derives from the complexity of the scripts that humans use every day (as well as scripts of dead languages that we choose to support still). That a and ´ and so on are related is a feature of the Latin scripts, and the need/desire to have a coding system that can express that relatedness would exist even if you object to that desire. Almost everything about Unicode that people love to hate comes from humans who have nothing to do with Unicode: - composition (thus multi-codepoint glyphs) - weird collation rules - weird equivalence rules - the need for normalization and/or normalization-insensitive string hashing and comparison Even emoji are like this because emoji are really just a new kind of script (with the extra feature of color, which all other scripts lack). The only complexity that could have been elided would have been pre-compositions and NFC and NFKC, and maybe the K mappings so that NFKD also could be dispensed with. Everything else would still necessarily be a part of Unicode or any competitor to Unicode. And no, lots of codesets, one per-script, does not work as an alternative -- we tried it and it sucks, and that's why Unicode exists. To complain about Unicode's complexity is to tilt at windmills.
- kazinator 3y ago> scripts of dead languages that we choose to support ... but it's not Unicode's fault! > To complain about Unicode's complexity is to tilt at windmills. A specification that people don't want to fully support is a bad specification; Complaining about programs and tools that don't implement everything there is to Unicode is what is tilting at windmills. At the end of the day, the programs win; they determine the functionality you have, not the specification. A specification that people don't want to fully embrace is a bad specification.
- cryptonector 3y ago> > scripts of dead languages that we choose to support > ... but it's not Unicode's fault! Right, it's not. > [the rest] You say that as if current support for Unicode will not improve, as if somehow it has never improved (but then, even what we have now wouldn't exist). That's clearly not the case.