3 ms·

> Never ending madness. [Unicode] didn't start the fire; it was always burning, since the world's been turning. (I.e. the "never-ending madness" is in the con

by derefr 3y ago

> Never ending madness. [Unicode] didn't start the fire; it was always burning, since the world's been turning. (I.e. the "never-ending madness" is in the continual desire of human cultures to invent new characters with unique semantics and then have them faithfully digitally encoded in a way that carries those semantics. Being upset that this makes the management of a universal text encoding a never-ending job, is a bit like being upset that the Library of Congress always needs reorganizing and expanding because it keeps getting new books.)

WalterBright 3y ago
It's different in that nobody expects anyone to implement their own copy of the LoC. This constant expansion of the character set puts an enormous burden on a very large collection of developers. The inevitable happens - they just don't bother - making full Unicode support a pipe dream.
- derefr 3y ago
  I'm perplexed in what you think "full support" for a novel Unicode codepoint entails. Most of the properties of a novel codepoint that matter, are attributes that the codepoint has defined within various Unicode-Consortium-defined data files. If your custom implementation of Unicode support already parses those data files, then a new codepoint is essentially "absorbed" automatically as soon as the new version of the Unicode data files are made available to your implementation. Are you just complaining that new codepoints don't automatically receive glyphs in major fonts? To that, I would say: who cares? As I just said upthread — character encoding is about semantics, not about rendering. A character doesn't have to be renderable to be useful! The character is still doing its job — "transmitting meaning into the future" — when it's showing up as a U+FFFD replacement character! Screen readers can still read it! Search engines can still index on it! People who want to know what the character was "meant to mean" can copy and paste it into a Unicode database website! It still sorts and pattern-matches correctly! And so forth. Rendering the codepoint as a glyph — if it even makes sense to render it standalone as a glyph (for which some codepoints don't, e.g. the country-code characters that only have graphical representations in ligatured pairs) — can come at some arbitrary later point, when there's enough non-niche, non-academic, supply-side usage of the codepoint, that there arises real demand to see that codepoint rendered as something. Until then, just documenting the semantics as a codepoint and otherwise leaving it un-rendered, works fine for the needs of the anthropologists and archivists who "legibilize" existing proprietary-encoding-encoded texts through Unicode conversion.
  - hasmanean 3y ago
    How hard is it to use AI to generate fonts on demand?
    - lifthrasiir 3y ago
      That still needs some encoding that can be converted to enough information that can reconstruct a glyph.

by derefr 3y ago

WalterBright 3y ago

It's different in that nobody expects anyone to implement their own copy of the LoC. This constant expansion of the character set puts an enormous burden on a very large collection of developers. The inevitable happens - they just don't bother - making full Unicode support a pipe dream.

derefr 3y ago
I'm perplexed in what you think "full support" for a novel Unicode codepoint entails. Most of the properties of a novel codepoint that matter, are attributes that the codepoint has defined within various Unicode-Consortium-defined data files. If your custom implementation of Unicode support already parses those data files, then a new codepoint is essentially "absorbed" automatically as soon as the new version of the Unicode data files are made available to your implementation. Are you just complaining that new codepoints don't automatically receive glyphs in major fonts? To that, I would say: who cares? As I just said upthread — character encoding is about semantics, not about rendering. A character doesn't have to be renderable to be useful! The character is still doing its job — "transmitting meaning into the future" — when it's showing up as a U+FFFD replacement character! Screen readers can still read it! Search engines can still index on it! People who want to know what the character was "meant to mean" can copy and paste it into a Unicode database website! It still sorts and pattern-matches correctly! And so forth. Rendering the codepoint as a glyph — if it even makes sense to render it standalone as a glyph (for which some codepoints don't, e.g. the country-code characters that only have graphical representations in ligatured pairs) — can come at some arbitrary later point, when there's enough non-niche, non-academic, supply-side usage of the codepoint, that there arises real demand to see that codepoint rendered as something. Until then, just documenting the semantics as a codepoint and otherwise leaving it un-rendered, works fine for the needs of the anthropologists and archivists who "legibilize" existing proprietary-encoding-encoded texts through Unicode conversion.
- hasmanean 3y ago
  How hard is it to use AI to generate fonts on demand?
  - lifthrasiir 3y ago
    That still needs some encoding that can be converted to enough information that can reconstruct a glyph.