3 ms·

> combining codepoints Combining diacritics are a good thing, I'm talking about Unicodes conflation^W deliberate lies that a code point is a meaningful unit of

by a1369209993 6y ago

> combining codepoints Combining diacritics are a good thing, I'm talking about Unicodes conflation^W deliberate lies that a code point is a meaningful unit of data in it's own right rather than a artifact of using a multi-level encoding from characters like "ä" to bytes like "61 CC 88". > For example, this is why Spanish used to drop diacritical marks when up-casing: because there was no room in those typewriter fonts for accents on upper-case letters Huh, I remembered that, but though it was some badly-designed computer typesetting system. > Third, it is much simpler to close canonical composition to new compositions by having canonical decomposition than it is to keep adding pre-compositions for every sensible and new character. Yeah, this is what conviced me that base+diacritic was a better design than precomposed characters. > Emoji are [...] with color. That - that they are not actually text - is one of several problems with them, yes. > but decomposition and combining codepoints are absolutely not a mistake. Decomposition and combining diacritics are not a mistake. The mistake is treating the decomposed elements as first-class entities. > Finally, we're not likely to ever replace Unicode -- not [soon] anyways. That has been said about literally every bad status quo in the history of problems that existed long enough for people to compain about them. "The dragon is bad!"[0], and no amount of status-quo bias is going to change that. 0: https://www.nickbostrom.com/fable/dragon.html https://www.nickbostrom.com/fable/dragon.html

cryptonector 6y ago
> I'm talking about Unicodes conflation^W deliberate lies that a code point is a meaningful unit of data in it's own right rather than a artifact of using a multi-level encoding from characters What on Earth are you talking about. Where does the UC "lie" like this? Remember, the base codepoint is generally a character in its own right when it's not combined, and combining codepoints are mostly not characters in their own right. > Yeah, this is what conviced me that base+diacritic was a better design than precomposed characters. If anything, precompositions probably exist primarily to make transcoding from other previously-existing codesets (e.g., ISO-8859-*) easier. > > Emoji are [...] with color. > > That - that they are not actually text - is one of several problems with them, yes. How are Kanji text and emoji not? Both are ideographic. What is the distinction? This is not rhetorical -- I'm genuinely curious what you think is the distinction. > > but decomposition and combining codepoints are absolutely not a mistake. > > Decomposition and combining diacritics are not a mistake. The mistake is treating the decomposed elements as first-class entities. The base codepoints are generally characters in their own rights ("first-class entities") while the combining ones are generally not. In what way am I getting that wrong? > > Finally, we're not likely to ever replace Unicode -- not [soon] anyways. > > That has been said about literally every bad status quo in the history of problems that existed long enough for people to compain about them. You are making arguments for why Unicode should be replaced, though I think they are unfounded, but you'll need more than that to get it replaced. Even granting you are right, how would it be made to happen?
- cryptonector 6y ago
  Ay, I screwed up all the quoting. Sorry.
- a1369209993 6y ago
  > and combining codepoints are mostly not characters in their own right. The Unicode standard originally defined "character" as a code point (not vice versa), and until less than five years ago I could not have a discussion about the differences between characters and code points (and why the latter are bad) without some idiot showing up to claim there was no difference, since Unicode defined them to be same. However on looking for a citation, it seems that http://www.unicode.org/glossary/ http://www.unicode.org/glossary/ does not actually support that claim (although bits such as "Character [...] (3) The basic unit of encoding for the Unicode character encoding" do little to oppose it). So I can't actualy prove that the lies of said idiots were deliberate on the part the UC. Which is to say I was mistaken in assuming that equivalence to be deliberate lie rather than a miscommunication (probably). > How are Kanji text and emoji not? Kanji are monochrome and can be written with physical writing implents such as pencils. Emoji aren't and can't; a chunk of graphite can't make green and purple marks for a image (not character) of a eggplant. > I'm genuinely curious what you think is the distinction. Apologies, I should have been a bit more explicit there. > The base codepoints are generally characters in their own rights And to extent that that's the case, we should, where possible, be talking about the characters, not the implement details of representing them. > Even granting you are right, how would it be made to happen? No idea, but I'm not inclined to pretend something isn't bad just because I can't do anything about it.
  - cryptonector 6y ago
    > So I can't actualy prove that the lies of said idiots were deliberate on the part the UC. I started having to deal with Unicode back around 2001. Even back then the distinctions between code unit, codepoint, character, and glyph, were all well-delineated. Very little has changed since then, really, just more scripts, some bug fixes, and -yes- the addition of emoji, > Kanji are monochrome and can be written with physical writing implents such as pencils. Emoji aren't and can't; a chunk of graphite can't make green and purple marks for a image (not character) of a eggplant. I mean, cave paintings were color and kinda ideographic, and you do know that you can use colored crayons/pencils/pens/paint/printers to make colored text, yes? :-) I'm sure you can make a monochrome emoji font. In a few thousand years we might evolve today's emoji to look like today's Han/Kanji -- that's what happened to those, after all! Seriously, I can't see how color is enough to make emoji not text. Kids nowadays write sentences in Romance languages and English using mostly emoji sometimes. It can be like a word game. In fact, writing in Japanese can be a bit like a word game. Animation begins to really push it though, because that can't be printed. > And to extent that that's the case, we should, where possible, be talking about the characters, not the implement details of representing them. And we do. We talk about LATIN SMALL LETTER A and so on. Yes, U+0061 is easier sometimes. > No idea, but I'm not inclined to pretend something isn't bad just because I can't do anything about it. Fair. Myself I think of Unicode as... mostly faithful to the actual features of human written language scripts, with a few unfortunate things, mostly just Han unification, UCS-2, and UTF-16. Honestly, considering how long the project has been ongoing, I think it's surprisingly well-done -- surprisingly not-fucked-up. So I'm inclined to see the glass as half-full. Let's put it this way: there are a lot of mistakes to fix once we get the time machine working. Mistakes in Unicode are very, very far down the priority list.
sbierwagen 6y ago
Comparing the Unicode Consortium to... Death... seems somewhat overheated.

cryptonector 6y ago

> I'm talking about Unicodes conflation^W deliberate lies that a code point is a meaningful unit of data in it's own right rather than a artifact of using a multi-level encoding from characters What on Earth are you talking about. Where does the UC "lie" like this? Remember, the base codepoint is generally a character in its own right when it's not combined, and combining codepoints are mostly not characters in their own right. > Yeah, this is what conviced me that base+diacritic was a better design than precomposed characters. If anything, precompositions probably exist primarily to make transcoding from other previously-existing codesets (e.g., ISO-8859-*) easier. > > Emoji are [...] with color. > > That - that they are not actually text - is one of several problems with them, yes. How are Kanji text and emoji not? Both are ideographic. What is the distinction? This is not rhetorical -- I'm genuinely curious what you think is the distinction. > > but decomposition and combining codepoints are absolutely not a mistake. > > Decomposition and combining diacritics are not a mistake. The mistake is treating the decomposed elements as first-class entities. The base codepoints are generally characters in their own rights ("first-class entities") while the combining ones are generally not. In what way am I getting that wrong? > > Finally, we're not likely to ever replace Unicode -- not [soon] anyways. > > That has been said about literally every bad status quo in the history of problems that existed long enough for people to compain about them. You are making arguments for why Unicode should be replaced, though I think they are unfounded, but you'll need more than that to get it replaced. Even granting you are right, how would it be made to happen?

cryptonector 6y ago
Ay, I screwed up all the quoting. Sorry.
a1369209993 6y ago
> and combining codepoints are mostly not characters in their own right. The Unicode standard originally defined "character" as a code point (not vice versa), and until less than five years ago I could not have a discussion about the differences between characters and code points (and why the latter are bad) without some idiot showing up to claim there was no difference, since Unicode defined them to be same. However on looking for a citation, it seems that http://www.unicode.org/glossary/ http://www.unicode.org/glossary/ does not actually support that claim (although bits such as "Character [...] (3) The basic unit of encoding for the Unicode character encoding" do little to oppose it). So I can't actualy prove that the lies of said idiots were deliberate on the part the UC. Which is to say I was mistaken in assuming that equivalence to be deliberate lie rather than a miscommunication (probably). > How are Kanji text and emoji not? Kanji are monochrome and can be written with physical writing implents such as pencils. Emoji aren't and can't; a chunk of graphite can't make green and purple marks for a image (not character) of a eggplant. > I'm genuinely curious what you think is the distinction. Apologies, I should have been a bit more explicit there. > The base codepoints are generally characters in their own rights And to extent that that's the case, we should, where possible, be talking about the characters, not the implement details of representing them. > Even granting you are right, how would it be made to happen? No idea, but I'm not inclined to pretend something isn't bad just because I can't do anything about it.
- cryptonector 6y ago
  > So I can't actualy prove that the lies of said idiots were deliberate on the part the UC. I started having to deal with Unicode back around 2001. Even back then the distinctions between code unit, codepoint, character, and glyph, were all well-delineated. Very little has changed since then, really, just more scripts, some bug fixes, and -yes- the addition of emoji, > Kanji are monochrome and can be written with physical writing implents such as pencils. Emoji aren't and can't; a chunk of graphite can't make green and purple marks for a image (not character) of a eggplant. I mean, cave paintings were color and kinda ideographic, and you do know that you can use colored crayons/pencils/pens/paint/printers to make colored text, yes? :-) I'm sure you can make a monochrome emoji font. In a few thousand years we might evolve today's emoji to look like today's Han/Kanji -- that's what happened to those, after all! Seriously, I can't see how color is enough to make emoji not text. Kids nowadays write sentences in Romance languages and English using mostly emoji sometimes. It can be like a word game. In fact, writing in Japanese can be a bit like a word game. Animation begins to really push it though, because that can't be printed. > And to extent that that's the case, we should, where possible, be talking about the characters, not the implement details of representing them. And we do. We talk about LATIN SMALL LETTER A and so on. Yes, U+0061 is easier sometimes. > No idea, but I'm not inclined to pretend something isn't bad just because I can't do anything about it. Fair. Myself I think of Unicode as... mostly faithful to the actual features of human written language scripts, with a few unfortunate things, mostly just Han unification, UCS-2, and UTF-16. Honestly, considering how long the project has been ongoing, I think it's surprisingly well-done -- surprisingly not-fucked-up. So I'm inclined to see the glass as half-full. Let's put it this way: there are a lot of mistakes to fix once we get the time machine working. Mistakes in Unicode are very, very far down the priority list.

sbierwagen 6y ago

Comparing the Unicode Consortium to... Death... seems somewhat overheated.