3 ms·

These are cute, but another sign that Unicode is straying from their original mandate which was to represent characters already in use in other systems (and to

These are cute, but another sign that Unicode is straying from their original mandate which was to represent characters already in use in other systems (and to help bootstrap under-digitized languages to become digital). Stop adding random emoji. Don't add fictional languages, no matter how cool. Don't do.. this. By continuing to extend Unicode like this, they risk diluting their core purpose and creating unnecessary complexity. Unicode should remain focused on its original goal and not cater to niche or novel additions. EDIT: I'm certain there's a proposal submitter out there that contorted the argument beyond the reasonable point that these existed in an old Usborne programming book text as inline images and need to be represented. I'm going to try to hunt it down. EDIT 2: The original proposal? https://www.unicode.org/L2/L2019/19025-terminals-prop.pdf https://www.unicode.org/L2/L2019/19025-terminals-prop.pdf Some of the proposed characters might have come from an older submission as well https://unicode.org/L2/L2021/21234-terminals-smalltalk.pdf https://unicode.org/L2/L2021/21234-terminals-smalltalk.pdf

lifthrasiir 2y ago
The thing is that, additional emojis and mere symbols like these are technically much easier to add than under-digitized languages which require much more research. (The initial introduction of emojis wasn't easy, let me clarify.) These characters are just another entry to the code chart and Unicode character database without any additional mechanism to fit. So it should be okay as long as there are not too many characters added in this way. Reply to EDIT: People who made the proposals are already well known in Unicode and they know what they are proposing. Michael Everson for example is responsible for encoding many other writing systems other than such symbols.
astrange 2y ago
The main reason to add new emoji is that it gets people to update their Unicode standards / phone OSes / etc. Otherwise they won't care enough to get the other less interesting stuff.
- throw-the-towel 2y ago
  Who's "people"? Consumers aren't gonna buy a new phone just to get U+22E5B VOMITING MALE PLATYPUS WITH DESCENDER, and system implementers are free to add the new emoji and ignore everything else in the standard.
  - rbits 2y ago
    They're not going to buy a new phone for it, but it might get them to update their OS
    - whycome 2y ago
      Darn. my phone can't run that os. I guess I'll upgrade
  - ksec 2y ago
    But they are going to update to new iOS just to see that annoying emoji people keep using.
  - troad 2y ago
    > U+22E5B VOMITING MALE PLAT[Y]PUS WITH DESCENDER Speak for yourself, I need this now! In general, I think new emoji is going to be more persuasive to the average person to update than 'bugfixes to VCard deserialisation in the Contacts app' or the other usual release notes.
  - astrange 2y ago
    Oh yes they are going to buy a new phone for that, at least if their friends send it to them and they can't see it. But mostly it just encourages them to install new free OS updates sooner.
  - lifthrasiir 2y ago
    Reminder: U+22E5B is an actual Unicode character, namely 𢹛 (which comes from Hanyu Dacidian and I have no further public information available, but anyway). New emoji usually goes into the specific portion of the Plane 1 so U+1Fxxx would have been more believable ;-)
mysterymath 2y ago
I'm definitely supportive of the original "Symbols for Legacy Computers" effort. We've been able to make great use of it in the llvm-mos toolchain for legacy systems; we can directly encode special characters for these computers directly in the UTF-8 source text of modern C++, then use C++ user-defined string literals to convert them to their original byte codes. It's an aid to hobbyist use and preservation of these systems, and thankfully, there's a relatively small fixed number of them to support, especially relative to the space available in Unicode. It should unlock quite a few of these kinds of projects for exploring and experiencing the history of computing. https://llvm-mos.org/wiki/Character_set https://llvm-mos.org/wiki/Character_set
- zzo38computer 2y ago
  It is, as the linked document says, a bug in clang, so it is clang that should be fixed.
  - mysterymath 2y ago
    See https://discourse.llvm.org/t/rfc-enabling-fexec-charset-support-to-llvm-and-clang-reposting/71512/64 https://discourse.llvm.org/t/rfc-enabling-fexec-charset-supp... There's a lot of discussion about this; practically changing the execution character set isn't trivial, and it's not necessarily even a desirable feature. Even with full -fexec-charset support, it still makes sense to provide compile-time translation from Unicode to target strings. For example, Commodore PETSCII makes vastly more sense as an execution character set than a source character set.
    - zzo38computer 2y ago
      > practically changing the execution character set isn't trivial, and it's not necessarily even a desirable feature I think it should be unnecessary to convert the character set; the execution shouldn't care about the character set except for ASCII and for whatever you program yourself in what the specific program you are writing is doing. It should not need to be a subset of Unicode, either; you should be able to use any character set that is a superset of ASCII (and where bytes in the ASCII range always mean ASCII characters and bytes not in the ASCII range always mean non-ASCII characters) (UTF-8 has this property and therefore may be used, but it is not the only character encoding with this property). The C preprocessor is limited in its capabilities, although it would be helpful to add extra steps both before and after the preprocessor runs, which can transform character encodings, but also can be useful for other purposes too. (With GCC, I think this could be done by -no-integrated-cpp and -wrapper; I don't know about doing with Clang.) (GCC will convert input to UTF-8 during preprocessing, but at least with the version of GCC that I have does not actually care if it is valid UTF-8 (at least for C; maybe not for C++ but I have not tried it), which is fortunate, since this means that you can implement your own character code handling.) In the case of C++, as described there, you can use user-defined literals. They shouldn't require user-defined literals to be UTF-8 (nor Unicode), although if you can do whatever calculation you want on them at compile-time, then you can treat them as UTF-8 if you want to, but shouldn't be required to do so. (Personally, I do not use C++, though; so I do not actually know all of the details about how it is working, so I may have made a mistake.) (There are several reasons you might deliberately not want UTF-8. One of them is security issues with the complicated text rendering involved with Unicode. Another might be the way that character widths are working. And there are many other possibilities, too. You might also prefer to put all non-ASCII text in a separate file; the #embed command can be used if you want to embed it into the program anyways, I suppose.) > Even with full -fexec-charset support, it still makes sense to provide compile-time translation from Unicode to target strings. Maybe, but I should think that this compile-time translation should be done separately as described above, and to be programmable to not be limited to only Unicode. It should not be required; I think it would be sensible that by default it should just pass through directly without conversion regardless of what the character set is. > For example, Commodore PETSCII makes vastly more sense as an execution character set than a source character set. I agree, but that is because Commodore PETSCII is not a superset of ASCII which is encoded as a superset of ASCII. The reason for this has nothing to do with Unicode.
floxy 2y ago
The real question is why don't we just have a <begin SVG> and <end SVG> Unicode tag, and let people go to town with whatever they want in the glyph space. It seems like that's what people really want? What are the down-sides? Sending obscene characters becomes easier. You lose the "meaning", since maybe Apple's version of the unicorn horse-shoe (unicorn-shoe?) has a different SVG encoding than Google's. Security-type denial-of-service-issues, with people rendering Mandelbrot fractal glyphs?
- mysterymath 2y ago
  This wouldn't have worked for the llvm-mos use case I mentioned; the actual identity is the useful part. I'd expect each of these symbols was associated with a numeric code on a legacy computing platform, and having a Unicode assignment makes it possible to machine-convert text from these systems to and from Unicode.
- lifthrasiir 2y ago
  Well, they tried multiple times! Some tried a direct image, some tried a compact hash of images that should be known in advance ("Coded Hashes of Arbitrary Images"), another tried a direct mapping to Wikidata entities ("QID emojis"). They ultimately failed because... yeah, they would have been much more difficult to implement than what we have right now.
gwervc 2y ago
I agree. Unicode needs to be splitted in two: one serious part focusing on the original mission and real world documented scripts, the other doing whatever it wants with stupid emojis.
pxc 2y ago
What kind of complexity is actually entailed by adding additional codepoints? Does it require changes to the structure and composition of the encoding, or is it just more stuff that's exactly the same in form as the other stuff?
Sniffnoy 2y ago
These were added precisely because they were in use in older systems, so I don't see what you're complaining about here. The emoji expansion is annoying, but this isn't an instance of that; this is specifically adding characters from legacy computing character sets for compatibility purposes.
nine_k 2y ago
WRT representing characters already in use, are the powerline characters, widely used in the wild, standardized yet? They are rather few, and abstract.

lifthrasiir 2y ago

The thing is that, additional emojis and mere symbols like these are technically much easier to add than under-digitized languages which require much more research. (The initial introduction of emojis wasn't easy, let me clarify.) These characters are just another entry to the code chart and Unicode character database without any additional mechanism to fit. So it should be okay as long as there are not too many characters added in this way. Reply to EDIT: People who made the proposals are already well known in Unicode and they know what they are proposing. Michael Everson for example is responsible for encoding many other writing systems other than such symbols.

astrange 2y ago

The main reason to add new emoji is that it gets people to update their Unicode standards / phone OSes / etc. Otherwise they won't care enough to get the other less interesting stuff.

throw-the-towel 2y ago
Who's "people"? Consumers aren't gonna buy a new phone just to get U+22E5B VOMITING MALE PLATYPUS WITH DESCENDER, and system implementers are free to add the new emoji and ignore everything else in the standard.
- rbits 2y ago
  They're not going to buy a new phone for it, but it might get them to update their OS
  - whycome 2y ago
    Darn. my phone can't run that os. I guess I'll upgrade
- ksec 2y ago
  But they are going to update to new iOS just to see that annoying emoji people keep using.
- troad 2y ago
  > U+22E5B VOMITING MALE PLAT[Y]PUS WITH DESCENDER Speak for yourself, I need this now! In general, I think new emoji is going to be more persuasive to the average person to update than 'bugfixes to VCard deserialisation in the Contacts app' or the other usual release notes.
- astrange 2y ago
  Oh yes they are going to buy a new phone for that, at least if their friends send it to them and they can't see it. But mostly it just encourages them to install new free OS updates sooner.
- lifthrasiir 2y ago
  Reminder: U+22E5B is an actual Unicode character, namely 𢹛 (which comes from Hanyu Dacidian and I have no further public information available, but anyway). New emoji usually goes into the specific portion of the Plane 1 so U+1Fxxx would have been more believable ;-)

mysterymath 2y ago

I'm definitely supportive of the original "Symbols for Legacy Computers" effort. We've been able to make great use of it in the llvm-mos toolchain for legacy systems; we can directly encode special characters for these computers directly in the UTF-8 source text of modern C++, then use C++ user-defined string literals to convert them to their original byte codes. It's an aid to hobbyist use and preservation of these systems, and thankfully, there's a relatively small fixed number of them to support, especially relative to the space available in Unicode. It should unlock quite a few of these kinds of projects for exploring and experiencing the history of computing. https://llvm-mos.org/wiki/Character_set https://llvm-mos.org/wiki/Character_set

zzo38computer 2y ago
It is, as the linked document says, a bug in clang, so it is clang that should be fixed.
- mysterymath 2y ago
  See https://discourse.llvm.org/t/rfc-enabling-fexec-charset-support-to-llvm-and-clang-reposting/71512/64 https://discourse.llvm.org/t/rfc-enabling-fexec-charset-supp... There's a lot of discussion about this; practically changing the execution character set isn't trivial, and it's not necessarily even a desirable feature. Even with full -fexec-charset support, it still makes sense to provide compile-time translation from Unicode to target strings. For example, Commodore PETSCII makes vastly more sense as an execution character set than a source character set.
  - zzo38computer 2y ago
    > practically changing the execution character set isn't trivial, and it's not necessarily even a desirable feature I think it should be unnecessary to convert the character set; the execution shouldn't care about the character set except for ASCII and for whatever you program yourself in what the specific program you are writing is doing. It should not need to be a subset of Unicode, either; you should be able to use any character set that is a superset of ASCII (and where bytes in the ASCII range always mean ASCII characters and bytes not in the ASCII range always mean non-ASCII characters) (UTF-8 has this property and therefore may be used, but it is not the only character encoding with this property). The C preprocessor is limited in its capabilities, although it would be helpful to add extra steps both before and after the preprocessor runs, which can transform character encodings, but also can be useful for other purposes too. (With GCC, I think this could be done by -no-integrated-cpp and -wrapper; I don't know about doing with Clang.) (GCC will convert input to UTF-8 during preprocessing, but at least with the version of GCC that I have does not actually care if it is valid UTF-8 (at least for C; maybe not for C++ but I have not tried it), which is fortunate, since this means that you can implement your own character code handling.) In the case of C++, as described there, you can use user-defined literals. They shouldn't require user-defined literals to be UTF-8 (nor Unicode), although if you can do whatever calculation you want on them at compile-time, then you can treat them as UTF-8 if you want to, but shouldn't be required to do so. (Personally, I do not use C++, though; so I do not actually know all of the details about how it is working, so I may have made a mistake.) (There are several reasons you might deliberately not want UTF-8. One of them is security issues with the complicated text rendering involved with Unicode. Another might be the way that character widths are working. And there are many other possibilities, too. You might also prefer to put all non-ASCII text in a separate file; the #embed command can be used if you want to embed it into the program anyways, I suppose.) > Even with full -fexec-charset support, it still makes sense to provide compile-time translation from Unicode to target strings. Maybe, but I should think that this compile-time translation should be done separately as described above, and to be programmable to not be limited to only Unicode. It should not be required; I think it would be sensible that by default it should just pass through directly without conversion regardless of what the character set is. > For example, Commodore PETSCII makes vastly more sense as an execution character set than a source character set. I agree, but that is because Commodore PETSCII is not a superset of ASCII which is encoded as a superset of ASCII. The reason for this has nothing to do with Unicode.

floxy 2y ago

The real question is why don't we just have a <begin SVG> and <end SVG> Unicode tag, and let people go to town with whatever they want in the glyph space. It seems like that's what people really want? What are the down-sides? Sending obscene characters becomes easier. You lose the "meaning", since maybe Apple's version of the unicorn horse-shoe (unicorn-shoe?) has a different SVG encoding than Google's. Security-type denial-of-service-issues, with people rendering Mandelbrot fractal glyphs?

mysterymath 2y ago
This wouldn't have worked for the llvm-mos use case I mentioned; the actual identity is the useful part. I'd expect each of these symbols was associated with a numeric code on a legacy computing platform, and having a Unicode assignment makes it possible to machine-convert text from these systems to and from Unicode.
lifthrasiir 2y ago
Well, they tried multiple times! Some tried a direct image, some tried a compact hash of images that should be known in advance ("Coded Hashes of Arbitrary Images"), another tried a direct mapping to Wikidata entities ("QID emojis"). They ultimately failed because... yeah, they would have been much more difficult to implement than what we have right now.

gwervc 2y ago

I agree. Unicode needs to be splitted in two: one serious part focusing on the original mission and real world documented scripts, the other doing whatever it wants with stupid emojis.

pxc 2y ago

What kind of complexity is actually entailed by adding additional codepoints? Does it require changes to the structure and composition of the encoding, or is it just more stuff that's exactly the same in form as the other stuff?

Sniffnoy 2y ago

These were added precisely because they were in use in older systems, so I don't see what you're complaining about here. The emoji expansion is annoying, but this isn't an instance of that; this is specifically adding characters from legacy computing character sets for compatibility purposes.

nine_k 2y ago

WRT representing characters already in use, are the powerline characters, widely used in the wild, standardized yet? They are rather few, and abstract.