10 ms·

The fixating effect of digitalization is scary. New characters were introduced by accident, and although they're useless, they're probably not going anywhere.

The fixating effect of digitalization is scary. New characters were introduced by accident, and although they're useless, they're probably not going anywhere. But what about the inverse? Will character sets still be able to change and grow new characters now that they're standardized like this?

innocenat 8y ago
The answer is easy -- No, it won't be able to change and grow. Have been to more than one character museum where they show evolution of character. For most of the history you can see some change between 10-20 years, but it stays completely same for last 20 years or so.
- emodendroket 8y ago
  Twenty? I don't know what you mean. If you pick up a Japanese book published in 1960 the characters are the same ones they use now. Nobody was using bone script or anything.
  - innocenat 8y ago
    I did not talk specifically about Japanese, and I don't think the OP did either. And I should clarify for that -- character mostly stop changing once printing press is introduced, and stop almost completely once digitization is completed. Of course common language get digitize long before, but all common language seems to have digitization completed 10-20 years ago.
    - rspeer 8y ago
      There are languages in India with tens of millions of speakers whose digital representations are still poorly implemented, partially because the speakers of those languages rely on English instead of their own language when using computers.
    - robjan 8y ago
      You are thinking about alphabet-based languages where the creation of a new word does not involve the creation of a new character. We are frequently creating new words which require "workarounds" (sometimes just using a romanisation) until it's properly encoded.
      - emodendroket 8y ago
        Who are we here? The Japanese have a syllabary. Even the Chinese are, as far as I know, almost never creating new characters (when something new is called for, as far as I know, a soundalike old character is the most common solution).
emodendroket 8y ago
The character set has been pretty stable for a long time with the exception of post-war character simplification (which could be done again without changing the codes)
- betterunix2 8y ago
  Actually the situation is more complicated than that. Japanese Kanji include some characters that were created by the Japanese, following the basic principles of Chinese characters, over the past 1000~ years. Everywhere that Chinese characters have been used there have been local variations. There have also been various hand-written simplifications, which in some cases became "new" characters that over time were repurposed. It is also not true that simplified characters could be do again without changing code points; it is true for Japanese but for Chinese simplification there were several characters that were "merged" i.e. two formerly different characters are now written the same way. I would actually argue that the idea of "code points" makes no sense for Chinese characters, and that instead an approach based on the components of each character should have been used.
  - emodendroket 8y ago
    Yes, everyone likes to talk about 働 and 峠. These characters are rare and the set is not expanding. Anyway, sure the simplification could happen. You just take the code point for 國 and start rendering 国. The principle that the font affects the rendering is already set with Han unification (probably not a good decision but we are stuck with it now). At worst you end up with redundancies (even in the Chinese case automated transition from simplified to traditional characters is mostly right; if you're going the other way it is always right).
    - gurkendoktor 8y ago
      > even in the Chinese case automated transition from simplified to traditional characters is mostly right; if you're going the other way it is always right Even Traditional-to-Simplified has its pitfalls because some characters have a 1:2 mapping: https://en.wikipedia.org/wiki/Ambiguities_in_Chinese_character_simplification#2_to_1 https://en.wikipedia.org/wiki/Ambiguities_in_Chinese_charact...
kryptiskt 8y ago
Print already fixed it pretty hard. If anything, it's easier to sneak in a custom font and use the private use area than what you had to do to get a new character in lead type.
- deleted 8y ago
  [deleted]
Freak_NL 8y ago
Character sets in general will certainly continue to grow — look no further than the ever-expanding emoji set. Unicode is designed to be able to incorporate new characters. Whether the CJKV¹ set will grow is another matter entirely. I expect that there will be a couple more extensions for characters found in historical sources; the goal of the Unicode standard is — roughly speaking — to be able to digitize any and all existing texts, after all. But for actual new characters to be added they would first have to become popular enough to warrant inclusion. Neither Chinese nor Japanese is a language that produces new Chinese characters like they did historically. The barrier to get a character excepted beyond a fringe group is quite high — in any language — and there really is no strong need to create new characters. 1: Chinese, Japanese, Korean, Vietnamese (the latter two historically used Chinese characters as well)
- nabla9 8y ago
  Emojis are mistake and added too soon. We end up with standard that is full of short lived symbols from 2000-2025 that were once in a fashion and new generations of people will abandon them for something else when the culture changes. Adding new stuff (excluding old languages) to Unicode should have a delay, at least 20-25 years from proposal to the standard (roughly a generation). In the meantime it's possible to insert emoji into text as :), smiley, -smiley-, smiley.jpg, ::smiley:: or whatever you want and the system you use is free to change it into a picture.
  - fermuch 8y ago
    Weren't emojis added because Japanese phones had them, since a long, long time (before smartphones), and the main pursuit of unicode is to be able to represent all written characters? The first emojis would be close to the 20 year lapse now, and they are the most used.
    - jsjohnst 8y ago
      1999, so yeah, the first set of almost 200 emoji are nearly 20 years old. And yes, they were added first in Japan and didn’t become a western fad until after the iphone added them (not claiming the iphone was the catalyst, but the timing was about the same).
panic 8y ago
It's much easier to reproduce images / vector graphics on a computer than it is on a printing press. Reaction GIFs, custom pseudo-emoji, "stickers", and so on could be considered variations on characters. Also, handwriting is possible with a stylus, though tablets aren't popular enough for this to have become a big thing yet (it may someday). I wouldn't worry too much!
masswerk 8y ago
German just saw the addition of the capital "ß" (U+1E9E) last year.
- ygra 8y ago
  A character that existed for a while, just not in digital typefaces.
  - majewsky 8y ago
    The upper-case ß is not in common use, though. If I'm not mistaken, it's only use is correctly reproducing names in official documents that write a name in all-caps. In common usage, when mapping to upper case, the lower-case ß gets mapped to double S (e.g. "Straße" -> "STRASSE"), which I guess is another fun edge case for Unicode implementations to cover.
    - contravariant 8y ago
      If you want that to work nicely that would be a heck of a headache. Not only are "Straße" and "STRASSE" different lengths, but converting "SS" to lower case would now depend on context (which is probably the case even when you're solely considering the German language, consider the name "Bessel" for instance, or compound words like Linnaeusstraße).
      - kuschku 8y ago
        True, but that issue has been existing for the past decades already. toLower(toUpper(a)) != a len(toLower(toUpper(a))) != a If your code assumes any of this, it already was broken
        komali2 8y ago
        Oh wait wtf, why
        8y ago
lmm 8y ago
> New characters were introduced by accident, and although they're useless, they're probably not going anywhere. I'd say that since they exist, people will find uses for them. The article links to a page of ghost stories that uses the ghost characters as names for them; I've also seen one TV show's title given as Magical�Pokaan (sic).
mcguire 8y ago
It's not a new question; I've seen arguments about Año Nuevo State Park and whether the tilde is necessary in English after some major publication discovered that they didn't have ñ in their standard font and their proofreaders didn't know Spanish.
- duskwuff 8y ago
  The ~ in "Año" is never optional. Removing it leads to some incredibly inappropriate readings. Año = year. Ano = anus.
  - Dylan16807 8y ago
    You're answering about Spanish. Ano does not mean that in English.
    - JasonFruit 8y ago
      But any context where you misspell "año" will suggest Spanish, so the misspelling will evoke anuses.
      - Dylan16807 8y ago
        If it actually did evoke that in the average person, the mistake wouldn't happen so often. And what if it evokes the very similar Portuguese, where "ano" does in fact mean year?
        mitchty 8y ago
        Seemingly it would then be ano novo instead of ano nuevo.
    - cdelsolar 8y ago
      What does Ano mean in English?
      - 8y ago
a3n 8y ago
Like useless genes.
cschmidt 8y ago
There was this story a few days ago about adding a character, for the new era in the Japanese calendar due to the Emperor's abdication and crowning of the new Emperor. They need to add a character for the name of the era, but it hasn't been decided yet. So the Unicode standards is going to need to rush it in. https://news.ycombinator.com/item?id=17608655 https://news.ycombinator.com/item?id=17608655
- deleted 8y ago
  [deleted]
adamrezich 8y ago
Controversial opinion but I believe the skin-tone modifiers fall into this category as well.
peterburkimsher 8y ago
I'm in the process of proposing 7 new Taiwanese characters, and 14 more Hakka characters to the Unicode IRG. They need scanned evidence of times when the characters are used. It then takes about 5 years to actually be introduced into the standard.

innocenat 8y ago

The answer is easy -- No, it won't be able to change and grow. Have been to more than one character museum where they show evolution of character. For most of the history you can see some change between 10-20 years, but it stays completely same for last 20 years or so.

emodendroket 8y ago
Twenty? I don't know what you mean. If you pick up a Japanese book published in 1960 the characters are the same ones they use now. Nobody was using bone script or anything.
- innocenat 8y ago
  I did not talk specifically about Japanese, and I don't think the OP did either. And I should clarify for that -- character mostly stop changing once printing press is introduced, and stop almost completely once digitization is completed. Of course common language get digitize long before, but all common language seems to have digitization completed 10-20 years ago.
  - rspeer 8y ago
    There are languages in India with tens of millions of speakers whose digital representations are still poorly implemented, partially because the speakers of those languages rely on English instead of their own language when using computers.
  - robjan 8y ago
    You are thinking about alphabet-based languages where the creation of a new word does not involve the creation of a new character. We are frequently creating new words which require "workarounds" (sometimes just using a romanisation) until it's properly encoded.
    - emodendroket 8y ago
      Who are we here? The Japanese have a syllabary. Even the Chinese are, as far as I know, almost never creating new characters (when something new is called for, as far as I know, a soundalike old character is the most common solution).

emodendroket 8y ago

The character set has been pretty stable for a long time with the exception of post-war character simplification (which could be done again without changing the codes)

betterunix2 8y ago
Actually the situation is more complicated than that. Japanese Kanji include some characters that were created by the Japanese, following the basic principles of Chinese characters, over the past 1000~ years. Everywhere that Chinese characters have been used there have been local variations. There have also been various hand-written simplifications, which in some cases became "new" characters that over time were repurposed. It is also not true that simplified characters could be do again without changing code points; it is true for Japanese but for Chinese simplification there were several characters that were "merged" i.e. two formerly different characters are now written the same way. I would actually argue that the idea of "code points" makes no sense for Chinese characters, and that instead an approach based on the components of each character should have been used.
- emodendroket 8y ago
  Yes, everyone likes to talk about 働 and 峠. These characters are rare and the set is not expanding. Anyway, sure the simplification could happen. You just take the code point for 國 and start rendering 国. The principle that the font affects the rendering is already set with Han unification (probably not a good decision but we are stuck with it now). At worst you end up with redundancies (even in the Chinese case automated transition from simplified to traditional characters is mostly right; if you're going the other way it is always right).
  - gurkendoktor 8y ago
    > even in the Chinese case automated transition from simplified to traditional characters is mostly right; if you're going the other way it is always right Even Traditional-to-Simplified has its pitfalls because some characters have a 1:2 mapping: https://en.wikipedia.org/wiki/Ambiguities_in_Chinese_character_simplification#2_to_1 https://en.wikipedia.org/wiki/Ambiguities_in_Chinese_charact...

kryptiskt 8y ago

Print already fixed it pretty hard. If anything, it's easier to sneak in a custom font and use the private use area than what you had to do to get a new character in lead type.

deleted 8y ago
[deleted]

Freak_NL 8y ago

Character sets in general will certainly continue to grow — look no further than the ever-expanding emoji set. Unicode is designed to be able to incorporate new characters. Whether the CJKV¹ set will grow is another matter entirely. I expect that there will be a couple more extensions for characters found in historical sources; the goal of the Unicode standard is — roughly speaking — to be able to digitize any and all existing texts, after all. But for actual new characters to be added they would first have to become popular enough to warrant inclusion. Neither Chinese nor Japanese is a language that produces new Chinese characters like they did historically. The barrier to get a character excepted beyond a fringe group is quite high — in any language — and there really is no strong need to create new characters. 1: Chinese, Japanese, Korean, Vietnamese (the latter two historically used Chinese characters as well)

nabla9 8y ago
Emojis are mistake and added too soon. We end up with standard that is full of short lived symbols from 2000-2025 that were once in a fashion and new generations of people will abandon them for something else when the culture changes. Adding new stuff (excluding old languages) to Unicode should have a delay, at least 20-25 years from proposal to the standard (roughly a generation). In the meantime it's possible to insert emoji into text as :), smiley, -smiley-, smiley.jpg, ::smiley:: or whatever you want and the system you use is free to change it into a picture.
- fermuch 8y ago
  Weren't emojis added because Japanese phones had them, since a long, long time (before smartphones), and the main pursuit of unicode is to be able to represent all written characters? The first emojis would be close to the 20 year lapse now, and they are the most used.
  - jsjohnst 8y ago
    1999, so yeah, the first set of almost 200 emoji are nearly 20 years old. And yes, they were added first in Japan and didn’t become a western fad until after the iphone added them (not claiming the iphone was the catalyst, but the timing was about the same).

panic 8y ago

It's much easier to reproduce images / vector graphics on a computer than it is on a printing press. Reaction GIFs, custom pseudo-emoji, "stickers", and so on could be considered variations on characters. Also, handwriting is possible with a stylus, though tablets aren't popular enough for this to have become a big thing yet (it may someday). I wouldn't worry too much!

masswerk 8y ago

German just saw the addition of the capital "ß" (U+1E9E) last year.

ygra 8y ago
A character that existed for a while, just not in digital typefaces.
- majewsky 8y ago
  The upper-case ß is not in common use, though. If I'm not mistaken, it's only use is correctly reproducing names in official documents that write a name in all-caps. In common usage, when mapping to upper case, the lower-case ß gets mapped to double S (e.g. "Straße" -> "STRASSE"), which I guess is another fun edge case for Unicode implementations to cover.
  - contravariant 8y ago
    If you want that to work nicely that would be a heck of a headache. Not only are "Straße" and "STRASSE" different lengths, but converting "SS" to lower case would now depend on context (which is probably the case even when you're solely considering the German language, consider the name "Bessel" for instance, or compound words like Linnaeusstraße).
    - kuschku 8y ago
      True, but that issue has been existing for the past decades already. toLower(toUpper(a)) != a len(toLower(toUpper(a))) != a If your code assumes any of this, it already was broken
      - komali2 8y ago
        Oh wait wtf, why
        8y ago

lmm 8y ago

> New characters were introduced by accident, and although they're useless, they're probably not going anywhere. I'd say that since they exist, people will find uses for them. The article links to a page of ghost stories that uses the ghost characters as names for them; I've also seen one TV show's title given as Magical�Pokaan (sic).

mcguire 8y ago

It's not a new question; I've seen arguments about Año Nuevo State Park and whether the tilde is necessary in English after some major publication discovered that they didn't have ñ in their standard font and their proofreaders didn't know Spanish.

duskwuff 8y ago
The ~ in "Año" is never optional. Removing it leads to some incredibly inappropriate readings. Año = year. Ano = anus.
- Dylan16807 8y ago
  You're answering about Spanish. Ano does not mean that in English.
  - JasonFruit 8y ago
    But any context where you misspell "año" will suggest Spanish, so the misspelling will evoke anuses.
    - Dylan16807 8y ago
      If it actually did evoke that in the average person, the mistake wouldn't happen so often. And what if it evokes the very similar Portuguese, where "ano" does in fact mean year?
      - mitchty 8y ago
        Seemingly it would then be ano novo instead of ano nuevo.
  - cdelsolar 8y ago
    What does Ano mean in English?
    - 8y ago

a3n 8y ago

Like useless genes.

cschmidt 8y ago

There was this story a few days ago about adding a character, for the new era in the Japanese calendar due to the Emperor's abdication and crowning of the new Emperor. They need to add a character for the name of the era, but it hasn't been decided yet. So the Unicode standards is going to need to rush it in. https://news.ycombinator.com/item?id=17608655 https://news.ycombinator.com/item?id=17608655

deleted 8y ago
[deleted]

adamrezich 8y ago

Controversial opinion but I believe the skin-tone modifiers fall into this category as well.

peterburkimsher 8y ago

I'm in the process of proposing 7 new Taiwanese characters, and 14 more Hakka characters to the Unicode IRG. They need scanned evidence of times when the characters are used. It then takes about 5 years to actually be introduced into the standard.