6 ms·

Unicode 18.0.0 Beta

60 points by birdculture 5mo ago

orangepanda 5mo ago
What everyone actually cares about — new emojis! * Cracking face * Left/Right thumb sign * Monarch butterfly * Pickle * Lighthouse * Meteor * Eraser * Net with handle
- chirsz 5mo ago
  Still no seahorse
  - JohnKemeny 5mo ago
    If the seahorse emoji is introduced, we will have to train new foundation models. The costs connected to the introduction of the seahorse emoji will be in the billions.
  - zarzavat 5mo ago
    You're absolutely right—the seahorse emoji was added in Unicode version 19.0.0 after OpenAI purchased the Unicode Consortium and converted it to a for-profit corporation.
    - simondotau 5mo ago
      The seahorse being, of course, among the first commercial Unicode characters that require a subscription to use.
- Rp8yXmdmr 5mo ago
  I would be more interested if they are ever going to cancel HAN unification. Looking at their "Factors for Exclusion" list it could be summarized by "we made some mistakes in past but are sticking to it" :D
  - lifthrasiir 5mo ago
    Han Unification was effectively "fixed" by Ideographic Variation Sequences, so no.
    - deleted 5mo ago
      [deleted]
    - account42 5mo ago
      You mean theoretically. Effectively, nothing is fixed yet.
      - lifthrasiir 5mo ago
        IVD works, theoretically and practically (recent versions of OpenType have an explicit support for them). It's not their fault that Japanese vendors have been not very quick to adopt them.
        account42 5mo ago
        If a Japanese and Taiwanese person type things with their keyboards and end up with the same bytes for different logical characters then no things do not work practically for any practical definition of "practically".
        lifthrasiir 5mo ago
        Your argument is absurd because people don't see code---they see glyphs, and using the same code for slightly different glyphs is a non-issue when they are not interchanged. (And when they are interchanged, both would see glyphs "correct" to them anyway.) Japaneses are sensitive to Han unification only because they recognize more glyph variations (Z-variants) than what Unicode originally could, and IVS is exactly a tool for ensuring exact glyphs assuming cooperative vendors. Not to mention that Han unification was already quite weakened by source separation principles in the first place.
        5mo ago
- wongarsu 5mo ago
  Other great additions: - Left and Right parenthesis with middle ring [1] - A wiggly exclamation mark expressing mirth or laughter [1] (edit: and something I completely missed: the inverted version can express sarcasm) - Cuneiform numerals, including lots of arranged dots that might be useful in other contexts [2] - New variations of "measured angle" and "sector" [3] - A transparent cube and a white cube [4] Also a couple new combining marks And for anyone who wants to see what the reference images for the new emojis look like: Lighthouse: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F680.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F680.p... Other new Emojis: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1FA70.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1FA70.p... 1: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-2E00.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-2E00.pd... 2: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-12550.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-12550.p... 3: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1CEC0.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1CEC0.p... 4: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F780.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F780.p...
- BadBadJellyBean 5mo ago
  I need a table emoji because then I could combine it with a horse emoji. This would be "Pferd Tisch" (Horse Table) in German which sounds similar to "Fertig" which translates to "done". Yes I want it only for that dumb joke.
- Symbiote 5mo ago
  Does anyone know why a monarch butterfly was added, when there is already a butterfly emoji?
- pentamassiv 5mo ago
  Sadly it looks like it will be a dead monarch butterfly https://www.emilydamstra.com/please-enough-dead-butterflies/ https://www.emilydamstra.com/please-enough-dead-butterflies/
computer23 5mo ago
The internet needs a "tin foil hat" emoji, but two proposals have been rejected :( Emoji proposals and status: https://unicode.org/emoji/emoji-proposals-status.html https://unicode.org/emoji/emoji-proposals-status.html
- brikym 5mo ago
  Seems like a conspiracy. Also it's so silly that pistol turned into water pistol.
- turblety 5mo ago
  Did they give a reason why it was declined? Was it some bureaucratic "form not filled in correct" thing, or are they actually against the concept of it?
  - lifthrasiir 5mo ago
    Vendors-have-no-capacity-to-handle-more-than-handful-number-of-emojis-per-each-release thing. To elaborate: it should be plain obvious that not every Emoji proposal can be accepted even though all of them are correctly filed, as there would be too many Emojis there then. So there has to be some threshold, and that threshold is mostly stipulated by vendors' willingness to process new Emoji characters for designing fonts and updating softwares in time.
  - zyx321 5mo ago
    That list only includes suggestions that were seriously considered and voted on. Since it's a vote, there is no single official 'reason' for rejection. If I had to guess: it would be confusing to anyone who didn't grow up with American TV shows.
    - pwdisswordfishq 5mo ago
      Eh, it's not like there are hundreds of emojises pretty much exclusively tied to Japanese culture.
poulpy123 5mo ago
looking at the changes it makes me wonder: - is there an usable font the cover all unicode ? - if not is there really a point to include everything possible in unicode ? - how many space is remaining for new alphabet and smileys ? - how do they handle changes in scripts, for example if new proto-cuneiform or seal script symbols are discovered ?
- pveierland 5mo ago
  The Noto fonts have great coverage: https://notofonts.github.io/overview/ https://notofonts.github.io/overview/
  - infinita740 5mo ago
    Pretty cool vizualisation. There is also GNU unifont [1] "The original intent of Unifont was to offer a simple font format with wide Unicode coverage to render something meaningful for each Unicode code point" [1] https://unifoundry.com/unifont/index.html https://unifoundry.com/unifont/index.html
- wongarsu 5mo ago
  > if not is there really a point to include everything possible in unicode ? Needing to load three fonts to show a single document that mixes vastly different character sets is still infinitely better than not being able to have those different characters in the same .txt or .md file at all > how many space is remaining for new alphabet and smileys ? Unicode can encode about 1100k code points, and about 800k of those are currently unassigned and available for future scripts or characters
  - xigoi 5mo ago
    Also, the 1.1M limit is because of UTF-16. If UTF-16 was deprecated in favor of UTF-8, the limit could be much higher.
    - hulitu 5mo ago
izissise 5mo ago
No toki pona?
- pwdisswordfishq 5mo ago
  Does it need any additional code points?
  - florianist 5mo ago
    The Toki Pona script (aka sitelen pona) needs some codepoints for its ideograms. While Toki Pona is not in Unicode, tokiponists have mostly agreed to use the U+F19xx range in the Private Use Area-A. Most fonts rendering sitelen pona uses that. But using PUA is problematic (no character properties, a lot of restrictions on the web, and constant clashes with other fonts [such as the "nerdfonts" for example]).
- lifthrasiir 5mo ago
  For now, follow UCSUR: https://sona.pona.la/wiki/Under-ConScript_Unicode_Registry https://sona.pona.la/wiki/Under-ConScript_Unicode_Registry
brikym 5mo ago
Hey ChatGPT show me a seahorse emoji.
sourcegrift 5mo ago
Personally the whole emoji thing is an unmitigated disaster. I'm okay with smileys and gestures but everything else is pointless
- vintermann 5mo ago
  I'm okay with smileys, but Unicode wasn't the right standard to deal with it. Unicode maybe wasn't the right standard to deal with anything. At least nothing is wiggling. Of those Unicode points which are graphical, at least all of them can still be printed on paper and won't require a screen. I wonder how long that invariant lasts.
- account42 5mo ago
  It's classic scope creep resulting in unmanageable bloat.
- adzm 5mo ago
  Personally I think the whole emoji thing is a triumph of Unicode. Being able to convey more subtext through emoji makes communication so much easier especially across language boundaries.
  - Bolwin 5mo ago
    I've never seen emoji used for subtext. Usually they just repeat or emphasize what's in the text
- trvz 5mo ago
  I enjoy putting emojis in folder names on my computer for easier visual identification. Also, in passwords on websites to keep developers on their toes.
  - sourcegrift 5mo ago
    The password thing is hilarious but the former breaks some of my Unix tools
- 5mo ago
creepe388 5mo ago
S
creepe388 5mo ago
E
rurban 5mo ago
I see a significant folding tables size increase: 13615 towctrans-5.h 14889 towctrans-6.h 15477 towctrans-7.h 15815 towctrans-8.h 16454 towctrans-9.h 16460 towctrans-10.h 16756 towctrans-11.h 16955 towctrans-12.h 17068 towctrans-13.h 17456 towctrans-15.h 17456 towctrans-14.h 17701 towctrans-17.h 17721 towctrans-16.h 18620 towctrans-18.h Made with cpan Unicode::Towctrans; for v in `seq 5 18`; do gen_wctrans --out towctrans.h -v $v --ud UnicodeData.$v.txt; done These are already optimized folding-tables.

60 points by birdculture 5mo ago

What everyone actually cares about — new emojis! * Cracking face * Left/Right thumb sign * Monarch butterfly * Pickle * Lighthouse * Meteor * Eraser * Net with handle

chirsz 5mo ago
Still no seahorse
- JohnKemeny 5mo ago
  If the seahorse emoji is introduced, we will have to train new foundation models. The costs connected to the introduction of the seahorse emoji will be in the billions.
- zarzavat 5mo ago
  You're absolutely right—the seahorse emoji was added in Unicode version 19.0.0 after OpenAI purchased the Unicode Consortium and converted it to a for-profit corporation.
  - simondotau 5mo ago
    The seahorse being, of course, among the first commercial Unicode characters that require a subscription to use.
Rp8yXmdmr 5mo ago
I would be more interested if they are ever going to cancel HAN unification. Looking at their "Factors for Exclusion" list it could be summarized by "we made some mistakes in past but are sticking to it" :D
- lifthrasiir 5mo ago
  Han Unification was effectively "fixed" by Ideographic Variation Sequences, so no.
  - deleted 5mo ago
    [deleted]
  - account42 5mo ago
    You mean theoretically. Effectively, nothing is fixed yet.
    - lifthrasiir 5mo ago
      IVD works, theoretically and practically (recent versions of OpenType have an explicit support for them). It's not their fault that Japanese vendors have been not very quick to adopt them.
      - account42 5mo ago
        If a Japanese and Taiwanese person type things with their keyboards and end up with the same bytes for different logical characters then no things do not work practically for any practical definition of "practically".
        lifthrasiir 5mo ago
        Your argument is absurd because people don't see code---they see glyphs, and using the same code for slightly different glyphs is a non-issue when they are not interchanged. (And when they are interchanged, both would see glyphs "correct" to them anyway.) Japaneses are sensitive to Han unification only because they recognize more glyph variations (Z-variants) than what Unicode originally could, and IVS is exactly a tool for ensuring exact glyphs assuming cooperative vendors. Not to mention that Han unification was already quite weakened by source separation principles in the first place.
      - 5mo ago
wongarsu 5mo ago
Other great additions: - Left and Right parenthesis with middle ring [1] - A wiggly exclamation mark expressing mirth or laughter [1] (edit: and something I completely missed: the inverted version can express sarcasm) - Cuneiform numerals, including lots of arranged dots that might be useful in other contexts [2] - New variations of "measured angle" and "sector" [3] - A transparent cube and a white cube [4] Also a couple new combining marks And for anyone who wants to see what the reference images for the new emojis look like: Lighthouse: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F680.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F680.p... Other new Emojis: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1FA70.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1FA70.p... 1: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-2E00.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-2E00.pd... 2: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-12550.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-12550.p... 3: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1CEC0.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1CEC0.p... 4: https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F780.pdf https://www.unicode.org/charts/PDF/Unicode-18.0/U180-1F780.p...
BadBadJellyBean 5mo ago
I need a table emoji because then I could combine it with a horse emoji. This would be "Pferd Tisch" (Horse Table) in German which sounds similar to "Fertig" which translates to "done". Yes I want it only for that dumb joke.
Symbiote 5mo ago
Does anyone know why a monarch butterfly was added, when there is already a butterfly emoji?
pentamassiv 5mo ago
Sadly it looks like it will be a dead monarch butterfly https://www.emilydamstra.com/please-enough-dead-butterflies/ https://www.emilydamstra.com/please-enough-dead-butterflies/

computer23 5mo ago

The internet needs a "tin foil hat" emoji, but two proposals have been rejected :( Emoji proposals and status: https://unicode.org/emoji/emoji-proposals-status.html https://unicode.org/emoji/emoji-proposals-status.html

brikym 5mo ago
Seems like a conspiracy. Also it's so silly that pistol turned into water pistol.
turblety 5mo ago
Did they give a reason why it was declined? Was it some bureaucratic "form not filled in correct" thing, or are they actually against the concept of it?
- lifthrasiir 5mo ago
  Vendors-have-no-capacity-to-handle-more-than-handful-number-of-emojis-per-each-release thing. To elaborate: it should be plain obvious that not every Emoji proposal can be accepted even though all of them are correctly filed, as there would be too many Emojis there then. So there has to be some threshold, and that threshold is mostly stipulated by vendors' willingness to process new Emoji characters for designing fonts and updating softwares in time.
- zyx321 5mo ago
  That list only includes suggestions that were seriously considered and voted on. Since it's a vote, there is no single official 'reason' for rejection. If I had to guess: it would be confusing to anyone who didn't grow up with American TV shows.
  - pwdisswordfishq 5mo ago
    Eh, it's not like there are hundreds of emojises pretty much exclusively tied to Japanese culture.

poulpy123 5mo ago

looking at the changes it makes me wonder: - is there an usable font the cover all unicode ? - if not is there really a point to include everything possible in unicode ? - how many space is remaining for new alphabet and smileys ? - how do they handle changes in scripts, for example if new proto-cuneiform or seal script symbols are discovered ?

pveierland 5mo ago
The Noto fonts have great coverage: https://notofonts.github.io/overview/ https://notofonts.github.io/overview/
- infinita740 5mo ago
  Pretty cool vizualisation. There is also GNU unifont [1] "The original intent of Unifont was to offer a simple font format with wide Unicode coverage to render something meaningful for each Unicode code point" [1] https://unifoundry.com/unifont/index.html https://unifoundry.com/unifont/index.html
wongarsu 5mo ago
> if not is there really a point to include everything possible in unicode ? Needing to load three fonts to show a single document that mixes vastly different character sets is still infinitely better than not being able to have those different characters in the same .txt or .md file at all > how many space is remaining for new alphabet and smileys ? Unicode can encode about 1100k code points, and about 800k of those are currently unassigned and available for future scripts or characters
- xigoi 5mo ago
  Also, the 1.1M limit is because of UTF-16. If UTF-16 was deprecated in favor of UTF-8, the limit could be much higher.
  - hulitu 5mo ago

izissise 5mo ago

No toki pona?

pwdisswordfishq 5mo ago
Does it need any additional code points?
- florianist 5mo ago
  The Toki Pona script (aka sitelen pona) needs some codepoints for its ideograms. While Toki Pona is not in Unicode, tokiponists have mostly agreed to use the U+F19xx range in the Private Use Area-A. Most fonts rendering sitelen pona uses that. But using PUA is problematic (no character properties, a lot of restrictions on the web, and constant clashes with other fonts [such as the "nerdfonts" for example]).
lifthrasiir 5mo ago
For now, follow UCSUR: https://sona.pona.la/wiki/Under-ConScript_Unicode_Registry https://sona.pona.la/wiki/Under-ConScript_Unicode_Registry

brikym 5mo ago

Hey ChatGPT show me a seahorse emoji.

sourcegrift 5mo ago

Personally the whole emoji thing is an unmitigated disaster. I'm okay with smileys and gestures but everything else is pointless

vintermann 5mo ago
I'm okay with smileys, but Unicode wasn't the right standard to deal with it. Unicode maybe wasn't the right standard to deal with anything. At least nothing is wiggling. Of those Unicode points which are graphical, at least all of them can still be printed on paper and won't require a screen. I wonder how long that invariant lasts.
account42 5mo ago
It's classic scope creep resulting in unmanageable bloat.
adzm 5mo ago
Personally I think the whole emoji thing is a triumph of Unicode. Being able to convey more subtext through emoji makes communication so much easier especially across language boundaries.
- Bolwin 5mo ago
  I've never seen emoji used for subtext. Usually they just repeat or emphasize what's in the text
trvz 5mo ago
I enjoy putting emojis in folder names on my computer for easier visual identification. Also, in passwords on websites to keep developers on their toes.
- sourcegrift 5mo ago
  The password thing is hilarious but the former breaks some of my Unix tools
5mo ago

creepe388 5mo ago

rurban 5mo ago

I see a significant folding tables size increase: 13615 towctrans-5.h 14889 towctrans-6.h 15477 towctrans-7.h 15815 towctrans-8.h 16454 towctrans-9.h 16460 towctrans-10.h 16756 towctrans-11.h 16955 towctrans-12.h 17068 towctrans-13.h 17456 towctrans-15.h 17456 towctrans-14.h 17701 towctrans-17.h 17721 towctrans-16.h 18620 towctrans-18.h Made with cpan Unicode::Towctrans; for v in `seq 5 18`; do gen_wctrans --out towctrans.h -v $v --ud UnicodeData.$v.txt; done These are already optimized folding-tables.