4 ms·

Unicode is a secret moat for the large tech companies. The more crap they add, the harder it is for anyone who starts fresh to catch up. For every new emoticon,

Unicode is a secret moat for the large tech companies. The more crap they add, the harder it is for anyone who starts fresh to catch up. For every new emoticon, that teenagers expect to have, the chance they have more platform choices diminishes. Standards like these should not exist, unless the standard body also provides a free conformant reference implementation.

baggy_trough 6y ago
"should not exist" is pretty strong. Unicode is a great improvement over what came before.
- centimeter 6y ago
  Is your life really improved by having access to <tilted heart face>? Unicode is a massive clusterfuck. We’ve completely eliminated most of the advantages it might have held in the first place, like unambiguous character seeking from an arbitrary offset into the encoded byte stream. Now you have characters composed of multiple codepoints, for a few legitimate use cases like diacritics (which should actually not be handled this way) but mostly for dumb hybrid emoji shit. “Zero width joiners” should not exist. Even the diacritic stuff is total nonsense, and unsurprisingly has resulted in multiple serious vulnerabilities on various platform (iOS has had like 5 diacritic-induced crashes). It’s fine to have a panlingual character standard, but it should be implemented as a minimal set of complete characters (no multi-codepoint bullshit) without pictorial nonsense like <various stupid multi-codepoint emoji> which should obviously be handled by a generalized vector graphics format. Edit: in a hilarious demonstration of my point, HN strips emoji from the post text!
fastball 6y ago
I'm not sure I follow. I use Unicode in my startup – what exactly do I need to catch up on?
- quelsolaar 6y ago
  (Not knowing your startup ill make loads of assumptions here, sorry if I get it wrong) If you are drawing Unicode characters that include a full range of Emoticons, you are most likely using a platform API provided by Apple/Google/Microsoft or similar. You, like most developers, most likely live in "Big-tech land", so you get this for "free". If you want to port your application to an open source platform or a new upstart platform, where this isn't available, they may not have the resources to draw thousands of Emoticons, so its harder for them to attract you as a developer. Lets say someone is writing a SMS app for something like a PinePhone. SMS is open enough protocol that this is possible. The UI for a chat app is something you can build in fairly short order. There are plenty of open fonts, you can use, but there are no open fonts that have thousands of detailed and sometimes animated Emoticons. Without Emoticons very few people are going to want to use a PinePhone as their main communications device. The amount of effort needed to implement a SMS chat vs a SMS chat + draw all emoticons people use is massive. Unicode, and SMS are both open, but given the size of the Unicode Spec, its effectively shutting out any small player form implementing it. Big tech gets to claim they use open standards, but at the same time make sure no one can threaten them. Its a bit like when Microsoft tried to make a standard for documents so complicated that only word could implement it.
  - lifthrasiir 6y ago
    You seems unaware of why emoji got accepted to Unicode in the first place. Emoji was originated from Japan's three big telcos, which had different sets and encodings of emojis (e.g. some used Shift_JIS extension, some used embedded images via HTML, some used SI/SO sequences). It was already impossible to write a SMS app without accounting for this peculiarity. Unicode thus made this situation much easier, not harder.
  - mschuster91 6y ago
    > If you want to port your application to an open source platform or a new upstart platform, where this isn't available, they may not have the resources to draw thousands of Emoticons, so its harder for them to attract you as a developer. License a commercial font or use Twitter's CC-BY font: https://twemoji.twitter.com/ https://twemoji.twitter.com/ > Unicode, and SMS are both open, but given the size of the Unicode Spec, its effectively shutting out any small player form implementing it. You only need to implement the Unicode spec if you want your own emoji designs.
centimeter 6y ago
It’s also a way for large tech companies to unilaterally exercise neurotic political projects, like replacing the emoji for “pistol” with a squirt gun. Apple or Google can do whatever they want to Emoji in violation of the Unicode standard and other vendors are more or less forced to follow suit.
lifthrasiir 6y ago
Unicode is a vast standard and have many separate parts. Unicode the character set is a significant but miniscule part of it. You don't need much to support Unicode if you don't do display (or delegate the display to someone else, like web browsers). The conformance requirements are also loosely worded that you can declare many parts of them as unsupported and move on [1]. Unicode also packages an essential but (in hindsight) complex concept and process like "what is the word boundary" or "what are characters usable as a part of identifiers" into a neat algorithm and data table. The algorithm is more complex than the data, which is readily available for you. Assigning a new character to Unicode mostly means a change to that data, not to the algorithm. If you consider one should be able to display every assigned character as expected to be fully conforming to Unicode---I stress it's not true, but if we assume so---then there would be no conforming implementation of Unicode at all. All Unicode implementations just implement what they need to support. [1] For example, the requirement C12 (https://www.unicode.org/versions/Unicode13.0.0/ch03.pdf#page=13 https://www.unicode.org/versions/Unicode13.0.0/ch03.pdf#page...) specifies that you need to implement the Unicode Bidirectional algorithm only if you support right-to-left characters. In the other words you don't have to implement the Bidi algorithm for conformance.

baggy_trough 6y ago

"should not exist" is pretty strong. Unicode is a great improvement over what came before.

centimeter 6y ago
Is your life really improved by having access to <tilted heart face>? Unicode is a massive clusterfuck. We’ve completely eliminated most of the advantages it might have held in the first place, like unambiguous character seeking from an arbitrary offset into the encoded byte stream. Now you have characters composed of multiple codepoints, for a few legitimate use cases like diacritics (which should actually not be handled this way) but mostly for dumb hybrid emoji shit. “Zero width joiners” should not exist. Even the diacritic stuff is total nonsense, and unsurprisingly has resulted in multiple serious vulnerabilities on various platform (iOS has had like 5 diacritic-induced crashes). It’s fine to have a panlingual character standard, but it should be implemented as a minimal set of complete characters (no multi-codepoint bullshit) without pictorial nonsense like <various stupid multi-codepoint emoji> which should obviously be handled by a generalized vector graphics format. Edit: in a hilarious demonstration of my point, HN strips emoji from the post text!

fastball 6y ago

I'm not sure I follow. I use Unicode in my startup – what exactly do I need to catch up on?

quelsolaar 6y ago
(Not knowing your startup ill make loads of assumptions here, sorry if I get it wrong) If you are drawing Unicode characters that include a full range of Emoticons, you are most likely using a platform API provided by Apple/Google/Microsoft or similar. You, like most developers, most likely live in "Big-tech land", so you get this for "free". If you want to port your application to an open source platform or a new upstart platform, where this isn't available, they may not have the resources to draw thousands of Emoticons, so its harder for them to attract you as a developer. Lets say someone is writing a SMS app for something like a PinePhone. SMS is open enough protocol that this is possible. The UI for a chat app is something you can build in fairly short order. There are plenty of open fonts, you can use, but there are no open fonts that have thousands of detailed and sometimes animated Emoticons. Without Emoticons very few people are going to want to use a PinePhone as their main communications device. The amount of effort needed to implement a SMS chat vs a SMS chat + draw all emoticons people use is massive. Unicode, and SMS are both open, but given the size of the Unicode Spec, its effectively shutting out any small player form implementing it. Big tech gets to claim they use open standards, but at the same time make sure no one can threaten them. Its a bit like when Microsoft tried to make a standard for documents so complicated that only word could implement it.
- lifthrasiir 6y ago
  You seems unaware of why emoji got accepted to Unicode in the first place. Emoji was originated from Japan's three big telcos, which had different sets and encodings of emojis (e.g. some used Shift_JIS extension, some used embedded images via HTML, some used SI/SO sequences). It was already impossible to write a SMS app without accounting for this peculiarity. Unicode thus made this situation much easier, not harder.
- mschuster91 6y ago
  > If you want to port your application to an open source platform or a new upstart platform, where this isn't available, they may not have the resources to draw thousands of Emoticons, so its harder for them to attract you as a developer. License a commercial font or use Twitter's CC-BY font: https://twemoji.twitter.com/ https://twemoji.twitter.com/ > Unicode, and SMS are both open, but given the size of the Unicode Spec, its effectively shutting out any small player form implementing it. You only need to implement the Unicode spec if you want your own emoji designs.

centimeter 6y ago

It’s also a way for large tech companies to unilaterally exercise neurotic political projects, like replacing the emoji for “pistol” with a squirt gun. Apple or Google can do whatever they want to Emoji in violation of the Unicode standard and other vendors are more or less forced to follow suit.

lifthrasiir 6y ago

Unicode is a vast standard and have many separate parts. Unicode the character set is a significant but miniscule part of it. You don't need much to support Unicode if you don't do display (or delegate the display to someone else, like web browsers). The conformance requirements are also loosely worded that you can declare many parts of them as unsupported and move on [1]. Unicode also packages an essential but (in hindsight) complex concept and process like "what is the word boundary" or "what are characters usable as a part of identifiers" into a neat algorithm and data table. The algorithm is more complex than the data, which is readily available for you. Assigning a new character to Unicode mostly means a change to that data, not to the algorithm. If you consider one should be able to display every assigned character as expected to be fully conforming to Unicode---I stress it's not true, but if we assume so---then there would be no conforming implementation of Unicode at all. All Unicode implementations just implement what they need to support. [1] For example, the requirement C12 (https://www.unicode.org/versions/Unicode13.0.0/ch03.pdf#page=13 https://www.unicode.org/versions/Unicode13.0.0/ch03.pdf#page...) specifies that you need to implement the Unicode Bidirectional algorithm only if you support right-to-left characters. In the other words you don't have to implement the Bidi algorithm for conformance.