4 ms·
Emoji has always been beyond 2 bytes. The Unicode spec also includes "surrogate pairs" which allows a higher plane code point to be represented as 4 bytes.
by alblue 11y ago
Emoji has always been beyond 2 bytes. The Unicode spec also includes "surrogate pairs" which allows a higher plane code point to be represented as 4 bytes.
- Animats 11y agoThere are a few 2-byte emoji: 0x2639 Frowning face ️0x263a Smiling face (Hacker News doesn't speak much Unicode; the Unicode symbols won't pass through.)
- vardump 11y agoSo what happens if I put a standard smiley in a message? (Grinning face Unicode: U+1F600, UTF-8: F0 9F 98 80) Edit: I see, it disappears.
- simoncion 11y agoHmm. Does HN use MySQL with "utf8" encoding as backend storage? ;)
- masklinn 11y agoIf that were the case, astral characters would be removed (e.g. shavian or the emoji block) but BMP symbols like U+263F MERCURY or U+262C ADI SHAKTI would be left alone, and they're not: "", "" Since U+A420 YI SYLLABLE JJUOX goes through (ꐠ) HN likely explicitly strips most symbols (but not all, U+00B6 PILCROW SIGN (¶) is unmolested)
- riffraff 11y agofor those not getting the joke: mysql has a thing called "utf8" which is not in fact utf8 and will (depending on settings) either truncate text when it meets a 4+ byte character, or raise an error. It also supports real utf8 in more recent versions calling it "utf8mb4"
- simoncion 11y agoEverywhere I've used mysql, the default has been to silently truncate data that doesn't fit in the "utf8" encoding! :(
- david-given 11y agoThis made filing a bug about Thunderbird not sizing astral plane code points correctly slightly more hilarious than it should have been (Mozilla's Bugzilla instance runs on MySQL)...