3 ms·
If it was called just "string" or something and then it quietly explained somewhere that it's actually only the BMP and not all of Unicode that's not so surpris
by tialaramex 5y ago
If it was called just "string" or something and then it quietly explained somewhere that it's actually only the BMP and not all of Unicode that's not so surprising, but they deliberately called it UTF-8 even though it just isn't.
Imagine if you bought a USB mouse and then you discovered, oh, they didn't bother implementing USB, this is just a PS/2 mouse with the USB connector. Works fine so long as you use the supplied PS/2 "adaptor" but it isn't actually USB even though it says it is on the box. That mouse is defective. MySQL is defective as supplied. Which won't come as a surprise to many experienced MySQL developers.
I actually rather like MySQL, but it's like bash or Excel, it is a good tool for quick estimates or prototyping, and a poor tool for actual engineering. For the cost of all the work needed to get reliable high quality results with MySQL you could have used a different database where the baseline effort to get it working was higher but what you get is more solid, and "utf8mb4" is an example of that extra work in MySQL.
- petee 5y agoWhile searching yesterday, I recall the mysql documentation did specify this this quirk. As I gather, this change happened prior to or at the change to 4 char max utf; was supplemental plane support required at that time? 3 char is still utf8, just not fully supported, but I really do not know. Not to defend, just saying things were new and fluid back then, though it should have been fixed properly alot sooner.
- tialaramex 5y agoIn Modern MySQL the documentation does - if you look - tell you that utf8mb4 exists to actually do UTF-8 and if you think to check it mentions that their utf8 is not in fact UTF-8 but only "utf8mb3". But fifteen years ago most of that documentation doesn't exist, and utf8mb4 doesn't exist, and if you either read the MySQL documentation carefully and realise "utf8" isn't UTF-8, and only encodes the Basic Multilingual Plane, you are stuck anyway, MySQL doesn't have a sensible way forward for you. I would argue that by 2002 when this bug (let's call it what it is) was introduced, the writing was on the wall, UCS-2 is dead, which means the "it's just easier" justification for choosing 16-bit is also dead, so UTF-8 is the natural choice for software that isn't meanwhile committed to a 16-bit encoding anyway (as Java and Windows are). That makes shipping UTF-8 a good idea, but it makes choosing utf8mb3 a really weird idea that already seems certain to cause pain.
- petee 5y agoCould be. And its weird especially when supporting all 4 bytes would be easy (so it seems). For fun I did dig around though, and found that even in the earliest documentation they clearly state only supporting 3 bytes, though not why. It says this in 2005 which only justifies not supporting 5/6 bytes, but not four. So confusing: "RFC 3629 describes encoding sequences that take from one to four bytes. Currently, MySQL UTF8 support does not include four-byte sequences. (An older standard for UTF8 encoding is given by RFC 2279, which describes UTF8 sequences that take from one to six bytes. RFC 3629 renders RFC 2279 obsolete; for this reason, sequences with five and six bytes are no longer used.)" ...Wha? (2003/2005, they changed urls) - https://web.archive.org/web/20031211033604/http://www.mysql.com/doc/en/Charset-Unicode.html https://web.archive.org/web/20031211033604/http://www.mysql.... - https://web.archive.org/web/20050226012911/http://dev.mysql.com/doc/mysql/en/charset-unicode.html https://web.archive.org/web/20050226012911/http://dev.mysql.... I personally think anything critical that was worth storing in a new character format still should have been properly tested and documentation consulted, that is on the end-user. Anything not critical where you wouldn't expect to care...still should have tested, but yeah that sucks.