5 ms·
I don't think it's specifically a 3-byte limit, I think it's just that lots of tools decode UTF-8 into UCS-2 internally instead of UTF-16.
by thristian 14y ago
I don't think it's specifically a 3-byte limit, I think it's just that lots of tools decode UTF-8 into UCS-2 internally instead of UTF-16.
- jrabone 14y agoThat is quite clearly broken, and any tool that does so should be fixed or dumped. This is not new, and Marcus Kuhn had made UTF8 test resources available for years at http://www.cl.cam.ac.uk/~mgk25/unicode.html http://www.cl.cam.ac.uk/~mgk25/unicode.html
- mef 14y agoI've found this sub-page super useful over the years for testing http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt
- thristian 14y agoI believe that MySQL is one such tool. In recent versions you can work around it by asking for the encoding "utf8mb4" instead of "utf8", but I think they have to be quite recent. So yes, another way in which MySQL is quite clearly broken.