3 ms·
> the same site that then fixed that bug and forced all blog descriptions to be utf-8, but then implemented size limits as a number of bytes. > so they truncat
by account42 5y ago
> the same site that then fixed that bug and forced all blog descriptions to be utf-8, but then implemented size limits as a number of bytes.
> so they truncated mid-utf-8 character, which made parsing the XML A REAL FUN TASK LET ME TELL YOU
How is this a problem unless you do UTF-8 validation when parsing the XML, in which case stop doing that for data you don't control.
- JimDabell 5y agoIf you truncate a UTF-8 string mid-character, then can’t that sequence swallow the next byte, such as <, which would break XML parsing completely?
- toast0 5y agoNo, UTF-8 is self synchronizing... The first byte of a code point is clearly the first byte of a code point, so it's possible to tell that there was a truncated code point, as long as the next one starts correctly (if you lost two bytes in a row, you might get a mashed up code point, but that's a different loss scenario)
- _moof 5y agoUTF-8 encoding works like so: 0xxxxxxx 110xxxxx 10xxxxxx 1110xxxx 10xxxxxx 10xxxxxx 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx ...where the "x" bits encode the code point. So if you're halfway through a four-byte sequence and see a byte like 0xxxxxxx, you know something's wrong.
- Joker_vD 5y agoRight, Postel's law: validating the data you receive reduces the interoperability, so don't do that.
- torstenvl 5y agoif (c & 0b10000000) continue;