4 ms·
With the benefit of hindsight, would we include the error detection bits of UTF8 if we could choose not to?
by gorgoiler 3y ago
With the benefit of hindsight, would we include the error detection bits of UTF8 if we could choose not to?
- ssokolow 3y agoYes. Give https://www.youtube.com/watch?v=_mZBa3sqTrI https://www.youtube.com/watch?v=_mZBa3sqTrI a watch... especially the "Oh my God! We've been hacked!" part at 36:20. TL;DR: They had a transient glitch in their network switch and, because Windows uses UTF-16 when sending remote event logs over the wire, whenever it dropped a single byte, it had the effect of swapping the endianness of the messages, resulting in scary Chinese text in the logs. You could get the same effect by naively applying byte-wise processing to UTF-16 or UTF-32, or having an off-by-one error. UTF-8 is self-synchronizing so one-byte errors like that only lose you one character, rather than corrupting the entire stream going forward.