4 ms·

East Asian HTML and XML documents are typically smaller in utf8 than utf16 because the of the markup. (on mobile so u can't prove it right now. maybe when I get

East Asian HTML and XML documents are typically smaller in utf8 than utf16 because the of the markup. (on mobile so u can't prove it right now. maybe when I get home tonight) utf16 is a bad default for almost everybody. The actual reason utf16 is used is because it was easier to port to utf16 from ucs16. Much the way it's often acceptable to assume utf8 is ASCII it's often acceptable to assume utf16 is utf8. Even today, I imagine there are a lot of applications which assume utf16 is actually ucs16.

edflsafoiewq 7y ago
That depends on what kind documents of course. It isn't usually true for text-centric content. Here's Dazai Osamu's Hashire Melos: https://www.aozora.gr.jp/cards/000035/files/1567_14913.html https://www.aozora.gr.jp/cards/000035/files/1567_14913.html. Despite having an unusually large amount of markup (most Japanese text wouldn't add the ruby characters), it's smaller as UTF-16 than UTF-8.