Did you read the article, including the part about Asian text? Like it or not, most text these days is embedded in markup languages like XML or HTML, in which a

Did you read the article, including the part about Asian text? Like it or not, most text these days is embedded in markup languages like XML or HTML, in which all of the markup is within the ASCII range. This, coupled with the fact that UTF-8 gives you a factor of 2 savings over UTF-16 for the ASCII range, while only a factor of 1.5 increase over UTF-16 for CJK characters, means that for much text (such as anything on the Web), UTF-8 is actually smaller than UTF-16 even for CJK text. Yes, ASCII is obviously too US centric; you can't encode any writing systems other than the Roman alphabet in ASCII. However, that's not at question here. The question is, which Unicode encoding should you use, so you can represent all writing systems in a single encoding. And the major contenders are UTF-8 and UTF-16. The point of this article is, for that purpose, UTF-8 is a far better choice. > Also, if there's variable length encoding why can't we just do a proper way and improve size for the same computational cost? What do you mean by a "proper way"? If size is what you care about, just compress your data. Compression will do a lot better for a much wider range of data than some clever encoding will. UTF-8 is a carefully constructed encoding designed to meet several design criteria. For instance, you could get better size for a wider range of character sets by having a single byte to represent switching between character sets; so you could use that byte, and then a whole bunch of 2 byte CJK characters. But that would defeat one of the design goals of UTF-8, which is to be self synchronizing. That means that if you get a partial sequence (such as a sequence that has been truncated), you can start decoding the characters after a fixed number of bytes. In the case of UTF-8, you will never have to go more than 3 bytes before you can start decoding again. In my hypothetical scheme where certain symbols were used to switch between character sets, you would not be able to interpret anything until you found the next such symbol. This makes UTF-8 more robust in the face of errors. Another design goal of UTF-8 was to be backwards-compatible with ASCII. Like it or not, ASCII has been the standard encoding for decades, and there is a lot of text in ASCII and a lot of software that uses ASCII delimiters and the like. So, while it would be possible, in theory, to define a character encoding that is more "fair" than UTF-8, that ignores many of the other goals of the design of UTF-8. And UTF-8 is widely supported and used (it is the most popular encoding on the Web, even in places like Japan, and a close second in China), while a new encoding would require another large, global, and painful transition process to introduce.