4 ms·
Why doesn't everybody use UTF-8? How much overhead is incurred in encoding a non-ascii language (say Chinese) in UTF-8 compared to UTF-16?
by herge 15y ago
Why doesn't everybody use UTF-8? How much overhead is incurred in encoding a non-ascii language (say Chinese) in UTF-8 compared to UTF-16?
- Karellen 15y agoBecause Windows uses UTF-16 as its Unicode encoding. Why? Because Unicode support in Windows started in NT, and that used UCS2. Why? Because when NT was first being developed in 1989/1990, UTF-8 hadn't been invented yet, and wouldn't be for another few years. And UTF-8 didn't have much traction at all until after 2000, by which time it was too late. Also, Windows has some problems with UTF-8 due to MB_LEN_MAX[0] being 2. Which cannot be changed because that would break the ABI. (e.g. "char foo[MB_LEN_MAX * len + 1];") Those problems aren't totally insurmountable as Windows does have UTF-8 code pages which work, but it's not totally plain sailing. In any case, going back now would be an admission of a mistake. Thus, Windows is UTF-16. And the infection spreads.... [0] http://www.kernel.org/doc/man-pages/online/pages/man3/MB_LEN_MAX.3.html http://www.kernel.org/doc/man-pages/online/pages/man3/MB_LEN...