7 ms·

"There is no way for any non-UTF-8 encoding to represent something more, since everything gets converted to UTF-16 internally." Internally in what? Certainly n

"There is no way for any non-UTF-8 encoding to represent something more, since everything gets converted to UTF-16 internally." Internally in what? Certainly not servers, but do all major browsers use UTF-16 internally? Also, does Unicode include all characters from various east Asian encodings now?

hoppipolla 15y ago
The DOM and ECMAScript both assume that strings are sequences of UTF-16 codepoints. So while a browser could use non-UTF-16 internally it wouldn't help much because you would have to convert to UTF-16 in all the externally-facing APIs anyway.
- robin_reala 15y ago
  Nearly 100% true, but not quite: http://mathiasbynens.be/notes/javascript-encoding http://mathiasbynens.be/notes/javascript-encoding
  - obtu 15y ago
    '𝌆'.length == 2 ಠ_ಠ They did the same kludge as the old narrow builds of Python.
- obtu 15y ago
  JS engines use various strategies depending on actual string content because it's more memory- and cache-friendly.
FigBug 15y ago
Windows is UTF-16 internally, unless using the old ASCII functions. NSString on the Mac is UTF-16. I think w_char in Linux is UTF-32. I still think its an awkward statement. It should say Unicode internally, the specific encoding is irrelevant.
- evmar 15y ago
  Even on Linux WebKit and Gecko use UTF-16.
ken 15y ago
> Also, does Unicode include all characters from various east Asian encodings now? I think the answer is either "almost certainly yes" or "it never will", depending on your view of https://en.wikipedia.org/wiki/Unihan https://en.wikipedia.org/wiki/Unihan
- herge 15y ago
  Is this the 'Our language is not a font, but its own set of characters different from chinese' argument the japanese are complaining about?
  - derleth 15y ago
    > Our language is not a font, but its own set of characters different from chinese Only to the extent that this is a gross oversimplification of a complex historical and cultural relationship.

hoppipolla 15y ago

The DOM and ECMAScript both assume that strings are sequences of UTF-16 codepoints. So while a browser could use non-UTF-16 internally it wouldn't help much because you would have to convert to UTF-16 in all the externally-facing APIs anyway.

robin_reala 15y ago
Nearly 100% true, but not quite: http://mathiasbynens.be/notes/javascript-encoding http://mathiasbynens.be/notes/javascript-encoding
- obtu 15y ago
  '𝌆'.length == 2 ಠ_ಠ They did the same kludge as the old narrow builds of Python.
obtu 15y ago
JS engines use various strategies depending on actual string content because it's more memory- and cache-friendly.

FigBug 15y ago

Windows is UTF-16 internally, unless using the old ASCII functions. NSString on the Mac is UTF-16. I think w_char in Linux is UTF-32. I still think its an awkward statement. It should say Unicode internally, the specific encoding is irrelevant.

evmar 15y ago
Even on Linux WebKit and Gecko use UTF-16.

ken 15y ago

> Also, does Unicode include all characters from various east Asian encodings now? I think the answer is either "almost certainly yes" or "it never will", depending on your view of https://en.wikipedia.org/wiki/Unihan https://en.wikipedia.org/wiki/Unihan

herge 15y ago
Is this the 'Our language is not a font, but its own set of characters different from chinese' argument the japanese are complaining about?
- derleth 15y ago
  > Our language is not a font, but its own set of characters different from chinese Only to the extent that this is a gross oversimplification of a complex historical and cultural relationship.