3 ms·

This used to be the case when JS was created. However, I don't think that this is true anymore. It's definitely not under the hood. Nor, if I recall correctly,

by Yoric 4y ago

This used to be the case when JS was created. However, I don't think that this is true anymore. It's definitely not under the hood. Nor, if I recall correctly, if you attempt to convert them from/to typed arrays. Maybe if you use `fromCharCode`?

Ferrotin 4y ago
JS strings are exposed as UTF-16 in the language — surrogate pairs have length 2 and you can index into half of them.
erutuon 4y ago
A JavaScript string is conceptually an array of 16-bit unsigned integers (https://tc39.es/ecma262/multipage/ecmascript-data-types-and-values.html#sec-ecmascript-language-types-string-type https://tc39.es/ecma262/multipage/ecmascript-data-types-and-...). String indexing and substring operations treat strings this way, and they couldn't change the model easily without breaking code. If you index a string containing an emoji with a code point greater than U+FFFF, you get string containing an unpaired surrogate code unit, which makes the string UCS-2, not UTF-16. I don't know how it's implemented memory-wise in the various JavaScript engines and am not familiar with typed arrays.
josephg 4y ago
> It's definitely not under the hood. Really? I'd love to see some references to that. I know there's some operations to convert a javascript string to UTF8, but everything which measures a string counts using 2-byte UCS2 items. For example, array-style indexing, string.length, str.charCodeAt(), split(), slice(), indexOf(), etc. Also lots of DOM APIs use those weird string lengths too. For example, input element selectionStart / selectionEnd fields measure the selection range by counting surrogate pairs, not characters.
- Yoric 4y ago
  > Really? I'd love to see some references to that. I remember that SpiderMonkey shifted to multiple encodings around 2019 and v8 had done that earlier. Source: I was part of the SpiderMonkey team when the shift happened.

by Yoric 4y ago

Ferrotin 4y ago

JS strings are exposed as UTF-16 in the language — surrogate pairs have length 2 and you can index into half of them.

erutuon 4y ago

A JavaScript string is conceptually an array of 16-bit unsigned integers (https://tc39.es/ecma262/multipage/ecmascript-data-types-and-values.html#sec-ecmascript-language-types-string-type https://tc39.es/ecma262/multipage/ecmascript-data-types-and-...). String indexing and substring operations treat strings this way, and they couldn't change the model easily without breaking code. If you index a string containing an emoji with a code point greater than U+FFFF, you get string containing an unpaired surrogate code unit, which makes the string UCS-2, not UTF-16. I don't know how it's implemented memory-wise in the various JavaScript engines and am not familiar with typed arrays.

josephg 4y ago

> It's definitely not under the hood. Really? I'd love to see some references to that. I know there's some operations to convert a javascript string to UTF8, but everything which measures a string counts using 2-byte UCS2 items. For example, array-style indexing, string.length, str.charCodeAt(), split(), slice(), indexOf(), etc. Also lots of DOM APIs use those weird string lengths too. For example, input element selectionStart / selectionEnd fields measure the selection range by counting surrogate pairs, not characters.

Yoric 4y ago
> Really? I'd love to see some references to that. I remember that SpiderMonkey shifted to multiple encodings around 2019 and v8 had done that earlier. Source: I was part of the SpiderMonkey team when the shift happened.