4 ms·
The short answer is somewhere between Go and Rust strings, which are newer languages that use UTF-8 for interior representation, and also favor it for exterior
by chubot 3y ago
The short answer is somewhere between Go and Rust strings, which are newer languages that use UTF-8 for interior representation, and also favor it for exterior encoding.
Roughly speaking, Java and JavaScript are in the UTF-16 camp, and Python 2 and 3 are in the code points camp. C and C++ have unique problems, but you could also put them in the code points camp.
So there are at least 3 different camps, and a whole bunch of weird variations, like string width being compile-time selectable in interpreters.
More details here:
https://www.oilshell.org/blog/2023/06/ysh-design.html#text https://www.oilshell.org/blog/2023/06/ysh-design.html#text
and here:
https://www.oilshell.org/blog/2023/06/surrogate-pair.html#history-windows-infected-javascript-json-and-python https://www.oilshell.org/blog/2023/06/surrogate-pair.html#hi...
A main design issue is that string APIs shouldn't depend on a mutable global variable -- the default encoding, or default file system encoding. That's an idea that's a disaster in C, and also a disaster in Python.
It leads to buggy programs. Go and Rust differ in their philosophies, but neither of them has that design problem.