5 ms·
Should new programming languages built around internal use of UTF-8 be opting their binaries into this behavior by default? (I'm mostly thinking about Rust, wh
by geofft 5y ago
Should new programming languages built around internal use of UTF-8 be opting their binaries into this behavior by default?
(I'm mostly thinking about Rust, which I think has to do some contortions on Windows. Probably also Go, to the extent it uses native APIs; arguably Python too.)
- ChrisSD 5y agoThe Rust standard library has contortions in order to support invalid UTF-16. To do this it uses an encoding called "WTF-8"[0], which is an extension to UTF-8. If the standard library dropped this requirement and only supported valid Unicode then it could simply use normal UTF-8 instead of WTF-8. [0]: https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/
- int_19h 5y agoThe problem right now is that forcing the codepage to UTF-8 only works on Windows 10 build 1903+. So quietly forcing this opt-in may result in situations where everything works fine for the devs, but some of their users see breakage for no clear reason. Long-term, though? Absolutely.