3 ms·
> Python 3 looks like it's UTF-8 No, it never looked like it was UTF-8. It did look like it was Unicode (of an unspecified encoding), but Python allows unpaire
by chrismorgan 5y ago
> Python 3 looks like it's UTF-8
No, it never looked like it was UTF-8. It did look like it was Unicode (of an unspecified encoding), but Python allows unpaired surrogates, which makes for ill-formed Unicode. Python lets you have '\udead'. Well-formed encodings of Unicode (such as UTF-8, UTF-16 and UTF-32) cannot represent U+DEAD.
There’s a big difference between “Unicode” and “UTF-8”.
> PyPy, though, finally went UTF-8.
Oh cool! I remember vaguely hearing the idea being mulled over, years ago. Good to see it happened: https://morepypy.blogspot.com/2019/03/pypy-v71-released-now-uses-utf-8.html https://morepypy.blogspot.com/2019/03/pypy-v71-released-now-....
Wonder what they’ve done about surrogates; the only way they can truly have switched to UTF-8 internally is if they’ve broken compatibility, which I doubt they’ve done; I’m guessing that they actually use WTF-8, not UTF-8.
(In the meantime, I’ve changed “Python 3.3” in the grandparent comment to “CPython 3.3” for accuracy.)