3 ms·
Right, Python 3 looks like it's UTF-8, but CPython has that weird 1, 2 or 4 byte representation. PyPy, though, finally went UTF-8. Not sure how they do indexing
by Animats 5y ago
Right, Python 3 looks like it's UTF-8, but CPython has that weird 1, 2 or 4 byte representation. PyPy, though, finally went UTF-8. Not sure how they do indexing.
Go strings are supposed to be UTF-8, but it's not enforced.
Javascript - bleah.
- chrismorgan 5y ago> Python 3 looks like it's UTF-8 No, it never looked like it was UTF-8. It did look like it was Unicode (of an unspecified encoding), but Python allows unpaired surrogates, which makes for ill-formed Unicode. Python lets you have '\udead'. Well-formed encodings of Unicode (such as UTF-8, UTF-16 and UTF-32) cannot represent U+DEAD. There’s a big difference between “Unicode” and “UTF-8”. > PyPy, though, finally went UTF-8. Oh cool! I remember vaguely hearing the idea being mulled over, years ago. Good to see it happened: https://morepypy.blogspot.com/2019/03/pypy-v71-released-now-uses-utf-8.html https://morepypy.blogspot.com/2019/03/pypy-v71-released-now-.... Wonder what they’ve done about surrogates; the only way they can truly have switched to UTF-8 internally is if they’ve broken compatibility, which I doubt they’ve done; I’m guessing that they actually use WTF-8, not UTF-8. (In the meantime, I’ve changed “Python 3.3” in the grandparent comment to “CPython 3.3” for accuracy.)