3 ms·

They didn't get it perfect, but, compared to what it was like to deal with Unicode in Python 2, I'd say they absolutely succeeded at improving the experience of

They didn't get it perfect, but, compared to what it was like to deal with Unicode in Python 2, I'd say they absolutely succeeded at improving the experience of working with Unicode text.

afiori 4y ago
weren't python 2 string essentially unencoded byte-strings? In my uninformed opinion they should have simply added the u"..." and b"..." formats and forbid mixing them or using a u with a b operation and viceversa. python 3 could have simply changed the default from b to u for untagged strings.
- mumblemumble 4y ago
  This is how Python 2 did it, only with the difference that you didn't need the b prefix for ascii/byte strings. The problem was that this made a muddle of all the string handling functions, which needed to be prepared to handle both types. Including ones you write yourself. And, Python being a dynamic language, getting this right was fiddly and error prone. "Forbid mixing the two" was a workable, if ergonomically annoying, in statically typed languages that predated and then subsequently had to adopt Unicode. For Python, though, it was an enormous minefield of runtime errors.
  - tyingq 4y ago
    And you also had to detect which version of Python2, since Python <= 2.5.x didn't have some of the unicode support that later versions did.
- akubera 4y ago
  Isn't that what they ended up doing in Python3? $ python2 >>> "abc" + b"123" 'abc123' >>> "abc" + u"123" u'abc123' $ python3 >>> "abc" + b"123" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can only concatenate str (not "bytes") to str >>> "abc" + u"123" 'abc123' >>> type(u"123") <class 'str'> P.S. Your use of the word "simply" there is a reminder of how easy it is to underestimate the complexity of handling encoding properly, since it took years (a decade?) to port libraries from Py2 to Py3, since it doesn't just affect string literals, but general IO when reading from files and sockets.

They didn't get it perfect, but, compared to what it was like to deal with Unicode in Python 2, I'd say they absolutely succeeded at improving the experience of working with Unicode text.

afiori 4y ago

weren't python 2 string essentially unencoded byte-strings? In my uninformed opinion they should have simply added the u"..." and b"..." formats and forbid mixing them or using a u with a b operation and viceversa. python 3 could have simply changed the default from b to u for untagged strings.

mumblemumble 4y ago
This is how Python 2 did it, only with the difference that you didn't need the b prefix for ascii/byte strings. The problem was that this made a muddle of all the string handling functions, which needed to be prepared to handle both types. Including ones you write yourself. And, Python being a dynamic language, getting this right was fiddly and error prone. "Forbid mixing the two" was a workable, if ergonomically annoying, in statically typed languages that predated and then subsequently had to adopt Unicode. For Python, though, it was an enormous minefield of runtime errors.
- tyingq 4y ago
  And you also had to detect which version of Python2, since Python <= 2.5.x didn't have some of the unicode support that later versions did.
akubera 4y ago
Isn't that what they ended up doing in Python3? $ python2 >>> "abc" + b"123" 'abc123' >>> "abc" + u"123" u'abc123' $ python3 >>> "abc" + b"123" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can only concatenate str (not "bytes") to str >>> "abc" + u"123" 'abc123' >>> type(u"123") <class 'str'> P.S. Your use of the word "simply" there is a reminder of how easy it is to underestimate the complexity of handling encoding properly, since it took years (a decade?) to port libraries from Py2 to Py3, since it doesn't just affect string literals, but general IO when reading from files and sockets.