3 ms·

The problem is that in Python 2, "str" is a bytestring and "unicode" is a Unicode string, whereas in Python 3 "str" is a Unicode string and "bytes" is a bytestr

The problem is that in Python 2, "str" is a bytestring and "unicode" is a Unicode string, whereas in Python 3 "str" is a Unicode string and "bytes" is a bytestring. In my experience this is the main reason why Python 2 users get confused. I can understand why the renaming might have been a good idea (to avoid changing the semantic meaning of an expression while leaving the syntax intact), but I can understand why people don't "grok it" immediately, though it is rather simple if you know it.

the_mitsuhiko 10y ago
> The problem is that in Python 2, "str" is a bytestring and "unicode" is a Unicode string, whereas in Python 3 "str" is a Unicode string and "bytes" is a bytestring. `str` in Python 3 is `unicode` in Python 2. The bytestring type was removed and replaced with a bytes type. Not sure what's particularly confusing about this. None of this however is the issue with Unicode in Python in general. I think all these articles fail to understand that the actual story behind unicode in the language is very complex.
- ThePhysicist 10y ago
  Yes that's what I said, and there's nothing "wrong" with it, I just think that it's where most of the confusion comes from, as many people that come from Python 2 think of `str` as a byte-string, while in Python 3 it's a Unicode string. I think the "unicode-by-default" approach of Python 3 is a great reason to switch to it (and I did so for almost all my codebases), converting code can sometimes cause surprises though, especially due to the lack of type checks in Python combined with the more stricter checking that Python 3 does when working with strings (i.e. adding Unicode and byte strings throws an exception in Python 3, whereas Python 2 would simply try to convert the byte-string implicitly into a string and only throw an exception if this fails). All theses changes combined make converting code from 2 to 3 more risky. I think especially the handling of type coercion between Unicode and bytes objects could have been handled differently, as it is (IMO) a big semantic change. As an example, just today I ran into a problem with WTForms and Flask (which is a great framework btw, thanks for writing it!), where I used a cookie-based session store provided by `itsdangerous`, which deserializes a value stored in the session either into a byte-string or a Unicode string. After switching my code from Python 2 to Python 3, there were still some clients in the wild that had a session object in their cookie generated by Python 2 containing byte-strings, and `itsdangerous` happily deserialized it into a byte-string as well, causing an exception when my Python 3 code tried adding it to a Unicode string. With new sessions this didn't happen of course (as the objects were serialized as Unicode), which made this thing quite fun to debug. Granted, this is not a problem with Unicode in Python 3, but I think it illustrates the kind of challenges you face when migrating codebases from one to the other. What I can agree on is that we'd be much better off if everyone was using Python 3, and eventually we'll get there.
- rtpg 10y ago
  I really wish they named it something different. Like `text` and `bytes` In that scenario you have no name link between Py2 objects and Py3 objects. The `str` name reuse for different objects makes the 2-3 transition that much harder.

the_mitsuhiko 10y ago

> The problem is that in Python 2, "str" is a bytestring and "unicode" is a Unicode string, whereas in Python 3 "str" is a Unicode string and "bytes" is a bytestring. `str` in Python 3 is `unicode` in Python 2. The bytestring type was removed and replaced with a bytes type. Not sure what's particularly confusing about this. None of this however is the issue with Unicode in Python in general. I think all these articles fail to understand that the actual story behind unicode in the language is very complex.

ThePhysicist 10y ago
Yes that's what I said, and there's nothing "wrong" with it, I just think that it's where most of the confusion comes from, as many people that come from Python 2 think of `str` as a byte-string, while in Python 3 it's a Unicode string. I think the "unicode-by-default" approach of Python 3 is a great reason to switch to it (and I did so for almost all my codebases), converting code can sometimes cause surprises though, especially due to the lack of type checks in Python combined with the more stricter checking that Python 3 does when working with strings (i.e. adding Unicode and byte strings throws an exception in Python 3, whereas Python 2 would simply try to convert the byte-string implicitly into a string and only throw an exception if this fails). All theses changes combined make converting code from 2 to 3 more risky. I think especially the handling of type coercion between Unicode and bytes objects could have been handled differently, as it is (IMO) a big semantic change. As an example, just today I ran into a problem with WTForms and Flask (which is a great framework btw, thanks for writing it!), where I used a cookie-based session store provided by `itsdangerous`, which deserializes a value stored in the session either into a byte-string or a Unicode string. After switching my code from Python 2 to Python 3, there were still some clients in the wild that had a session object in their cookie generated by Python 2 containing byte-strings, and `itsdangerous` happily deserialized it into a byte-string as well, causing an exception when my Python 3 code tried adding it to a Unicode string. With new sessions this didn't happen of course (as the objects were serialized as Unicode), which made this thing quite fun to debug. Granted, this is not a problem with Unicode in Python 3, but I think it illustrates the kind of challenges you face when migrating codebases from one to the other. What I can agree on is that we'd be much better off if everyone was using Python 3, and eventually we'll get there.
rtpg 10y ago
I really wish they named it something different. Like `text` and `bytes` In that scenario you have no name link between Py2 objects and Py3 objects. The `str` name reuse for different objects makes the 2-3 transition that much harder.