4 ms·

You do realize unicode has been "built into Python" pretty much since the beginning (Python 2.0, 17 years ago), right? The main difference is that unicode has

by Radim 9y ago

You do realize unicode has been "built into Python" pretty much since the beginning (Python 2.0, 17 years ago), right? The main difference is that unicode has a more efficient internal storage since Python 3.3+ (a neat technical detail), and that mixing bytestrings and unicode will fail with an explicit error since 3.0+ (a good idea IMO). But that Python 2.7 didn't support unicode is simply FUD.

viraptor 9y ago
I don't think anyone claimed it didn't support Unicode. Only that it allowed mixing bytes / strings and the default type most people used from the beginning was str. That's a trap that they'll regret the moment they actually need to handle something outside of Latin-1. Lots of python 2 code out there fails because the default option was simple but bad. I know, because my name broke the CI pipelines in a few different projects.
- Radim 9y ago
  Did you actually read the thread you're replying to, or are you on auto-pilot?
wiz21c 9y ago
I have moved away from Python 2.7 because unicode support was not good enough. Not good enough means I had to prefix every single string with "u" to make sure it's unicode. It was especially painful with code like this : logger.debug("Name of the person I'm debugging {}".format(name)) forgetting the "u" simply lead to various crashes if the name had characters that could not be encoded in whatever the debug was piped to. Always thinking about the "u" was nothing I had the time to. Just an example.
- Radim 9y ago
  Well now you have to prefix b"" instead for bytestrings. The argument works both ways -- there's no magical improvement that happened in Python 3 (except that nice storage optimization in 3.3 that I mentioned above). It's actually good practice to be explicit about the type, and write b"" and u"" always (easier on the person reading the code). The u'' literal prefix was re-introduced in 3.3 for this reason.
  - pvg 9y ago
    The argument works both ways It doesn't because strings and text are a lot more common than bytes. Yours is a really weird line of argument - that the 3.x changes and what's in 2.7 are fundamentally equivalent and thus the changes are outright unnecessary and that the people who made them just got it wrong and did it for no apparent reason or benefit. I get that someone might not like what they did or how they did it but your take should give you pause just by its general improbability.
    - Radim 9y ago
      You're mixing up two things: separating string types in the language, and using prefix literals. Completely orthogonal concerns. As I said, u'' literals were re-introduced by the Python developers themselves, in Python 3.3.
      - deleted 9y ago
        [deleted]
      - pvg 9y ago
        I don't think I am. In this thread you're repeatedly making the point that 2.7 supported Unicode and the difference is mostly technical details of things like internal representation and/or a matter of prefixes or whatnot. This just isn't true. The fundamental change is - in Python 2, strings are bags of bytes and in Python 3 strings are collections of Unicode codepoints and you need to go through an encoding to convert to and from bytes. This is a big (and necessary) deal. No amount of debating the finer points of implementation, feature retention or reintroduction, etc is going to make that difference not matter.
        Radim 9y ago
        What I said: 1. BOTH Python 2 and Python 3 come with built-in support for BOTH bytestring and unicode (contrary to OP's claims I responded to) 2. That mixing bytestrings and unicode will fail with an explicit error since 3.0+ (a good idea IMO) 3. Unicode has a more efficient internal storage since Python 3.3+ (a neat technical detail) 4. It's good practice to be explicit about the type of literals, and write b"" and u"" always 5. That Python 2.7 doesn't support unicode is simply FUD. Can you articulate which point you're actually contesting? I'll be happy to clarify, but I'm honestly unsure what you're responding to.

by Radim 9y ago

viraptor 9y ago

I don't think anyone claimed it didn't support Unicode. Only that it allowed mixing bytes / strings and the default type most people used from the beginning was str. That's a trap that they'll regret the moment they actually need to handle something outside of Latin-1. Lots of python 2 code out there fails because the default option was simple but bad. I know, because my name broke the CI pipelines in a few different projects.

Radim 9y ago
Did you actually read the thread you're replying to, or are you on auto-pilot?

wiz21c 9y ago

I have moved away from Python 2.7 because unicode support was not good enough. Not good enough means I had to prefix every single string with "u" to make sure it's unicode. It was especially painful with code like this : logger.debug("Name of the person I'm debugging {}".format(name)) forgetting the "u" simply lead to various crashes if the name had characters that could not be encoded in whatever the debug was piped to. Always thinking about the "u" was nothing I had the time to. Just an example.

Radim 9y ago
Well now you have to prefix b"" instead for bytestrings. The argument works both ways -- there's no magical improvement that happened in Python 3 (except that nice storage optimization in 3.3 that I mentioned above). It's actually good practice to be explicit about the type, and write b"" and u"" always (easier on the person reading the code). The u'' literal prefix was re-introduced in 3.3 for this reason.
- pvg 9y ago
  The argument works both ways It doesn't because strings and text are a lot more common than bytes. Yours is a really weird line of argument - that the 3.x changes and what's in 2.7 are fundamentally equivalent and thus the changes are outright unnecessary and that the people who made them just got it wrong and did it for no apparent reason or benefit. I get that someone might not like what they did or how they did it but your take should give you pause just by its general improbability.
  - Radim 9y ago
    You're mixing up two things: separating string types in the language, and using prefix literals. Completely orthogonal concerns. As I said, u'' literals were re-introduced by the Python developers themselves, in Python 3.3.
    - deleted 9y ago
      [deleted]
    - pvg 9y ago
      I don't think I am. In this thread you're repeatedly making the point that 2.7 supported Unicode and the difference is mostly technical details of things like internal representation and/or a matter of prefixes or whatnot. This just isn't true. The fundamental change is - in Python 2, strings are bags of bytes and in Python 3 strings are collections of Unicode codepoints and you need to go through an encoding to convert to and from bytes. This is a big (and necessary) deal. No amount of debating the finer points of implementation, feature retention or reintroduction, etc is going to make that difference not matter.
      - Radim 9y ago
        What I said: 1. BOTH Python 2 and Python 3 come with built-in support for BOTH bytestring and unicode (contrary to OP's claims I responded to) 2. That mixing bytestrings and unicode will fail with an explicit error since 3.0+ (a good idea IMO) 3. Unicode has a more efficient internal storage since Python 3.3+ (a neat technical detail) 4. It's good practice to be explicit about the type of literals, and write b"" and u"" always 5. That Python 2.7 doesn't support unicode is simply FUD. Can you articulate which point you're actually contesting? I'll be happy to clarify, but I'm honestly unsure what you're responding to.