3 ms·

My main criticism of Python 3's changes to strings is that it has become much more specific about strings. In Python 2, if you have a series of bytes -or- a "s

My main criticism of Python 3's changes to strings is that it has become much more specific about strings. In Python 2, if you have a series of bytes -or- a "string", the language has no opinion about the encoding. It just passes around the bytes. If that set of bytes enters and exits Python without being changed, its format is of no concern. Interactions do not force you to define an encoding. This is not correct, but it is often functional. Python 3, on the other hand, if you ever treat bytes as a string, forces you to have an opinion about the encoding. Same goes for if you convert back to bytes. For uncommon or unexpected encodings, the chance of this going wrong in a casual, accidental way is much higher. Of course, the approach is more correct, but it doesn't feel more correct to the programmer.

ubernostrum 8y ago
In Python 2, if you have a series of bytes -or- a "string", the language has no opinion about the encoding This is incorrect. Python 2 lets you get away with a lot of things, but some string-y operations on Python 2 bytestrings will still trip the "need to know the encoding" issue. And absent you telling it the encoding, Python 2 will assume ASCII and begin throwing exceptions as soon as it sees a byte outside the ASCII range.
ak217 8y ago
> it doesn't feel more correct to the programmer. I agree with the details of what you said, but the insidious thing about how Python 2 organized strings and encodings is that most programmers were free to ignore it and produce buggy software. Then, later, people who had to use that software on non-ascii data would try to use it and it would blow up. This would lead to a very painful cycle of shaking out bugs that the original author may not even be motivated to fix. The decision to force encodings to be explicit and strings/bytes to be separate was a great design change. It literally made all our code more valuable by removing hidden bugs from it.
josteink 8y ago
> Python 3, on the other hand, if you ever treat bytes as a string, forces you to have an opinion about the encoding. Of course. How else can you translate the bytes into a meaningful textual representation without an encoding? Python 3 requires you to consider the real world, and that’s a good thing. (Apart from when you want to port sloppily written Python on 2.x code)
contravariant 8y ago
If you want a string that has no opinion on encoding you should just use bytes. Trying to force bytes into a string without having an opinion in its encoding is asking for problems.

ubernostrum 8y ago

In Python 2, if you have a series of bytes -or- a "string", the language has no opinion about the encoding This is incorrect. Python 2 lets you get away with a lot of things, but some string-y operations on Python 2 bytestrings will still trip the "need to know the encoding" issue. And absent you telling it the encoding, Python 2 will assume ASCII and begin throwing exceptions as soon as it sees a byte outside the ASCII range.

ak217 8y ago

> it doesn't feel more correct to the programmer. I agree with the details of what you said, but the insidious thing about how Python 2 organized strings and encodings is that most programmers were free to ignore it and produce buggy software. Then, later, people who had to use that software on non-ascii data would try to use it and it would blow up. This would lead to a very painful cycle of shaking out bugs that the original author may not even be motivated to fix. The decision to force encodings to be explicit and strings/bytes to be separate was a great design change. It literally made all our code more valuable by removing hidden bugs from it.

josteink 8y ago

> Python 3, on the other hand, if you ever treat bytes as a string, forces you to have an opinion about the encoding. Of course. How else can you translate the bytes into a meaningful textual representation without an encoding? Python 3 requires you to consider the real world, and that’s a good thing. (Apart from when you want to port sloppily written Python on 2.x code)

contravariant 8y ago

If you want a string that has no opinion on encoding you should just use bytes. Trying to force bytes into a string without having an opinion in its encoding is asking for problems.