5 ms·

I've been using Python since 1.5.2. I find converting a Python 2 codebase to Python 3 fairly challenging because it cannot be done mechanically. You have to car

by js2 4y ago

I've been using Python since 1.5.2. I find converting a Python 2 codebase to Python 3 fairly challenging because it cannot be done mechanically. You have to carefully consider each string and whether it is bytes or a string. But for new Python 3 code, I do not find it painful at all. The reason for the transition (to distinct byte/string types) was that it was too easy to confuse the two in Python 2. Is your critique that you don't feel that the distinct types are needed, or that it's too difficult to convert from bytes to string and vice-versa? I'm not sure how it can be much easier: b"".decode("utf8") and "".encode("utf8") are quite clear to me. Is it just that decode/encode are hard to remember? You could use str(b"", "utf8") and bytes("", "utf8"). Careful with str(b"", "utf8") though because if you leave out the encoding you'll get back a string, but probably not the string you want.

andrewstuart 4y ago
>>> I'm not sure how it can be much easier: b"".decode("utf8") and "".encode("utf8") are quite clear to me. They look arcane and unclear to me. Which direction is encode and which is decode? What does decode and encode actually mean? There should be some hint in the function name. Here is a question for you…. for the example you give, what are all the possible permutations of every possible component of each command? What happens if you prefix encode with b? What if you don’t prefix decode with b? What other possible values can go in the brackets? Can I encode and decode all types of input?
- tremon 4y ago
  What's stopping you from finding the answers to these questions yourself? What happens if you prefix encode with b? >>> b'these are bytes'.encode('utf8') Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'bytes' object has no attribute 'encode' What if you don’t prefix decode with b? >>> 'this is a string'.decode('utf8') Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'str' object has no attribute 'decode' What other possible values can go in the brackets? Can I encode and decode all types of input? https://docs.python.org/3/library/codecs.html#standard-encodings https://docs.python.org/3/library/codecs.html#standard-encod... Proper Unicode handling is hard for developers because many prefer not to think about it at all. Do you have an example of a language that has a Unicode conversion API that does work for you?
  - andrewstuart 4y ago
    If you read the thread you'll understand.
- layer8 4y ago
  Maybe the following helps as a mnemonic: Encoding is always “transform into the cryptic format” and decoding is always “transform from the cryptic format into something I understand”. This should be natural, like “decoding a cryptic message”. Now, bytes are the cryptic thing, a binary representation (the “code”), whereas strings are the noncryptic thing (just text). Therefore you encode to bytes, and decode them to a string. It’s also analogous to encryption/decryption, and to compression/decompression. Bytes are the opaque external representation, the on-disk format. Strings are the intelligible in-program format. You decode the former into the latter. (Caveat: I don’t know any Python, but the topic of character encoding is independent from the programming language.)
- js2 4y ago
  You encode from a Unicode string to bytes and decode from bytes to a Unicode string. Remember that things like "utf8" and "ascii" are so called "encodings" of Unicode. Base64 is also an encoding (of bytes to ascii). Now the terms encode and decode may be jargon, but they are not terms that Python made up. This is standard terminology for our industry. I understand you find them arcane and unclear. I'm not trying to convince you otherwise. Which is why I suggested the alternative of using bytes("", encoding) and str(b"", encoding) as an option that you might find more clear. Maybe Python should have named these to_bytes() and to_string() instead. I can't argue against that. I'm only saying that I don't find encode/decode unclear and that they are standard industry terminology.
  - andrewstuart 4y ago
    I'm not asking you to explain it to me. You explaining does not make the commands easy. I'm addressing the point that you made that the commands are obvious, they're not. I was asking you to identify all the permutations because that is what makes those commands complex and need constant looking up.

by js2 4y ago

andrewstuart 4y ago

>>> I'm not sure how it can be much easier: b"".decode("utf8") and "".encode("utf8") are quite clear to me. They look arcane and unclear to me. Which direction is encode and which is decode? What does decode and encode actually mean? There should be some hint in the function name. Here is a question for you…. for the example you give, what are all the possible permutations of every possible component of each command? What happens if you prefix encode with b? What if you don’t prefix decode with b? What other possible values can go in the brackets? Can I encode and decode all types of input?

tremon 4y ago
What's stopping you from finding the answers to these questions yourself? What happens if you prefix encode with b? >>> b'these are bytes'.encode('utf8') Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'bytes' object has no attribute 'encode' What if you don’t prefix decode with b? >>> 'this is a string'.decode('utf8') Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'str' object has no attribute 'decode' What other possible values can go in the brackets? Can I encode and decode all types of input? https://docs.python.org/3/library/codecs.html#standard-encodings https://docs.python.org/3/library/codecs.html#standard-encod... Proper Unicode handling is hard for developers because many prefer not to think about it at all. Do you have an example of a language that has a Unicode conversion API that does work for you?
- andrewstuart 4y ago
  If you read the thread you'll understand.
layer8 4y ago
Maybe the following helps as a mnemonic: Encoding is always “transform into the cryptic format” and decoding is always “transform from the cryptic format into something I understand”. This should be natural, like “decoding a cryptic message”. Now, bytes are the cryptic thing, a binary representation (the “code”), whereas strings are the noncryptic thing (just text). Therefore you encode to bytes, and decode them to a string. It’s also analogous to encryption/decryption, and to compression/decompression. Bytes are the opaque external representation, the on-disk format. Strings are the intelligible in-program format. You decode the former into the latter. (Caveat: I don’t know any Python, but the topic of character encoding is independent from the programming language.)
js2 4y ago
You encode from a Unicode string to bytes and decode from bytes to a Unicode string. Remember that things like "utf8" and "ascii" are so called "encodings" of Unicode. Base64 is also an encoding (of bytes to ascii). Now the terms encode and decode may be jargon, but they are not terms that Python made up. This is standard terminology for our industry. I understand you find them arcane and unclear. I'm not trying to convince you otherwise. Which is why I suggested the alternative of using bytes("", encoding) and str(b"", encoding) as an option that you might find more clear. Maybe Python should have named these to_bytes() and to_string() instead. I can't argue against that. I'm only saying that I don't find encode/decode unclear and that they are standard industry terminology.
- andrewstuart 4y ago
  I'm not asking you to explain it to me. You explaining does not make the commands easy. I'm addressing the point that you made that the commands are obvious, they're not. I was asking you to identify all the permutations because that is what makes those commands complex and need constant looking up.