9 ms·

I like Python 3. It eliminated a whole category of encoding/decoding errors. Even Python 2 codebases benefited from it, as libraries were updated to handle Unic

by Flimm 4y ago

I like Python 3. It eliminated a whole category of encoding/decoding errors. Even Python 2 codebases benefited from it, as libraries were updated to handle Unicode better in an effort to achieve compatibility with Python 3. I didn't experience much pain migrating codebases to it, but I'm just speaking for myself here. Congratulations to Debian on upgrading to Python 3!

thrdbndndn 4y ago
Yeah, coming from C#, Python 2's unicode support was so bad I almost abandoned it immediately as a Chinese speaker (and to make it worse, I use Windows). You literally can't use IDLE for learning/testing properly half of time due to encoding issues. And what surprised me most is that every time I mentioned this, there would be lots people telling me how this is a superior design because you can operate string like bytes. I mean, it of course has its upside, but I don't think it's worth it if you care even slightly beyond ASCII.
- mrweasel 4y ago
  We had crazy amounts of code handling unicode support and conversion from our ecommerce site to our ERP system (running on Windows using some Windows code page thing). With Python 3 all that went away, you can now just seamlessly parse text from one system to another. For me, the unicode handling alone was worth the time spend migrating from Python 2. That was a decade ago, to finding that the "python" command still launches a Python 2.7 interpreter in 2023 is just beyond belief. Personally I feel like they should have yanked Python 2 in Jessie (Debian 8) in 2015, more realistically in Stretch in 2017.
  - Jenda_ 4y ago
    > Personally I feel like they should have yanked Python 2 in Jessie (Debian 8) in 2015, more realistically in Stretch in 2017. For example GnuRadio started supporting Python 3 with GnuRadio 3.8 released in 2019, and then you had to port all your programs using it to this version. So no, in 2017, the ecosystem was not ready.
  - mike_hock 4y ago
    > finding that the "python" command still launches a Python 2.7 apt install python-is-python3
    - mrweasel 4y ago
      I don't think my co-workers would like that :-)
      - tux3 4y ago
        dpkg-reconfigure co-workers?
        bb88 4y ago
        apt purge co-workers
    - worldsayshi 4y ago
      I've broken a bunch of stuff when I tried to replace python 2 with 3.
  - Nextgrid 4y ago
    The biggest danger in Python 2's unicode handling is that incorrect things somewhat worked (until you got a non-ascii character at which point it exploded or produced incorrect behaviour). I'm sure you could do things well in Python 2 with proper combinations of encode/decode, but it wasn't obvious where you even needed those because with ascii text, things "just worked" transparently. With Python 3 it's very obvious where you need encoding/decoding because bytes != str.
    - vetinari 4y ago
      You could do things correctly in Python 2, but as soon as you used any third-party library in your project, chances were it is going to explode underneath you anyway. In the early 2000s, I maintained an py2, wxPython app with the users having the system encoding win-1250; the effort to patch this was unbelievable. The migration to python3-style handling forced everyone to think about these issues, not just few people for which things were crashing. Even just popularizing the issue was great, until then, many maintainers of third-party libraries didn't even understand what is the problem that you want to "needlessly complicated" fix in their libs.
  - heinrichhartman 4y ago
    > That was a decade ago, to finding that the "python" command still launches a Python 2.7 interpreter in 2023 is just beyond belief. The problem is not the end-user invoking the command. The problem is scrips expecting `#!/usr/bin/env python` to invoke python-2.
    - deleted 4y ago
      [deleted]
- dannymi 4y ago
  FWIW, in Linux, this problem does not exist. Everything is UTF-8 and Python 2 would work just fine (and always did). In order to support Windows better, Python 3 introduced support for UCS-4 (or worse, UTF-16) strings (depending on a compilation setting when Python was compiled) and they had to introduce extra string types to distinguish readable strings from binary strings ("bytes"). These extra types made Python 3 a lot harder to teach (I teach 30 person classes every year). So it's not all roses now. In the end, I got used to it, BUT I just gave up asking encode()/decode() questions at the exams. Very few people understand it, or care enough (and I understand why--it's a ridiculous thing to have). You only need it if your OS somehow slept through the introduction of UTF-8, which is backward compatible with ASCII, resilient even if there are transfer errors and can encode all unicode characters. Encoding problem used to be really common in UNIX (and before that, in mainframes), but with the introduction of UTF-8, all encoding problems I had vanished and never appeared again. Even Windows 10 has an UTF-8 mode now and the Windows API functions that end in "A" can be made to use UTF-8. Now, in a sense, Python 3 has this entire complication for no reason. That said, Python 3 is ok to use now--and, conceptually, distinguishing byte strings from unicode strings is better (for example so that you don't accidentially print the former to the terminal). It just uses up brain cycles that you could be using for solving your actual problems.
  - thrdbndndn 4y ago
    > I just gave up asking encode()/decode() questions at the exams. Very few people understand it, or care enough (and I understand why--it's a ridiculous thing to have). I get it from the "pass the exam" perspective, since that's one more thing to worry about. But from my experience in teaching others, doing the conversion between bytes and string implicitly (à la Python 2's way) hinders actual understanding of this very important concept, and it's quite harmful in further study. Bytes should be considered as a separate, more low-evel thing, away from int/float/strings; at the very least, it should be considered as bits/hex numbers. If you want strings, you explicitly encode/decode them in a way, even if everything is UTF-8. On top of that, "byte string" is just a confusing concept. It might works for English speaker (by "it's a ridiculous thing to have" I assume you mean that, "'english'.encode() is just b'english', why bother?"), not at all for Chinese speakers, even in UTF-8. There is no b'中文' -- only b'\xe4\xb8\xad\xe6\x96\x87' which has zero meanings in their own. And even from an easy-to-use perspective: most people don't even work on bytes often nowadays. A more abstract "string" type is all they need, without worrying about how it works under the hood (and if they do, they need to understand how encode/decode works properly anyway).
ilyt 4y ago
> Congratulations to Debian on upgrading to Python 3! Debian had Py3 for ages. It wasn't upgrade, that happened ages ago, just removing old packages kept for compatibility
mort96 4y ago
Honestly, I just found Python 3 to introduce a whole bunch of complexity when working with text. Why can't it just be a byte buffer? Why must you complicate interacting with the OS so much? Text is just a buffer of mostly UTF-8 encoded bytes, why make it needlessly hard?

by Flimm 4y ago

thrdbndndn 4y ago

Yeah, coming from C#, Python 2's unicode support was so bad I almost abandoned it immediately as a Chinese speaker (and to make it worse, I use Windows). You literally can't use IDLE for learning/testing properly half of time due to encoding issues. And what surprised me most is that every time I mentioned this, there would be lots people telling me how this is a superior design because you can operate string like bytes. I mean, it of course has its upside, but I don't think it's worth it if you care even slightly beyond ASCII.

mrweasel 4y ago
We had crazy amounts of code handling unicode support and conversion from our ecommerce site to our ERP system (running on Windows using some Windows code page thing). With Python 3 all that went away, you can now just seamlessly parse text from one system to another. For me, the unicode handling alone was worth the time spend migrating from Python 2. That was a decade ago, to finding that the "python" command still launches a Python 2.7 interpreter in 2023 is just beyond belief. Personally I feel like they should have yanked Python 2 in Jessie (Debian 8) in 2015, more realistically in Stretch in 2017.
- Jenda_ 4y ago
  > Personally I feel like they should have yanked Python 2 in Jessie (Debian 8) in 2015, more realistically in Stretch in 2017. For example GnuRadio started supporting Python 3 with GnuRadio 3.8 released in 2019, and then you had to port all your programs using it to this version. So no, in 2017, the ecosystem was not ready.
- mike_hock 4y ago
  > finding that the "python" command still launches a Python 2.7 apt install python-is-python3
  - mrweasel 4y ago
    I don't think my co-workers would like that :-)
    - tux3 4y ago
      dpkg-reconfigure co-workers?
      - bb88 4y ago
        apt purge co-workers
  - worldsayshi 4y ago
    I've broken a bunch of stuff when I tried to replace python 2 with 3.
- Nextgrid 4y ago
  The biggest danger in Python 2's unicode handling is that incorrect things somewhat worked (until you got a non-ascii character at which point it exploded or produced incorrect behaviour). I'm sure you could do things well in Python 2 with proper combinations of encode/decode, but it wasn't obvious where you even needed those because with ascii text, things "just worked" transparently. With Python 3 it's very obvious where you need encoding/decoding because bytes != str.
  - vetinari 4y ago
    You could do things correctly in Python 2, but as soon as you used any third-party library in your project, chances were it is going to explode underneath you anyway. In the early 2000s, I maintained an py2, wxPython app with the users having the system encoding win-1250; the effort to patch this was unbelievable. The migration to python3-style handling forced everyone to think about these issues, not just few people for which things were crashing. Even just popularizing the issue was great, until then, many maintainers of third-party libraries didn't even understand what is the problem that you want to "needlessly complicated" fix in their libs.
- heinrichhartman 4y ago
  > That was a decade ago, to finding that the "python" command still launches a Python 2.7 interpreter in 2023 is just beyond belief. The problem is not the end-user invoking the command. The problem is scrips expecting `#!/usr/bin/env python` to invoke python-2.
  - deleted 4y ago
    [deleted]
dannymi 4y ago
FWIW, in Linux, this problem does not exist. Everything is UTF-8 and Python 2 would work just fine (and always did). In order to support Windows better, Python 3 introduced support for UCS-4 (or worse, UTF-16) strings (depending on a compilation setting when Python was compiled) and they had to introduce extra string types to distinguish readable strings from binary strings ("bytes"). These extra types made Python 3 a lot harder to teach (I teach 30 person classes every year). So it's not all roses now. In the end, I got used to it, BUT I just gave up asking encode()/decode() questions at the exams. Very few people understand it, or care enough (and I understand why--it's a ridiculous thing to have). You only need it if your OS somehow slept through the introduction of UTF-8, which is backward compatible with ASCII, resilient even if there are transfer errors and can encode all unicode characters. Encoding problem used to be really common in UNIX (and before that, in mainframes), but with the introduction of UTF-8, all encoding problems I had vanished and never appeared again. Even Windows 10 has an UTF-8 mode now and the Windows API functions that end in "A" can be made to use UTF-8. Now, in a sense, Python 3 has this entire complication for no reason. That said, Python 3 is ok to use now--and, conceptually, distinguishing byte strings from unicode strings is better (for example so that you don't accidentially print the former to the terminal). It just uses up brain cycles that you could be using for solving your actual problems.
- thrdbndndn 4y ago
  > I just gave up asking encode()/decode() questions at the exams. Very few people understand it, or care enough (and I understand why--it's a ridiculous thing to have). I get it from the "pass the exam" perspective, since that's one more thing to worry about. But from my experience in teaching others, doing the conversion between bytes and string implicitly (à la Python 2's way) hinders actual understanding of this very important concept, and it's quite harmful in further study. Bytes should be considered as a separate, more low-evel thing, away from int/float/strings; at the very least, it should be considered as bits/hex numbers. If you want strings, you explicitly encode/decode them in a way, even if everything is UTF-8. On top of that, "byte string" is just a confusing concept. It might works for English speaker (by "it's a ridiculous thing to have" I assume you mean that, "'english'.encode() is just b'english', why bother?"), not at all for Chinese speakers, even in UTF-8. There is no b'中文' -- only b'\xe4\xb8\xad\xe6\x96\x87' which has zero meanings in their own. And even from an easy-to-use perspective: most people don't even work on bytes often nowadays. A more abstract "string" type is all they need, without worrying about how it works under the hood (and if they do, they need to understand how encode/decode works properly anyway).

ilyt 4y ago

> Congratulations to Debian on upgrading to Python 3! Debian had Py3 for ages. It wasn't upgrade, that happened ages ago, just removing old packages kept for compatibility

mort96 4y ago

Honestly, I just found Python 3 to introduce a whole bunch of complexity when working with text. Why can't it just be a byte buffer? Why must you complicate interacting with the OS so much? Text is just a buffer of mostly UTF-8 encoded bytes, why make it needlessly hard?