6 ms·

Yes, I have a big sense of tragedy about Python 3. Python should run on something like (or maybe the actual) Erlang BEAM with lightweight isolated processes.

Yes, I have a big sense of tragedy about Python 3. Python should run on something like (or maybe the actual) Erlang BEAM with lightweight isolated processes. All my threaded Python code is written using that style anyway (threads communicating through synchronized queues) and I've almost never needed traditional shared mutable objects. Maybe completely never, but I'm not sure about a certain program any more. Added: I don't understand the downvotes. If Python 3 was going to make an incompatible departure from Python 2, they might as well have done stuff like the above, that brought real benefits. Instead they had 10+ years of pain over relatively minor changes that arguably weren't all improvements.

BeetleB 5y ago
You are likely being downvoted because most claims about the pain of a Python 3 transition are inflated/hyperbole. It took less than a day to migrate all my code to Python 3. And by "less than a day" I mean "less than 2 hours". Granted, bigger projects would take longer, but saying stuff like "10+ years of pain" is ridiculous. Probably less than 1% of projects had serious issues with the migration. We just hear of a few popular ones that had some pain and assume that was representative.
- throwaway81523 5y ago
  The entire Python community was in pain over Python 3 for 10 years, even if migrating any particular program wasn't much trouble. If you want to contest the notion that there was pain, then fine: most of the community simply ignored Python 3 for 10 years, because there was no reason until quite late in the process to worry about it. I myself never bothered migrating any of my Python 2 stuff. It might not be difficult to do so, but continuing to run it under Python 2 still works fine. If you migrated all of yours in 2 hours, you must not have had much to start with. I do use Python 3 for new stuff most of the time by now, but I keep running into little snags, like the .decode() method not working on strings, or having some (but not all) of the codecs removed from the string module so you have to use the codecs module. There's also the matter of stuff that is supposedly ported but isn't completely. For example, Beautiful Soup works nicely under py3, but it doesn't have a typeshed entry so its import needs a special annotation to stop mypy from complaining about it. The real loss with Python 3 is that it could have been so much better than it is. I remember hearing that Go expected to pick up a lot of migrating C++ users, but it got migrating Python users instead. Here's a pain story about a 2 to 3 migration though: https://dropbox.tech/application/how-we-rolled-out-one-of-the-largest-python-3-migrations-ever https://dropbox.tech/application/how-we-rolled-out-one-of-th...
  - dagenix 5y ago
    IMO, it's a good thing that decode() doesn't on strings in Python 3.
    - throwaway81523 5y ago
      There is a case to be made either way. The point is that it's yet another quirky difference that pops up sometimes, and using python3 keeps bringing up a stream of such issues. It does seem in the spirit of Python's duck typing to be able to say '616263'.decode('hex') and get 'abc', and that does work fine in py2. Try it in py3 and you get a type error. So ok, convert the string to bytes, e.g. by saying b'616263' instead of '616263'. That doesn't work either, but I'll leave the experiment to you.
      - adwn 5y ago
        So, like this? codecs.decode(b'616263', 'hex') If that's correct, what's so hard/bad about this?
  - __turbobrew__ 5y ago
    Some people don’t get that migrating large legacy code bases from 2 to 3 is not trivial. Your dropbox example is a good one: the migration took 3 years to complete. What else could dropbox have achieved with a 3 year long project if they didn’t have to migrate from 2 to 3?
    - throwaway81523 5y ago
      And Guido van Rossum himself was working at dropbox at the time, contributing to that port. Imagine how much harder it might have been without him.
  - zzzeek 5y ago
    And yet python tops the Tiobe index anyway. The python 3 transition was not that big a deal, and I migrated or assisted in the migration of many codebases some of which are as big as they get (openstack). the ten year thing especially made it really gradual, and we're all done now. Python 3 is great. Folks migrating to rust / go etc are looking at performance concerns for high volume server software, which python is simply never going to specialize in, and for which rust and go were both introduced only very recently. They didn't switch because python 3 was so disappointing.
- qstwall 5y ago
  It took several years to migrate Twisted or NumPy. The 10 years is the cumulative pain of having 30% of packages not migrated yet. It took at least 7 years to have 90% migrated.
beebmam 5y ago
I think some people are upset about the irony/sarcasm here, and many people don't appreciate ironic/sarcastic posts (but I do!)
- throwaway81523 5y ago
  I was being quite literal, not sarcastic. I really do wish that Python had switched to a BEAM like architecture at the Python 3 transition. There is no way that can possibly happen now.
slaymaker1907 5y ago
Yeah, I'd agree it's kind of stupid to use OS threads if you are going to have a GIL. It does make the implementation simpler, but it comes at tremendous cost to IO bound programs. If you are actually trying to do computationally intensive work, you should really be using multiple processes instead of threads in a language with GC. When writing a UI, moving work to a separate thread can still lag because GC will also block the UI thread. Even if you don't care about latency, having separate memory pools via separate processes often helps GC because then GC is embarrassingly parallel regardless of GC implementation.
- throwaway81523 5y ago
  The GIL is mostly a problem for cpu bound programs. GC pauses in Python really aren't much of an issue, partly because Python uses refcounting (ironically the cause of the GIL) so it usually frees stuff as it goes along, keeping pauses pretty short. Even with a real GC though, I think it's usually not much of a problem. I've written Python gui's in multi-threaded (but not too compute intensive) apps on quite slow processors by today's standards, and there wasn't much of a problem in UI responsiveness. I think it's important to remember the difference between parallelism (trying to compute faster by using multiple cpu's simultaneously) and concurrency (communicating on multiple communication channels that are sending stuff non-deterministically). The GIL gets in the way of parallelism but except in extreme cases, it doesn't interfere with concurrency.

BeetleB 5y ago

You are likely being downvoted because most claims about the pain of a Python 3 transition are inflated/hyperbole. It took less than a day to migrate all my code to Python 3. And by "less than a day" I mean "less than 2 hours". Granted, bigger projects would take longer, but saying stuff like "10+ years of pain" is ridiculous. Probably less than 1% of projects had serious issues with the migration. We just hear of a few popular ones that had some pain and assume that was representative.

throwaway81523 5y ago
The entire Python community was in pain over Python 3 for 10 years, even if migrating any particular program wasn't much trouble. If you want to contest the notion that there was pain, then fine: most of the community simply ignored Python 3 for 10 years, because there was no reason until quite late in the process to worry about it. I myself never bothered migrating any of my Python 2 stuff. It might not be difficult to do so, but continuing to run it under Python 2 still works fine. If you migrated all of yours in 2 hours, you must not have had much to start with. I do use Python 3 for new stuff most of the time by now, but I keep running into little snags, like the .decode() method not working on strings, or having some (but not all) of the codecs removed from the string module so you have to use the codecs module. There's also the matter of stuff that is supposedly ported but isn't completely. For example, Beautiful Soup works nicely under py3, but it doesn't have a typeshed entry so its import needs a special annotation to stop mypy from complaining about it. The real loss with Python 3 is that it could have been so much better than it is. I remember hearing that Go expected to pick up a lot of migrating C++ users, but it got migrating Python users instead. Here's a pain story about a 2 to 3 migration though: https://dropbox.tech/application/how-we-rolled-out-one-of-the-largest-python-3-migrations-ever https://dropbox.tech/application/how-we-rolled-out-one-of-th...
- dagenix 5y ago
  IMO, it's a good thing that decode() doesn't on strings in Python 3.
  - throwaway81523 5y ago
    There is a case to be made either way. The point is that it's yet another quirky difference that pops up sometimes, and using python3 keeps bringing up a stream of such issues. It does seem in the spirit of Python's duck typing to be able to say '616263'.decode('hex') and get 'abc', and that does work fine in py2. Try it in py3 and you get a type error. So ok, convert the string to bytes, e.g. by saying b'616263' instead of '616263'. That doesn't work either, but I'll leave the experiment to you.
    - adwn 5y ago
      So, like this? codecs.decode(b'616263', 'hex') If that's correct, what's so hard/bad about this?
- __turbobrew__ 5y ago
  Some people don’t get that migrating large legacy code bases from 2 to 3 is not trivial. Your dropbox example is a good one: the migration took 3 years to complete. What else could dropbox have achieved with a 3 year long project if they didn’t have to migrate from 2 to 3?
  - throwaway81523 5y ago
    And Guido van Rossum himself was working at dropbox at the time, contributing to that port. Imagine how much harder it might have been without him.
- zzzeek 5y ago
  And yet python tops the Tiobe index anyway. The python 3 transition was not that big a deal, and I migrated or assisted in the migration of many codebases some of which are as big as they get (openstack). the ten year thing especially made it really gradual, and we're all done now. Python 3 is great. Folks migrating to rust / go etc are looking at performance concerns for high volume server software, which python is simply never going to specialize in, and for which rust and go were both introduced only very recently. They didn't switch because python 3 was so disappointing.
qstwall 5y ago
It took several years to migrate Twisted or NumPy. The 10 years is the cumulative pain of having 30% of packages not migrated yet. It took at least 7 years to have 90% migrated.

beebmam 5y ago

I think some people are upset about the irony/sarcasm here, and many people don't appreciate ironic/sarcastic posts (but I do!)

throwaway81523 5y ago
I was being quite literal, not sarcastic. I really do wish that Python had switched to a BEAM like architecture at the Python 3 transition. There is no way that can possibly happen now.

slaymaker1907 5y ago

Yeah, I'd agree it's kind of stupid to use OS threads if you are going to have a GIL. It does make the implementation simpler, but it comes at tremendous cost to IO bound programs. If you are actually trying to do computationally intensive work, you should really be using multiple processes instead of threads in a language with GC. When writing a UI, moving work to a separate thread can still lag because GC will also block the UI thread. Even if you don't care about latency, having separate memory pools via separate processes often helps GC because then GC is embarrassingly parallel regardless of GC implementation.

throwaway81523 5y ago
The GIL is mostly a problem for cpu bound programs. GC pauses in Python really aren't much of an issue, partly because Python uses refcounting (ironically the cause of the GIL) so it usually frees stuff as it goes along, keeping pauses pretty short. Even with a real GC though, I think it's usually not much of a problem. I've written Python gui's in multi-threaded (but not too compute intensive) apps on quite slow processors by today's standards, and there wasn't much of a problem in UI responsiveness. I think it's important to remember the difference between parallelism (trying to compute faster by using multiple cpu's simultaneously) and concurrency (communicating on multiple communication channels that are sending stuff non-deterministically). The GIL gets in the way of parallelism but except in extreme cases, it doesn't interfere with concurrency.