6 ms·

I'm impressed how much Python seems stuck on older versions. What went wrong?

by xutopia 9y ago

rjgonza 9y ago
Python 3 came about with a split in the community.
- vapemaster 9y ago
  what is the split that has to do with 2vs3? Really it just seems like some devs are still griping about fairly minor backwards compatibility issues. the whole print vs print() is obviously a straw man. but the major one, strings, unicode, and bytes, changes for the good in py3. the fact that people have to go back and "un-hack" their code is time, effort, and more commits but it's for the best. <rant> i think the real split is between CPython + C-Extensions and the JIT (PyPy,Numba) gang for heavy lifting in python. I for one wish Guido would embrace the effort of integrating the GILectomy into CPython and make python thread parrallism real for CPU bound work as opposed to the high overhead multiprocessing approach. Also, including mypy in the stdlib to make genuine optional static typing would make python a defacto standard for much of what it does now without the "we love python but...." production code pain points. </rant>
  - Jach 9y ago
    I don't think the whole print vs print() problem is a straw man (or similarly the division operator switch) apart from being relatively easy to fix or avoid by importing from __future__, but crucial in understanding the attitude the language owners have for its users. They could have kept print and introduced printf(), but they didn't. They didn't even get rid of the statement vs. expression distinction, which would have been a compelling reason to break print (and would give us real lambdas). The split you mention is real, but it's existed even longer than the 2vs3 split. All the various alternate Python implementations that have come and gone or are still around with the exception of Numba were started pre-3k. I wouldn't even argue that upstream was necessarily wrong not to prioritize performance and more specific use cases (like certain production uses, or scientific uses (a lot of begging to finally get the @ operator)), it just didn't matter as much for a long time. But things are different now. There are many other very expressive and much faster (either dynamically typed or static) languages in competition that lessen Python's effectiveness at reducing dev time in exchange for more hardware. Without a more serious focus on performance, Python will be driven only by momentum. That can last a long time, and is essentially the end point anyway when performance is "good enough" compared to the close alternatives, but it's not a great look when there's still much that could be done.
  - dom0 9y ago
    So, you see, I know a bit about writing efficient code. I have held a firm belief, and still do, that if you're doing heavy lifting in CPython, you're doing it wrong, and better parallelism will not significantly help you.
SloopJon 9y ago
One of the chapters added in the second edition of Peopleware quotes from a presentation by Steve McMenamin: "People hate change ... and that’s because people hate change ... I want to be sure that you get my point. People really hate change. They really, really do."
- coldtea 9y ago
  And still tons of projects managed to get people on board. Change has to be compelling. Python 3 wasn't until at least recently (3.5 and on). Change has to be backwards compatible as possible. Python 3 wasn't, often for BS reasons. Change better has some major selling feature. Python 3 has nothing that special over 2. Change must come with nice conversion tools. 2to3 wasn't that. etc...
ajross 9y ago
It wasn't broke and they fixed it. Python 2 is a mature language with a robust ecosystem. Those just don't go away no matter how much developers nag people to upgrade.
- baq 9y ago
  Except it was, and badly. Unicode support reached sanity matching Java or C#, if only just, but for me the most important fix was making exceptions actually work instead of being a broken pile of workarounds glued together with ambiguous and/or confusing syntax.
  - ajross 9y ago
    For some definition of "badly" that doesn't include a huge community of working software I guess. My point wasn't that Python 3 isn't better (or even "better enough") it's that no matter how much better Your Favorite Different Thing is, it will never be better enough to make people stop using their working software.
Jach 9y ago
Because after nearly 10 years this mediocre list is the best they can offer as consolidation for breaking your code, and for much of that time they didn't have everything it does now. If py3k shipped with something like https://gregoryszorc.com/blog/2017/03/13/from-__past__-import-bytes_literals/ https://gregoryszorc.com/blog/2017/03/13/from-__past__-impor... there probably would have been more uptake.
- mixmastamyk 9y ago
  This piece barely scratches the surface, there's formatted string literals, type checking available, and compact ordered dicts by default in the latest. All awesomeness. I'm glad they did the "mediocre" fixes as well, they lead to a lot fewer encoding (and other) bugs, which were a real problem on nontrivial programs.
  - deleted 9y ago
    [deleted]
  - Jach 9y ago
    Type checking is given in the piece. Compact dicts are nice for memory performance, but I don't use Python for performance. Ordered dicts I only care about rarely and explicitly use one in such cases. Formatted string literals have not really been a "feature" but a concern. The original plan was to get rid of the % way of formatting and just use .format with __format__ (except now there are pitfalls and inconsistencies with different types of strings). Now Python 3.6 has string interpolation with f-strings, which is kind of nice, but I've never really missed it when I come back from using other languages with it, and if I really wanted it there has been tools like Interpy. That last bit is kind of the nail in the coffin, a lot of Py3 features have been available as Py2 libraries or higher source parsing tools longer than the feature has been in Py3. I didn't like how the changes to strings and bytes were made, but that's probably a larger discussion, and fits within how Py3 has evolved. If anyone is really confused by the struggle with adoption, just look through the release notes of version by version, and ask at what point there's a compelling reason to upgrade. I still don't see one, but at this point, nearly 10 years after Py3k, Py3.6 is finally usable enough that I wouldn't mind as much if I had to work on a Py3.6 codebase instead of a Py2.7 one.
problems 9y ago
It was source-level incompatible with older code so it couldn't be used as a smooth transition like... almost any other language upgrade I've ever seen. Breaking code like this was a big mistake in my opinion - it resulted in many people sticking on the old version for code and library compatibility. There are still libraries which don't work on Python 3, though now fairly few of them. In addition to that, the unicode changes were in my opinion another mistake - they made it far more complicated to deal with strings and byte arrays as now even for the simplest of applications you have to put thought into character encodings. Another annoyance for me personally is that I often use python to do things like hex and base64 encoding, the removal of .encode("hex") really wrecked this one for me. Now I have to remember weird import libraries like "binascii" with weird function names like "hexlify". Ultimately Python 2 works. And it works well, there just aren't that many compelling reasons to move to 3 given that it often is more complicated and breaks existing code. They have a few cool things now, so I sometimes use it for new code but there's still nothing that's making me want to delve into a port of my older codebases.
- maxerickson 9y ago
  `hex` and `fromhex` were added to the bytes type in Python 3.5. And while it isn't trivial, it is easy enough to just deal with bytes in 3.x.
  - problems 9y ago
    Nice, do they have similar for base64 too?
    - dom0 9y ago
      import base64
      - problems 9y ago
        Yeah, I know about that, but that's still longer and more annoying than the old way. Maybe I should just make my own library that just modifies strings and byte arrays and adds a few things for the common binary operations I do.
gh02t 9y ago
A combination of Unicode strings breaking a lot of libraries and Py3K not having a lot of user-visible improvements early in its lifetime. It built up a base of people who said "they broke all my code and I don't see any reason to fix it". A lot of people recommended (legitimately) that you stick with Python 2 until Py3K was more widely supported. The momentum from this group persisted even after the point where it made sense to switch to 3. And so we are stuck, especially in enterprise where it could cost a lot of time and money to port because the transition wasn't managed well, both by users and the Python team.
seanp2k2 9y ago
See also: IPv4, Windows XP, Java 7, Java 6, Flash, RHEL/Cent 5, VB6, IE6 People just can't be bothered to upgrade when it works well enough for them.

I'm impressed how much Python seems stuck on older versions. What went wrong?

rjgonza 9y ago

Python 3 came about with a split in the community.

vapemaster 9y ago
what is the split that has to do with 2vs3? Really it just seems like some devs are still griping about fairly minor backwards compatibility issues. the whole print vs print() is obviously a straw man. but the major one, strings, unicode, and bytes, changes for the good in py3. the fact that people have to go back and "un-hack" their code is time, effort, and more commits but it's for the best. <rant> i think the real split is between CPython + C-Extensions and the JIT (PyPy,Numba) gang for heavy lifting in python. I for one wish Guido would embrace the effort of integrating the GILectomy into CPython and make python thread parrallism real for CPU bound work as opposed to the high overhead multiprocessing approach. Also, including mypy in the stdlib to make genuine optional static typing would make python a defacto standard for much of what it does now without the "we love python but...." production code pain points. </rant>
- Jach 9y ago
  I don't think the whole print vs print() problem is a straw man (or similarly the division operator switch) apart from being relatively easy to fix or avoid by importing from __future__, but crucial in understanding the attitude the language owners have for its users. They could have kept print and introduced printf(), but they didn't. They didn't even get rid of the statement vs. expression distinction, which would have been a compelling reason to break print (and would give us real lambdas). The split you mention is real, but it's existed even longer than the 2vs3 split. All the various alternate Python implementations that have come and gone or are still around with the exception of Numba were started pre-3k. I wouldn't even argue that upstream was necessarily wrong not to prioritize performance and more specific use cases (like certain production uses, or scientific uses (a lot of begging to finally get the @ operator)), it just didn't matter as much for a long time. But things are different now. There are many other very expressive and much faster (either dynamically typed or static) languages in competition that lessen Python's effectiveness at reducing dev time in exchange for more hardware. Without a more serious focus on performance, Python will be driven only by momentum. That can last a long time, and is essentially the end point anyway when performance is "good enough" compared to the close alternatives, but it's not a great look when there's still much that could be done.
- dom0 9y ago
  So, you see, I know a bit about writing efficient code. I have held a firm belief, and still do, that if you're doing heavy lifting in CPython, you're doing it wrong, and better parallelism will not significantly help you.

SloopJon 9y ago

One of the chapters added in the second edition of Peopleware quotes from a presentation by Steve McMenamin: "People hate change ... and that’s because people hate change ... I want to be sure that you get my point. People really hate change. They really, really do."

coldtea 9y ago
And still tons of projects managed to get people on board. Change has to be compelling. Python 3 wasn't until at least recently (3.5 and on). Change has to be backwards compatible as possible. Python 3 wasn't, often for BS reasons. Change better has some major selling feature. Python 3 has nothing that special over 2. Change must come with nice conversion tools. 2to3 wasn't that. etc...

ajross 9y ago

It wasn't broke and they fixed it. Python 2 is a mature language with a robust ecosystem. Those just don't go away no matter how much developers nag people to upgrade.

baq 9y ago
Except it was, and badly. Unicode support reached sanity matching Java or C#, if only just, but for me the most important fix was making exceptions actually work instead of being a broken pile of workarounds glued together with ambiguous and/or confusing syntax.
- ajross 9y ago
  For some definition of "badly" that doesn't include a huge community of working software I guess. My point wasn't that Python 3 isn't better (or even "better enough") it's that no matter how much better Your Favorite Different Thing is, it will never be better enough to make people stop using their working software.

Jach 9y ago

Because after nearly 10 years this mediocre list is the best they can offer as consolidation for breaking your code, and for much of that time they didn't have everything it does now. If py3k shipped with something like https://gregoryszorc.com/blog/2017/03/13/from-__past__-import-bytes_literals/ https://gregoryszorc.com/blog/2017/03/13/from-__past__-impor... there probably would have been more uptake.

mixmastamyk 9y ago
This piece barely scratches the surface, there's formatted string literals, type checking available, and compact ordered dicts by default in the latest. All awesomeness. I'm glad they did the "mediocre" fixes as well, they lead to a lot fewer encoding (and other) bugs, which were a real problem on nontrivial programs.
- deleted 9y ago
  [deleted]
- Jach 9y ago
  Type checking is given in the piece. Compact dicts are nice for memory performance, but I don't use Python for performance. Ordered dicts I only care about rarely and explicitly use one in such cases. Formatted string literals have not really been a "feature" but a concern. The original plan was to get rid of the % way of formatting and just use .format with __format__ (except now there are pitfalls and inconsistencies with different types of strings). Now Python 3.6 has string interpolation with f-strings, which is kind of nice, but I've never really missed it when I come back from using other languages with it, and if I really wanted it there has been tools like Interpy. That last bit is kind of the nail in the coffin, a lot of Py3 features have been available as Py2 libraries or higher source parsing tools longer than the feature has been in Py3. I didn't like how the changes to strings and bytes were made, but that's probably a larger discussion, and fits within how Py3 has evolved. If anyone is really confused by the struggle with adoption, just look through the release notes of version by version, and ask at what point there's a compelling reason to upgrade. I still don't see one, but at this point, nearly 10 years after Py3k, Py3.6 is finally usable enough that I wouldn't mind as much if I had to work on a Py3.6 codebase instead of a Py2.7 one.

problems 9y ago

It was source-level incompatible with older code so it couldn't be used as a smooth transition like... almost any other language upgrade I've ever seen. Breaking code like this was a big mistake in my opinion - it resulted in many people sticking on the old version for code and library compatibility. There are still libraries which don't work on Python 3, though now fairly few of them. In addition to that, the unicode changes were in my opinion another mistake - they made it far more complicated to deal with strings and byte arrays as now even for the simplest of applications you have to put thought into character encodings. Another annoyance for me personally is that I often use python to do things like hex and base64 encoding, the removal of .encode("hex") really wrecked this one for me. Now I have to remember weird import libraries like "binascii" with weird function names like "hexlify". Ultimately Python 2 works. And it works well, there just aren't that many compelling reasons to move to 3 given that it often is more complicated and breaks existing code. They have a few cool things now, so I sometimes use it for new code but there's still nothing that's making me want to delve into a port of my older codebases.

maxerickson 9y ago
`hex` and `fromhex` were added to the bytes type in Python 3.5. And while it isn't trivial, it is easy enough to just deal with bytes in 3.x.
- problems 9y ago
  Nice, do they have similar for base64 too?
  - dom0 9y ago
    import base64
    - problems 9y ago
      Yeah, I know about that, but that's still longer and more annoying than the old way. Maybe I should just make my own library that just modifies strings and byte arrays and adds a few things for the common binary operations I do.

gh02t 9y ago

A combination of Unicode strings breaking a lot of libraries and Py3K not having a lot of user-visible improvements early in its lifetime. It built up a base of people who said "they broke all my code and I don't see any reason to fix it". A lot of people recommended (legitimately) that you stick with Python 2 until Py3K was more widely supported. The momentum from this group persisted even after the point where it made sense to switch to 3. And so we are stuck, especially in enterprise where it could cost a lot of time and money to port because the transition wasn't managed well, both by users and the Python team.

seanp2k2 9y ago

See also: IPv4, Windows XP, Java 7, Java 6, Flash, RHEL/Cent 5, VB6, IE6 People just can't be bothered to upgrade when it works well enough for them.