7 ms·

It’s also about (damn) time. I know it shouldn’t, but the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the la

by zionic 5y ago

It’s also about (damn) time. I know it shouldn’t, but the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language. Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech? If 2.X -> 3.X was such a drama fest what’s gonna happen next time they need to upgrade? Etc

mikewarot 5y ago
They forced incompatible changes on their users, all in the name of a Unicode fetish. They burned their credibility. The users acted rationally, it's the developers of Python who were abusive.
- Spivak 5y ago
  I think it’s less of a fetish and more of a “we made a huge mistake in the initial design of the language and confused text and the encoding of that text making a giant footgun that was hard to avoid. The migration was difficult because you had to actually think about this stuff instead of it working by accident and living as a landmine if you encountered non UTF-8 text. The fact that you actually have to decode your bytes and specify an encoding makes it really obvious when you’ve just assumed the world is UTF-8 without really backing it up.
  - ant6n 5y ago
    What I don't get is it they break the language anyway, why they didn't fix classes. Most of python is pretty beautiful, but classes are so ugly, even basics like what is the Syntax for instance vs static cars, self/this, setting up constructors, calling inherited constructors. Like come on, how is it that they are uglier than Java in this regard.
    - mattgreenrocks 5y ago
      super() is a little cleaner to call now at least. The print() change drives me nuts: many scripts use the old form, and Python forces you to change it despite recognizing the old functionality. They could've simply allowed print expr to work as is and accepted it as a wart. The async stuff feels a bit like a bridge too far for me WRT the language. Simplicity would be standardizing on a function color (as it were). I don't envy their position of preserving backwards compat while evolving the language.
      - dragonwriter 5y ago
        > They could've simply allowed print expr to work as is and accepted it as a wart. Old print isn't an expr at all, that was a major point of the change.
        olliej 5y ago
        I think the person you're replying to was saying that there was no reason to remove the statement form. The only thing that removing the statement form of print did was break existing code. You could easily allow print to act as a function by removing it as a reserved word in expression contexts. They made print a function out of a misplaced sense of language "purity", where they made a random ideal and broke code to support it. Let's compare to JS: 20 years of new features and old code still works. Keywords were added, and old code still works. Even code that uses those keywords still works. It took more effort from the language implementers, but that's as it should be: it took a bit more effort for a few teams (who make the engines), instead of a large amount of effort for tens of thousands of groups.
    - loeg 5y ago
      They made substantial changes to classes in Python3.
    - franga2000 5y ago
      I really don't know what you're talking about. I've written a lot of object-oriented code in Python and Java and there isn't a single thing I like better in Java (that doesn't come from the typing system). I don't know about 3.0, but it's been great for years. Take a look at some Django code.
nvusuvu 5y ago
Inertia seems to best describe the hesitancy to change. The language improvements stand on their own I would think.
- samhw 5y ago
  > Inertia seems to best describe the hesitancy to change. I'd have gone for 'rigor mortis', but hey, we can't agree on everything ... like, say, a very minor change to string encoding with a 12-year managed rollout.
  - orangecat 5y ago
    like, say, a very minor change to string encoding with a 12-year managed rollout That's exactly the point: it required a massive amount of effort for relatively minor benefits.
- DannyBee 5y ago
  That's the problem - they literally do not. Technically better is not a good enough reason to change something. It has to provide some actual greater user value along axes that matter to users, and make the cost worth it. What you are seeing is that python3, well, doesn't. The options in that case are reduce cost or improve along axes that matter enough that cost is worth it.
fault1 5y ago
> Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech? What you say was bad about it? And who were the bad people specifically? The people who were using python 2 or python 3? For what it's worth, python3 >= 3.0 && python <= 3.2 were hideously broken in their unicode support. Arguably had worse/unusable uncode relative to python 2.6 or 2.7. So there was a huge failure to launch type of problem, especially given how long python3 had been development. It most definitely left a very sour taste in many people's mouth that didn't start dissipating till 3.5 or 3.6 when enough "killer" features had accumulated. Even then, for a lot of usages, python 2.7 'just works'.
Alex3917 5y ago
> the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language. There weren't that many people who were outright against Python 3, other than Zed Shaw for a few years. It just took until Python 3.4 for the language to really be usable in production, and after that it took a couple more years for every library to be updated. But the community has been pretty unified for 5+ years at this point.
- ssully 5y ago
  This is my experience as well. There were plenty of people online vocally against Python3, but once we hit 3.4 I never worked with anyone IRL that was against it. The mindset was almost always "this is the way forward", so that's how we moved with our projects.
- lanstin 5y ago
  If by unified you mean no longer arguing, sure I switched from Python 2 to Go lang. (I will concede if I am doing exploratory stuff, I will start new repos in Python 3, e.g. for boto quick exploration.) I am still bitter about the fine Python 2 code I supported and extended that is now all go or Java.
- 3pt14159 5y ago
  I used it production just fine before that. What didn't work for you? The unicode support was shoddy, but lots of other stuff worked fine. The problem as I remember it was that there were some critical tools that just were not making the switch. The last one I could remember having to have a separate runtime for was graph-tool, but there were just a ton of them in the scientific computing community. Also, I don't think making print a function was worth it. So many people had muscle memory on that that personally I think it was worth just single casing it.
  - loudthing 5y ago
  - Alex3917 5y ago
roywashere 5y ago
My company started life in about 2014 and for python is only on 3. And python 3 is great. Having written python 2 before and also perl, I really appreciate UTF-8 just works. And I hope that python learned from 2 -> 3 and that in the future it will be better
rcarmo 5y ago
I saw the drama and ignored it. Transition was seamless for 99% of the stuff I used, and the 1% I left behind I don't miss.
DannyBee 5y ago
The fact that you think this way shows why that community is so broken about things like this. You think the user community behaved badly? It was the developers. For the vast majority of companies, python 3.x is not better enough for the cost. That is what happened. You may think it's awesome. It may be technically better it, etc. But the cost of moving was much more for most companies than the productivity gains. We've even measured it before. That is entirely on the python folks for not providing something people want enough to move to. All CSAT surveys of python i've seen at my company, and that my counterparts had ever seen, said literally the same thing. So it seems like something the community could have discovered pretty easily in a rigorous way if they wanted. I used to spend a lot of time on python-dev. It's a group of good people and engineers for sure. There was always good rigor around technical reasons to do something, and around performance benchmarking. But i don't remember seeing a lot of productivity research or CSAT or .... It's fair to point out that most OSS communities don't do that, but that's a bad thing, not a good one, and one of the things that often leads to community trouble One of the other major factors, besides the productivity issue, that makes it not better enough is the migration cost. Software of this kind needs to be built to be migrated from and to. If not, it's broken and you deserve all the animosity you will surely get from your users. It doesn't matter if you screwed up the language, etc. Tough crap. In python, for lots of folk with substantial stacks, the amount that could be auto-migrated by the tooling provided was low - 40-50%. Yes, simple stuff could be auto-migrated, but not most real stuff. We actually ended up building our own tooling to get closer to 70% because how expensive the overall thing. It processed many millions of lines of python code, automatically kept track of which things would be best to rewrite/finish migrating first (because it would unblock other things), and continuously tried to auto-migrate any non-migrated changed code in a loop to maximize automation. The total engineer cost of moving was enormous. We are talking well over a thousand SWE years. For what, exactly? As I said, we already measured (because plenty of new projects were python-3 only) and determined that all the improvements and hullaboo did not meaningfully improve productivity for people. Not enough. It was small, at best. It also didn't reduce the rate of production bugs, etc. So all this work, for no meaningful value. Because not enough were willing to do it, the result was to force it. Damn the users, full steam ahead. The users are just lazy. I talked with lots of counterparts at other companies - most felt the same way. The result for us is lots of teams gave up python forever. The only reason it's growing at all is because of ML. IMHO, python2->3 is an object lesson in doing it wrong and then victim blaming.
closeparen 5y ago
"Bizarre behavior" like not immediately throwing away hundreds of thousands of hours of work product because you said so?
StewardMcOy 5y ago
While I happily use Python 3 now, the transition was very difficult, and I was grumpy about it at the time. I work with code that uses a lot of files in a lot of different text encodings. Some are XML, some plain text, and some binary. Coming from other languages, Python 2's Unicode support was very difficult to work with, and my team was excited to move to Python 3, until we actually had to do the work. Long story short, Python 3's str/bytes separation was a nightmare, especially when having to deal with third-party libraries that were expecting one or the other. This was especially true for libraries that came from Python 2, and were still expecting strs as function parameters when they should have switched to bytes. There was so much encode() and decode()ing going on that we occasionally caught ourselves getting them backwards in code reviews. We took a step back to see if we were just architecting things poorly, but no, we were doing things the best we could for our problem domain. In the end, we traded one set of Python 2 text encoding footguns for a mostly different set of Python 3 text encoding footguns. It's not like Python 2 was great. Having a single str type instead of the str/bytes separation is worse in theory. But because Python 3 didn't design the separation of those types well enough, and still has a ton of text encoding footguns. It wasn't better enough to really justify all the work that went into the conversion.
- supergarfield 5y ago
  What do you think the current text encoding footguns are? In a different direction, I don't know what your problem domain is/was, but in general when I'm dealing with UTF8, I don't need to convert back to bytes very often. Was the need for conversion mostly due to the libraries that still expected strings instead of bytes?
  - StewardMcOy 5y ago
    It's been quite a few years since we went through the conversion, and at this point, working in Python 3 is natural to me, so I may not be able to recall all the footguns. I can say a lot of the difficulty was due to libraries, both third-party and standard, and that hasn't improved very much. I don't want to single anyone out here, because it's pervasive. In Pytuon 2, str was the bag of bytes type. I think a lot of libraries didn't want to change to accepting bytes types instead, because it broke API compatibility, but it caused a lot of issues. I should also say that we were working with files in tons of encodings, not just UTF-8. We had UTF-16 and UTF-32, both little and big endian, with and without BOMs, but we also had S-JIS and a bunch of legacy 8-bit encodings. Often we wouldn't know what encoding a file was in, so we'd have to use the chardet library, along with some home-grown heuristics to guess. Off the top of my head, the two biggest footguns are: - There should be no way to read or write the contents of a file into a str without specifying an encoding. locale.getpreferredencoding() is a mistake. File operations should be on bytes only, or require an explicit encoding. - .encode() and .decode() are very poorly named for what they do, and it wasn't that uncommon that someone would get them backwards. Sometimes, exceptions aren't even thrown for getting them wrong, you just get incorrect data. Both of which were still issues with Python 2. There's a valid architectural argument to be had between the Python 2 way, where str was a bag of bytes, and the unicode type was for decoded bytes, and the Python 3 way, where the bytes type is your bag of bytes and str holds your decoded string. I favor Python 3's way of doing it, but it's almost six of one, half a dozen of the other. The advantages of one over the other are slight, and given how many library functions relied on the old behavior, it was probably a mistake to change it like that, rather than continuing the Python 2 way, and fixing issues like those above that caused problems.
janto 5y ago
If the reactions seem bizarre, maybe you're missing something. For me it was more than "just the tech". Python 3.x was not the Python 3000 I was promised for a long time: many of the additions did not warrant breaking changes (as demonstrated by the backports) and many of Python's deeper warts still remain unaddressed, even today. I think the deeper issue is that at the rate that 2.x got popular, the original motivation, principles and process behind the language's development got diluted. The orchestration of the push for 3.x adoption was not technically motivated in a convincing manner and signaled a change in the process. I stopped trusting the process.
shoo 5y ago
I did a bit of contract work for a place with over a million lines of custom python 2.x scripts that ran inside of a python interpreter embedded in a proprietary product which made it difficult to do things like write and execute unit tests for the code. I think they were still writing new python 2.x code in 2018. A lot of the scripts were supporting processes that had a finite lifespan during the initial stages when the company infrastructure was being built out, rather than ongoing operational processes that needed to be performed indefinitely, so hopefully they'll be able to set a lot of that codebase on fire instead of maintaining or porting it to python 3.

by zionic 5y ago

mikewarot 5y ago

They forced incompatible changes on their users, all in the name of a Unicode fetish. They burned their credibility. The users acted rationally, it's the developers of Python who were abusive.

Spivak 5y ago
I think it’s less of a fetish and more of a “we made a huge mistake in the initial design of the language and confused text and the encoding of that text making a giant footgun that was hard to avoid. The migration was difficult because you had to actually think about this stuff instead of it working by accident and living as a landmine if you encountered non UTF-8 text. The fact that you actually have to decode your bytes and specify an encoding makes it really obvious when you’ve just assumed the world is UTF-8 without really backing it up.
- ant6n 5y ago
  What I don't get is it they break the language anyway, why they didn't fix classes. Most of python is pretty beautiful, but classes are so ugly, even basics like what is the Syntax for instance vs static cars, self/this, setting up constructors, calling inherited constructors. Like come on, how is it that they are uglier than Java in this regard.
  - mattgreenrocks 5y ago
    super() is a little cleaner to call now at least. The print() change drives me nuts: many scripts use the old form, and Python forces you to change it despite recognizing the old functionality. They could've simply allowed print expr to work as is and accepted it as a wart. The async stuff feels a bit like a bridge too far for me WRT the language. Simplicity would be standardizing on a function color (as it were). I don't envy their position of preserving backwards compat while evolving the language.
    - dragonwriter 5y ago
      > They could've simply allowed print expr to work as is and accepted it as a wart. Old print isn't an expr at all, that was a major point of the change.
      - olliej 5y ago
        I think the person you're replying to was saying that there was no reason to remove the statement form. The only thing that removing the statement form of print did was break existing code. You could easily allow print to act as a function by removing it as a reserved word in expression contexts. They made print a function out of a misplaced sense of language "purity", where they made a random ideal and broke code to support it. Let's compare to JS: 20 years of new features and old code still works. Keywords were added, and old code still works. Even code that uses those keywords still works. It took more effort from the language implementers, but that's as it should be: it took a bit more effort for a few teams (who make the engines), instead of a large amount of effort for tens of thousands of groups.
  - loeg 5y ago
    They made substantial changes to classes in Python3.
  - franga2000 5y ago
    I really don't know what you're talking about. I've written a lot of object-oriented code in Python and Java and there isn't a single thing I like better in Java (that doesn't come from the typing system). I don't know about 3.0, but it's been great for years. Take a look at some Django code.

nvusuvu 5y ago

Inertia seems to best describe the hesitancy to change. The language improvements stand on their own I would think.

samhw 5y ago
> Inertia seems to best describe the hesitancy to change. I'd have gone for 'rigor mortis', but hey, we can't agree on everything ... like, say, a very minor change to string encoding with a 12-year managed rollout.
- orangecat 5y ago
  like, say, a very minor change to string encoding with a 12-year managed rollout That's exactly the point: it required a massive amount of effort for relatively minor benefits.
DannyBee 5y ago
That's the problem - they literally do not. Technically better is not a good enough reason to change something. It has to provide some actual greater user value along axes that matter to users, and make the cost worth it. What you are seeing is that python3, well, doesn't. The options in that case are reduce cost or improve along axes that matter enough that cost is worth it.

fault1 5y ago

> Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech? What you say was bad about it? And who were the bad people specifically? The people who were using python 2 or python 3? For what it's worth, python3 >= 3.0 && python <= 3.2 were hideously broken in their unicode support. Arguably had worse/unusable uncode relative to python 2.6 or 2.7. So there was a huge failure to launch type of problem, especially given how long python3 had been development. It most definitely left a very sour taste in many people's mouth that didn't start dissipating till 3.5 or 3.6 when enough "killer" features had accumulated. Even then, for a lot of usages, python 2.7 'just works'.

Alex3917 5y ago

> the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language. There weren't that many people who were outright against Python 3, other than Zed Shaw for a few years. It just took until Python 3.4 for the language to really be usable in production, and after that it took a couple more years for every library to be updated. But the community has been pretty unified for 5+ years at this point.

ssully 5y ago
This is my experience as well. There were plenty of people online vocally against Python3, but once we hit 3.4 I never worked with anyone IRL that was against it. The mindset was almost always "this is the way forward", so that's how we moved with our projects.
lanstin 5y ago
If by unified you mean no longer arguing, sure I switched from Python 2 to Go lang. (I will concede if I am doing exploratory stuff, I will start new repos in Python 3, e.g. for boto quick exploration.) I am still bitter about the fine Python 2 code I supported and extended that is now all go or Java.
3pt14159 5y ago
I used it production just fine before that. What didn't work for you? The unicode support was shoddy, but lots of other stuff worked fine. The problem as I remember it was that there were some critical tools that just were not making the switch. The last one I could remember having to have a separate runtime for was graph-tool, but there were just a ton of them in the scientific computing community. Also, I don't think making print a function was worth it. So many people had muscle memory on that that personally I think it was worth just single casing it.
- loudthing 5y ago
- Alex3917 5y ago

roywashere 5y ago

My company started life in about 2014 and for python is only on 3. And python 3 is great. Having written python 2 before and also perl, I really appreciate UTF-8 just works. And I hope that python learned from 2 -> 3 and that in the future it will be better

rcarmo 5y ago

I saw the drama and ignored it. Transition was seamless for 99% of the stuff I used, and the 1% I left behind I don't miss.

DannyBee 5y ago

The fact that you think this way shows why that community is so broken about things like this. You think the user community behaved badly? It was the developers. For the vast majority of companies, python 3.x is not better enough for the cost. That is what happened. You may think it's awesome. It may be technically better it, etc. But the cost of moving was much more for most companies than the productivity gains. We've even measured it before. That is entirely on the python folks for not providing something people want enough to move to. All CSAT surveys of python i've seen at my company, and that my counterparts had ever seen, said literally the same thing. So it seems like something the community could have discovered pretty easily in a rigorous way if they wanted. I used to spend a lot of time on python-dev. It's a group of good people and engineers for sure. There was always good rigor around technical reasons to do something, and around performance benchmarking. But i don't remember seeing a lot of productivity research or CSAT or .... It's fair to point out that most OSS communities don't do that, but that's a bad thing, not a good one, and one of the things that often leads to community trouble One of the other major factors, besides the productivity issue, that makes it not better enough is the migration cost. Software of this kind needs to be built to be migrated from and to. If not, it's broken and you deserve all the animosity you will surely get from your users. It doesn't matter if you screwed up the language, etc. Tough crap. In python, for lots of folk with substantial stacks, the amount that could be auto-migrated by the tooling provided was low - 40-50%. Yes, simple stuff could be auto-migrated, but not most real stuff. We actually ended up building our own tooling to get closer to 70% because how expensive the overall thing. It processed many millions of lines of python code, automatically kept track of which things would be best to rewrite/finish migrating first (because it would unblock other things), and continuously tried to auto-migrate any non-migrated changed code in a loop to maximize automation. The total engineer cost of moving was enormous. We are talking well over a thousand SWE years. For what, exactly? As I said, we already measured (because plenty of new projects were python-3 only) and determined that all the improvements and hullaboo did not meaningfully improve productivity for people. Not enough. It was small, at best. It also didn't reduce the rate of production bugs, etc. So all this work, for no meaningful value. Because not enough were willing to do it, the result was to force it. Damn the users, full steam ahead. The users are just lazy. I talked with lots of counterparts at other companies - most felt the same way. The result for us is lots of teams gave up python forever. The only reason it's growing at all is because of ML. IMHO, python2->3 is an object lesson in doing it wrong and then victim blaming.

closeparen 5y ago

"Bizarre behavior" like not immediately throwing away hundreds of thousands of hours of work product because you said so?

StewardMcOy 5y ago

While I happily use Python 3 now, the transition was very difficult, and I was grumpy about it at the time. I work with code that uses a lot of files in a lot of different text encodings. Some are XML, some plain text, and some binary. Coming from other languages, Python 2's Unicode support was very difficult to work with, and my team was excited to move to Python 3, until we actually had to do the work. Long story short, Python 3's str/bytes separation was a nightmare, especially when having to deal with third-party libraries that were expecting one or the other. This was especially true for libraries that came from Python 2, and were still expecting strs as function parameters when they should have switched to bytes. There was so much encode() and decode()ing going on that we occasionally caught ourselves getting them backwards in code reviews. We took a step back to see if we were just architecting things poorly, but no, we were doing things the best we could for our problem domain. In the end, we traded one set of Python 2 text encoding footguns for a mostly different set of Python 3 text encoding footguns. It's not like Python 2 was great. Having a single str type instead of the str/bytes separation is worse in theory. But because Python 3 didn't design the separation of those types well enough, and still has a ton of text encoding footguns. It wasn't better enough to really justify all the work that went into the conversion.

supergarfield 5y ago
What do you think the current text encoding footguns are? In a different direction, I don't know what your problem domain is/was, but in general when I'm dealing with UTF8, I don't need to convert back to bytes very often. Was the need for conversion mostly due to the libraries that still expected strings instead of bytes?
- StewardMcOy 5y ago
  It's been quite a few years since we went through the conversion, and at this point, working in Python 3 is natural to me, so I may not be able to recall all the footguns. I can say a lot of the difficulty was due to libraries, both third-party and standard, and that hasn't improved very much. I don't want to single anyone out here, because it's pervasive. In Pytuon 2, str was the bag of bytes type. I think a lot of libraries didn't want to change to accepting bytes types instead, because it broke API compatibility, but it caused a lot of issues. I should also say that we were working with files in tons of encodings, not just UTF-8. We had UTF-16 and UTF-32, both little and big endian, with and without BOMs, but we also had S-JIS and a bunch of legacy 8-bit encodings. Often we wouldn't know what encoding a file was in, so we'd have to use the chardet library, along with some home-grown heuristics to guess. Off the top of my head, the two biggest footguns are: - There should be no way to read or write the contents of a file into a str without specifying an encoding. locale.getpreferredencoding() is a mistake. File operations should be on bytes only, or require an explicit encoding. - .encode() and .decode() are very poorly named for what they do, and it wasn't that uncommon that someone would get them backwards. Sometimes, exceptions aren't even thrown for getting them wrong, you just get incorrect data. Both of which were still issues with Python 2. There's a valid architectural argument to be had between the Python 2 way, where str was a bag of bytes, and the unicode type was for decoded bytes, and the Python 3 way, where the bytes type is your bag of bytes and str holds your decoded string. I favor Python 3's way of doing it, but it's almost six of one, half a dozen of the other. The advantages of one over the other are slight, and given how many library functions relied on the old behavior, it was probably a mistake to change it like that, rather than continuing the Python 2 way, and fixing issues like those above that caused problems.

janto 5y ago

If the reactions seem bizarre, maybe you're missing something. For me it was more than "just the tech". Python 3.x was not the Python 3000 I was promised for a long time: many of the additions did not warrant breaking changes (as demonstrated by the backports) and many of Python's deeper warts still remain unaddressed, even today. I think the deeper issue is that at the rate that 2.x got popular, the original motivation, principles and process behind the language's development got diluted. The orchestration of the push for 3.x adoption was not technically motivated in a convincing manner and signaled a change in the process. I stopped trusting the process.

shoo 5y ago

I did a bit of contract work for a place with over a million lines of custom python 2.x scripts that ran inside of a python interpreter embedded in a proprietary product which made it difficult to do things like write and execute unit tests for the code. I think they were still writing new python 2.x code in 2018. A lot of the scripts were supporting processes that had a finite lifespan during the initial stages when the company infrastructure was being built out, rather than ongoing operational processes that needed to be performed indefinitely, so hopefully they'll be able to set a lot of that codebase on fire instead of maintaining or porting it to python 3.