8 ms·

Can we please stop complaining about Python 2to3 already. Feel free to use and maintain Python 2 indefinitely or move to a different language. Most of the rest

Can we please stop complaining about Python 2to3 already. Feel free to use and maintain Python 2 indefinitely or move to a different language. Most of the rest of us will move on since the conversions were not that hard and the improvements worth it and we were given 10+ years to do it. Python is a fantastically productive language in many contexts. The maintainers would probably do some things different with the 2to3 conversion if they had to do it over again. But whatever mistakes were made weren't malicious and they were learning as they went just like the rest of us. I greatly appreciate their efforts to fix the things that were broken, regardless of the hassle involved in upgrading.

jnxx 5y ago
> Most of the rest of us will move on since the conversions were not that hard and the improvements worth it and we were given 10+ years to do it. You are describing it as if it is for everyone easy to convert. That might be true for fast-paced start-ups and web companies which rewrite their code every few years. I work in science, in projects which have complex infrastructure and are developed in the course of many years. There is a vast amount of code which is written in the language and virtually nobody has time to rewrite and convert it. Me neither. As a consequence, Python is not really ideal for this kind of work, and I will tend to avoid it in my next job. It might not seem to matter to you, but a good part of Python's success is actually because of things like Numpy, Scientific Python, and statistics and science libaries which were written by scientists. Some, like Konrad Hinsen (one of the authors of Numpy) are now even considering to move to another language: http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/ http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stabilit... The idea that converting is easy is also not generally true for any kind of infrastructure. Consider the Python Wiki: https://wiki.python.org/moin/ https://wiki.python.org/moin/ At the bottom of the page, you find the note "MoinMoin powered". The Wiki software is the MoinMoin wiki software. It is the only large wiki software with multi-markup support written in Python. It has so far no Python 3 port. This is because people have written it in their spare time, and they have no time to rewrite it. Do you have the time? Would you volunteer for that? From the point of a naïve language author or a library author, change is part of life and users of their software should just upgrade to the latest version, as you write. But this is not friendly to the users of the software. Users really have other things to do than to constantly rewrite their software according to your latest syntax change or broken dependency. More proficient languages and infrastructure projects, like C, C++, Common Lisp, Java, and the Linux Kernel, do not have this approach - they maintain strict backwards compatibility. "WE DO NOT BREAK USER SPACE", in all caps - do you recognize that phrase? You can google for it. And this rule has a reason. You can think a few minutes how successful that project would have been without this rule. What if if you want to run a new Python on Linux, you first need to upgrade your Linux kernel, and then you need to upgrade your Linux distribution, and then you find that the new distribution release does not support HTML1, so the library you were going to use is broken, and, oh, your printer won't work with it? This is more or less what Python expects from its users. The Python developers do not adhere to this - they break backwards compatibility. If one reads the fine print about the release cycle, it says clearly that there might be breaking changes even in minor releases. At the same time, the Python developers did not want people to realize that Python 3 is not the same language, and used the same file extension for the new language. This has as an effect more backwards incompatibility, for example there are libraries which continue to support Python 2 for some time and then stop that support - without using a major version number increment. As a consequence, Python 2 projects which use such a library will break if using normal tools, which assume that semantic versioning is adhered to. In other words, one cannot rely on that Python library authors adhere to semantic versioning. All this is, regardless how much Python developers fuss about it, a choice. It would have been possible to keep Python 3 backwards compatible without breaking anything. Common Lisp has done it and it is decades older. But this has not happened and as a consequence, Python is less suitable for some types of projects and infrastructure. That's not something to complain about, and I don't. It is just a fact.
busterarm 5y ago
> Feel free to use and maintain Python 2 indefinitely or move to a different language. Most of the rest of us will move on since the conversions were not that hard and the improvements worth it and we were given 10+ years to do it. Maintaining Python 2 indefinitely is an option actively being taken away from people. More importantly, fuck anyone who wants to direct my budget! There's things written in Python 2 ages ago that don't justify additional developer time in my company. Except the community is making us do it. If you think migrating some ancient Django app to modern Django is trivially easy, why don't you come do it for us?
- jefftk 5y ago
  > Maintaining Python 2 indefinitely is an option actively being taken away from people Where do you see this happening? All I see are people and organizations deciding that they don't want to do the work of maintaining Python 2.
  - busterarm 5y ago
    Python 2 removed from homebrew. Python 2.7 is not getting any security/crypto updates backported. Eventually PyPI's SSL configuration will change to a point where Python 2.7 won't handle it and then that's it. Game over, no more pip for you, even if you froze on < 21.0. There are Linux distros out there in LTS until late into this decade that shipped with Python 2.7. Ubuntu 18 comes to mind, which EOLs in 2028. They've got a real hard problem to solve.
    - dagw 5y ago
      Python 2.7 is not getting any security/crypto updates backported. There are several projects out there that are supporting python 2.7 with security updates. Tauthon is probably the biggest one.
      - busterarm 5y ago
        Thank you. Yes it is good to see some projects stepping up the fill the gaps. I only wonder as time goes on how well socialized these will be...and if people who suddenly find themselves in the position of maintaining an old Python code base will be able to find out what to do.
dekhn 5y ago
The problem is, they are calling it a mistake but they were very well informed of the consequences of doing that they did. Everybody said "just fix integer division and print, finish the migration then start adding new features", and they went ahead and totally ignored that.
- sophacles 5y ago
  Whenever I work on a project that gathers interest in multiple places I get input like: * If you do X you'll fail * If you don't do X you'll fail * X is neat but don't worry about it til later. I imagine that the people in charge of python, a wildly popular programming language (at the time, and still), got all sorts of input, much of which conflicted. They chose a path and it was a bad choice, a mistake. This is clear in retrospect 13 years later, but at pycon in 2008 (a few months before python 3 officially released) the viewpoint was more mixed - a lot of people thought it was a good path and a lot of people thought it was terrible and others still thought the transition plan was annoying but would be cleared up pretty quick.
  - Blikkentrekker 5y ago
    It's not so much the issue that it is clear that it was a bad decision. — they admit it, owe up to it, and I do not blame them for making the mistake and admitting it was unfortunate and that they learned their lessons. What drives me to ire to no end are the many people that defend it on message boards with “Just upgrade; you had ten years.”, clearly not realizing the risks and money involved in such an endeavor. It's as though the government suddenly start putting something in the water that improves it's health benefits, but will erode contemporary plumping pipes away in ten years and people say. “Just break down your walls and replace your plumbing; you have ten years.”. — it is quite expensive, and risks that my house fall over.
  - nindalf 5y ago
    This is a reasonable take, but think about this from the perspective of engineers who've never made any mistakes in their lives. It's hard for them to fathom how anyone else can make a mistake like the Python devs did. The person you're answering clearly had the foresight to know what was the right thing to do, why can't everyone?
otabdeveloper4 5y ago
> and the improvements worth it No. That's just the Stockholm Syndrome talking.
Blikkentrekker 5y ago
> Can we please stop complaining about Python 2to3 already. Feel free to use and maintain Python 2 indefinitely or move to a different language. Most of the rest of us will move on since the conversions were not that hard and the improvements worth it and we were given 10+ years to do it. Have you done such an upgrade with a million line codebase? No company wants to do that with such a large codebase with the potential of regressions that can cost quite a bit of money. Reading this article makes it sound as though the experience taught him a valuable lesson and that he agrees that in retrospect Python 3 was a mistake, and it was. There is quite a good reason why so many platforms and languages take indefinite backwards compatibility quit seriously. — it is simply a very demanding thing of large codebases to rewrite to a newer version of the language with all the potential of subtle regressions. > The maintainers would probably do some things different with the 2to3 conversion if they had to do it over again. The article suggests that if they could go back in time, they would not do it and make no backwards incompatible change to Python 2. > But whatever mistakes were made weren't malicious and they were learning as they went just like the rest of us. I greatly appreciate their efforts to fix the things that were broken, regardless of the hassle involved in upgrading. It was certainly not malicious, but it was indeed a bad idea which is what the complaining is: that Python 3, or any such backwards incompatible changes should have never happened. This is why Rust uses editions and many other languages a system where backwards incompatible changes must explicitly be imported and the old behavior continues to be supported indefinitely.
- misnome 5y ago
  > Have you done such an upgrade with a million line codebase? Yes. With a transition plan everything was ratcheted over to python3 syntax, then test compatibility. Now developers no longer have to remember the nuance between python 2 and 3 and how to write compatibility layers for both. Now we no longer need to worry about maintaining or working around issues in dependencies that haven't been updated in a decade, and we don't need to teach python 2.7 to new developers.
  - Blikkentrekker 5y ago
    > Yes. With a transition plan everything was ratcheted over to python3 syntax, then test compatibility. Now developers no longer have to remember the nuance between python 2 and 3 and how to write compatibility layers for both. Then tell me, what were the costs of the transition when completed? and what where the projected costs when starting? > Now we no longer need to worry about maintaining or working around issues in dependencies that haven't been updated in a decade, and we don't need to teach python 2.7 to new developers. You wouldn't if Python 3 didn't exist either.
mkl95 5y ago
I refused to get involved in 2to3 migration projects for years. I recently took care of a 2to3 migration involving a popular web framework and a few technologies for one of my employer's clients. The codebase is large and convoluted but I still found it surprisingly easier and way more interesting than I expected. There is some actual difficulty in dealing with packages that have deprecated for years, and other gory details. But these days 2to3 drama is mostly a meme. The Python community, with a few exceptions, has done an amazing job at keeping things working.
- stuaxo 5y ago
  I've done this for a few upstream projects. If you want have a bug you want attention on in an upstream project, the easiest way to get it is to do a task for them first, and doing python 2 to 3 is a usually a pretty straightforward one.
sackerhews 5y ago
I agree with what you say, but: > The maintainers would probably do some things different with the 2to3 conversion if they had to do it over again. Saying that "probably do some things different" is quite an understatement. Python 2 to 3 was an articulated train collision in slow motion that spans a decade. I'm sorry to say that, but it was monumentally awful. I really, really wish that it didn't happen because I think it kept Python back at least 5 years. People got really burnt by this upgrade process, and the answer was "we said we'd do backwards incompatible changes". "And what, are we going to have this every few years?" one might have though. So, it isn't enough to tell people to "stop complaining about Python 2to3" and things might be done differently in the future or past.
- nindalf 5y ago
  > it kept Python back at least 5 years. Yeah, it's done alright for itself hasn't it? Look at the stats, it's a top #2 language on Github, StackOverflow questions, Developer surveys or any other measure. Complaining that it could have achieved this 5 years earlier, I don't see the point. We can just appreciate where it's at now, and the people who were instrumental in getting us here.
- Zababa 5y ago
  > "And what, are we going to have this every few years?" Isn't this the current situation now? Granted, not as big as Python 2 to 3, but there are backwards incompatible changes often.
alerighi 5y ago
The migration could have happened more smoothly if they had let both python2 and python3 code coexist in the same project. The problem was forking the language like they did. Why couldn't they make the same interpreter run both python3 and python2 code, for a transition period, deprecating python2 features gradually? So you could have had legacy python2 code and new code written in python3. The low level implementation is not that different to require that big break. The only major difference was on string types, but that could have been handled. Breaking stuff as they did made no sense. And the problem is that after 10+ years that python3 came out in most Linux distribution if you type python you get python2, and it will probably ever will for maintaining compatibility with scripts that started with `#!/usr/bin/python` expecting it to be python2 and not 3. That is a problem for everyone, for maintainers of the distributions, for developer and sysadmins that cannot assume that the default version of python is 3 (you don't know how many time I wated debugging programs written by people that use Arch or whatever fancy distribution and assumed that /usr/bin/python is python3, and then the script breaks on the Debian production server...) At least since they made a new language, they could invented a new extension for the files, so the `/usr/bin/python` executable could have detected the version to use based on the extension (launch .py3 files with python3, and .py files with python2).
- apple4ever 5y ago
  They should've absolutely maintained a compatibility layer in Python3 for a couple versions (say until Python 3.5?) instead of splitting python2 and python3. Something similar to Perl's "use" feature flag.

jnxx 5y ago

> Most of the rest of us will move on since the conversions were not that hard and the improvements worth it and we were given 10+ years to do it. You are describing it as if it is for everyone easy to convert. That might be true for fast-paced start-ups and web companies which rewrite their code every few years. I work in science, in projects which have complex infrastructure and are developed in the course of many years. There is a vast amount of code which is written in the language and virtually nobody has time to rewrite and convert it. Me neither. As a consequence, Python is not really ideal for this kind of work, and I will tend to avoid it in my next job. It might not seem to matter to you, but a good part of Python's success is actually because of things like Numpy, Scientific Python, and statistics and science libaries which were written by scientists. Some, like Konrad Hinsen (one of the authors of Numpy) are now even considering to move to another language: http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/ http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stabilit... The idea that converting is easy is also not generally true for any kind of infrastructure. Consider the Python Wiki: https://wiki.python.org/moin/ https://wiki.python.org/moin/ At the bottom of the page, you find the note "MoinMoin powered". The Wiki software is the MoinMoin wiki software. It is the only large wiki software with multi-markup support written in Python. It has so far no Python 3 port. This is because people have written it in their spare time, and they have no time to rewrite it. Do you have the time? Would you volunteer for that? From the point of a naïve language author or a library author, change is part of life and users of their software should just upgrade to the latest version, as you write. But this is not friendly to the users of the software. Users really have other things to do than to constantly rewrite their software according to your latest syntax change or broken dependency. More proficient languages and infrastructure projects, like C, C++, Common Lisp, Java, and the Linux Kernel, do not have this approach - they maintain strict backwards compatibility. "WE DO NOT BREAK USER SPACE", in all caps - do you recognize that phrase? You can google for it. And this rule has a reason. You can think a few minutes how successful that project would have been without this rule. What if if you want to run a new Python on Linux, you first need to upgrade your Linux kernel, and then you need to upgrade your Linux distribution, and then you find that the new distribution release does not support HTML1, so the library you were going to use is broken, and, oh, your printer won't work with it? This is more or less what Python expects from its users. The Python developers do not adhere to this - they break backwards compatibility. If one reads the fine print about the release cycle, it says clearly that there might be breaking changes even in minor releases. At the same time, the Python developers did not want people to realize that Python 3 is not the same language, and used the same file extension for the new language. This has as an effect more backwards incompatibility, for example there are libraries which continue to support Python 2 for some time and then stop that support - without using a major version number increment. As a consequence, Python 2 projects which use such a library will break if using normal tools, which assume that semantic versioning is adhered to. In other words, one cannot rely on that Python library authors adhere to semantic versioning. All this is, regardless how much Python developers fuss about it, a choice. It would have been possible to keep Python 3 backwards compatible without breaking anything. Common Lisp has done it and it is decades older. But this has not happened and as a consequence, Python is less suitable for some types of projects and infrastructure. That's not something to complain about, and I don't. It is just a fact.

busterarm 5y ago

> Feel free to use and maintain Python 2 indefinitely or move to a different language. Most of the rest of us will move on since the conversions were not that hard and the improvements worth it and we were given 10+ years to do it. Maintaining Python 2 indefinitely is an option actively being taken away from people. More importantly, fuck anyone who wants to direct my budget! There's things written in Python 2 ages ago that don't justify additional developer time in my company. Except the community is making us do it. If you think migrating some ancient Django app to modern Django is trivially easy, why don't you come do it for us?

jefftk 5y ago
> Maintaining Python 2 indefinitely is an option actively being taken away from people Where do you see this happening? All I see are people and organizations deciding that they don't want to do the work of maintaining Python 2.
- busterarm 5y ago
  Python 2 removed from homebrew. Python 2.7 is not getting any security/crypto updates backported. Eventually PyPI's SSL configuration will change to a point where Python 2.7 won't handle it and then that's it. Game over, no more pip for you, even if you froze on < 21.0. There are Linux distros out there in LTS until late into this decade that shipped with Python 2.7. Ubuntu 18 comes to mind, which EOLs in 2028. They've got a real hard problem to solve.
  - dagw 5y ago
    Python 2.7 is not getting any security/crypto updates backported. There are several projects out there that are supporting python 2.7 with security updates. Tauthon is probably the biggest one.
    - busterarm 5y ago
      Thank you. Yes it is good to see some projects stepping up the fill the gaps. I only wonder as time goes on how well socialized these will be...and if people who suddenly find themselves in the position of maintaining an old Python code base will be able to find out what to do.

dekhn 5y ago

The problem is, they are calling it a mistake but they were very well informed of the consequences of doing that they did. Everybody said "just fix integer division and print, finish the migration then start adding new features", and they went ahead and totally ignored that.

sophacles 5y ago
Whenever I work on a project that gathers interest in multiple places I get input like: * If you do X you'll fail * If you don't do X you'll fail * X is neat but don't worry about it til later. I imagine that the people in charge of python, a wildly popular programming language (at the time, and still), got all sorts of input, much of which conflicted. They chose a path and it was a bad choice, a mistake. This is clear in retrospect 13 years later, but at pycon in 2008 (a few months before python 3 officially released) the viewpoint was more mixed - a lot of people thought it was a good path and a lot of people thought it was terrible and others still thought the transition plan was annoying but would be cleared up pretty quick.
- Blikkentrekker 5y ago
  It's not so much the issue that it is clear that it was a bad decision. — they admit it, owe up to it, and I do not blame them for making the mistake and admitting it was unfortunate and that they learned their lessons. What drives me to ire to no end are the many people that defend it on message boards with “Just upgrade; you had ten years.”, clearly not realizing the risks and money involved in such an endeavor. It's as though the government suddenly start putting something in the water that improves it's health benefits, but will erode contemporary plumping pipes away in ten years and people say. “Just break down your walls and replace your plumbing; you have ten years.”. — it is quite expensive, and risks that my house fall over.
- nindalf 5y ago
  This is a reasonable take, but think about this from the perspective of engineers who've never made any mistakes in their lives. It's hard for them to fathom how anyone else can make a mistake like the Python devs did. The person you're answering clearly had the foresight to know what was the right thing to do, why can't everyone?

otabdeveloper4 5y ago

> and the improvements worth it No. That's just the Stockholm Syndrome talking.

Blikkentrekker 5y ago

> Can we please stop complaining about Python 2to3 already. Feel free to use and maintain Python 2 indefinitely or move to a different language. Most of the rest of us will move on since the conversions were not that hard and the improvements worth it and we were given 10+ years to do it. Have you done such an upgrade with a million line codebase? No company wants to do that with such a large codebase with the potential of regressions that can cost quite a bit of money. Reading this article makes it sound as though the experience taught him a valuable lesson and that he agrees that in retrospect Python 3 was a mistake, and it was. There is quite a good reason why so many platforms and languages take indefinite backwards compatibility quit seriously. — it is simply a very demanding thing of large codebases to rewrite to a newer version of the language with all the potential of subtle regressions. > The maintainers would probably do some things different with the 2to3 conversion if they had to do it over again. The article suggests that if they could go back in time, they would not do it and make no backwards incompatible change to Python 2. > But whatever mistakes were made weren't malicious and they were learning as they went just like the rest of us. I greatly appreciate their efforts to fix the things that were broken, regardless of the hassle involved in upgrading. It was certainly not malicious, but it was indeed a bad idea which is what the complaining is: that Python 3, or any such backwards incompatible changes should have never happened. This is why Rust uses editions and many other languages a system where backwards incompatible changes must explicitly be imported and the old behavior continues to be supported indefinitely.

misnome 5y ago
> Have you done such an upgrade with a million line codebase? Yes. With a transition plan everything was ratcheted over to python3 syntax, then test compatibility. Now developers no longer have to remember the nuance between python 2 and 3 and how to write compatibility layers for both. Now we no longer need to worry about maintaining or working around issues in dependencies that haven't been updated in a decade, and we don't need to teach python 2.7 to new developers.
- Blikkentrekker 5y ago
  > Yes. With a transition plan everything was ratcheted over to python3 syntax, then test compatibility. Now developers no longer have to remember the nuance between python 2 and 3 and how to write compatibility layers for both. Then tell me, what were the costs of the transition when completed? and what where the projected costs when starting? > Now we no longer need to worry about maintaining or working around issues in dependencies that haven't been updated in a decade, and we don't need to teach python 2.7 to new developers. You wouldn't if Python 3 didn't exist either.

mkl95 5y ago

I refused to get involved in 2to3 migration projects for years. I recently took care of a 2to3 migration involving a popular web framework and a few technologies for one of my employer's clients. The codebase is large and convoluted but I still found it surprisingly easier and way more interesting than I expected. There is some actual difficulty in dealing with packages that have deprecated for years, and other gory details. But these days 2to3 drama is mostly a meme. The Python community, with a few exceptions, has done an amazing job at keeping things working.

stuaxo 5y ago
I've done this for a few upstream projects. If you want have a bug you want attention on in an upstream project, the easiest way to get it is to do a task for them first, and doing python 2 to 3 is a usually a pretty straightforward one.

sackerhews 5y ago

I agree with what you say, but: > The maintainers would probably do some things different with the 2to3 conversion if they had to do it over again. Saying that "probably do some things different" is quite an understatement. Python 2 to 3 was an articulated train collision in slow motion that spans a decade. I'm sorry to say that, but it was monumentally awful. I really, really wish that it didn't happen because I think it kept Python back at least 5 years. People got really burnt by this upgrade process, and the answer was "we said we'd do backwards incompatible changes". "And what, are we going to have this every few years?" one might have though. So, it isn't enough to tell people to "stop complaining about Python 2to3" and things might be done differently in the future or past.

nindalf 5y ago
> it kept Python back at least 5 years. Yeah, it's done alright for itself hasn't it? Look at the stats, it's a top #2 language on Github, StackOverflow questions, Developer surveys or any other measure. Complaining that it could have achieved this 5 years earlier, I don't see the point. We can just appreciate where it's at now, and the people who were instrumental in getting us here.
Zababa 5y ago
> "And what, are we going to have this every few years?" Isn't this the current situation now? Granted, not as big as Python 2 to 3, but there are backwards incompatible changes often.

alerighi 5y ago

The migration could have happened more smoothly if they had let both python2 and python3 code coexist in the same project. The problem was forking the language like they did. Why couldn't they make the same interpreter run both python3 and python2 code, for a transition period, deprecating python2 features gradually? So you could have had legacy python2 code and new code written in python3. The low level implementation is not that different to require that big break. The only major difference was on string types, but that could have been handled. Breaking stuff as they did made no sense. And the problem is that after 10+ years that python3 came out in most Linux distribution if you type python you get python2, and it will probably ever will for maintaining compatibility with scripts that started with `#!/usr/bin/python` expecting it to be python2 and not 3. That is a problem for everyone, for maintainers of the distributions, for developer and sysadmins that cannot assume that the default version of python is 3 (you don't know how many time I wated debugging programs written by people that use Arch or whatever fancy distribution and assumed that /usr/bin/python is python3, and then the script breaks on the Debian production server...) At least since they made a new language, they could invented a new extension for the files, so the `/usr/bin/python` executable could have detected the version to use based on the extension (launch .py3 files with python3, and .py files with python2).

apple4ever 5y ago
They should've absolutely maintained a compatibility layer in Python3 for a couple versions (say until Python 3.5?) instead of splitting python2 and python3. Something similar to Perl's "use" feature flag.